Home » Tech » Archive for category 'Linux'

Linux Archive

CLI image conversion JPG to PNG

Posted 2014/08/16 By mikep345678

Using ImageMagick (on Linux) to convert all jpgs in a folder to png, with alpha layer, with longest side 400 pixels:

for f in *.jpg; do convert -verbose -resize "400x400>" -alpha on "$f" "${f/%jpg/png}"; done


Michael Battista
(originally from http://www.ccs.neu.edu/home/battista/articles/winbind/winbind.pdf)

1. Introduction

I recently was assigned the task of joining a Debian workstation to a Windows Server 2003 Active Directory domain. Though most of the documentation I read for doing this was rather straightforward, 100% of that documentation turned out to be only 85% useful. While all were easy to follow, at the end of the day I found myself piecing together bits of information from all different sources. Only after deciphering cryptic log messages, consulting my colleagues, and experimenting did I finally have this working. Here I try to provide a complete walkthrough of this procedure, paying close attention to the often overlooked details I encountered in my trials.

The procedure described in this article was performed on a Debian-based system using a Windows Server 2003 domain controller (PDC). Therefore, some of the details in this article (package names, file locations, commands) mentioned here may be inconsistent with those of your distribution. Consult your system’s documentation for these details.

2. Install the Software

You will need to install packages for MIT Kerberos, Samba, Winbind, and NTP. The following command should suffice:[1]

sudo apt-get install libkrb53 krb5-config samba winbind ntpdate ntp-server

Just accept the default settings during package installation. We will be configuring these services later.

3. Stop the Services

To avoid unnecessary complications, stop the Samba, Winbind, and NTP services.

sudo /etc/init.d/samba stop
sudo /etc/init.d/winbind stop
sudo /etc/init.d/ntp-server stop

4. Configure Kerberos

Active Directory uses the Kerberos protocol for service requests. To configure your workstation as a Kerberos client, you will need to modify /etc/krb5.conf.

The first thing you will need to configure is the Kerberos realm of your domain. For all intensive purposes, this merely means pointing the Kerberos libraries to your primary domain controller. In /etc/krb5.conf, there will be a section named [realms]. Add a subsection like the following:[2]

kdc = pdc_ip_address

in all uppercase. If your domain name is AD.domain, for instance, you would replace REALMNAME above with AD.DOMAIN.

pdc_ip_address is the IP address of your primary domain controller. Heed this warning from Samba-3 By Example (http://www.samba.org/samba/docs/man/Samba-Guide/DomApps.html#id2585260):

Kerberos needs to be able to do a reverse DNS lookup on the IP address of your KDC. Also, the name that this reverse lookup maps to must either be the NetBIOS name of the KDC (i.e., the hostname with no domain attached) or the NetBIOS name followed by the realm. If all else fails, you can add a /etc/hosts entry mapping the IP address of your KDC to its NetBIOS name. If Kerberos cannot do this reverse lookup, you will get a local error when you try to join the realm.

If the output of host pdc_ip_address does not contain the correct DNS name, reconfigure hostname resolution before moving on.

With your realm in place, navigate to the [libdefaults] section in the same file and set the default Kerberos realm to REALMNAME:

default_realm = REALMNAME

5. Configure NTP

The Kerberos protocol relies heavily on timestamps. If the clock on the Debian workstation is out of sync with the primary domain controller, things will break. Windows Server 2003 by default broadcasts its time via the Network Time Protocol (NTP). To synchronize your clock with the primary domain controller, try the following:[3]

sudo ntpdate pdc_ip_address

If you receive an “NTP socket in use” error, you need to stop the NTP daemon (sudo /etc/init.d/ntp-server stop) and try again. If ntpdate still fails, chances are that either the Windows Time Service is not running or one or more firewalls between the workstation and the domain controller are blocking port 123/UDP. Start the service and/or create exceptions for this port and try again.

When successful, ntpdate synchronizes your clock enough to start the NTP daemon, which handles all further synchronization. To point this daemon at the Windows domain controller, locate the first uncommented line in /etc/ntp.conf beginning with server and make the following change:

server pdc_ip_address

With this setting in place, restart the NTP daemon:

sudo /etc/init.d/ntp-server start

To confirm that your workstation is contacting the primary domain controller for time updates, run ntpq -p. If everything is configured correctly, you should see your primary domain controller’s IP address or DNS name in the list of time servers.

6. Configure Winbind

The Winbind service is the engine of this operation. It handles all communication with the Active Directory domain controller and manages the Windows-to-Unix translations that must occur.

You configure this service in /etc/samba/smb.conf. The following lines should be added to its global section:

workgroup = DOMAINNAME
security = ads
idmap uid = 10000-20000
idmap gid = 10000-20000
template shell = /bin/bash
template homedir = /home/%D/%U
winbind use default domain = yes

Replace REALMNAME with the name of the Kerberos realm you configured in the previous section. DOMAINNAME is the shortened version of the domain name. The idmap lines list the range of ID’s that Winbind will use when translating Windows users and groups to Unix.

Template shell and template homedir list the user’s login shell and home directory. The %D translates to the workgroup (DOMAINNAME) while the %U translates to the current user. Of course, you could leave out the %D and choose a more traditional location, such as /home/%U, if you wish. Just make sure that the parent directory of your Winbind home directories (in our case /home/%D) exists before attempting to login as a Windows user.[4]

The winbind use default domain option modifies the representation of Windows usernames. By default, Windows users must login by prefixing their username with workgroup followed by a ’\’ (DOMAINNAME\battista). As a convenience for users, you can set winbind use default domain to yes so that they no longer need to include this prefix. Just be wary of conflicts with existing local accounts.

7. Configure Nsswitch

Your system uses /etc/nsswitch.conf to determine where it should look to resolve various types of lookups. To resolve users and groups from Active Directory, add a reference to the Winbind name service module in the passwd and group lines. Below is the relevant portion of /etc/nsswitch.conf:

files winbind
files winbind

The order in which these modules are listed reflects the order in which they will be used. With the above configuration, our system will first query files on the local drive (/etc/passwd and /etc/group) before querying Active Directory. Other modules, such as nis or ldap, may be listed in this file. Just make sure that you add the appropriate winbind entries to passwd and group. To activate these changes run the following command:

sudo ldconfig

This registers the winbind module with the system so that resolution can take place.

8. Join the Domain

With Kerberos and Winbind configured, you’re now ready to join your Debian workstation to the Windows Active Directory domain.

Before you do, check the hostname of your Debian workstation. Due to restrictions in the NetBIOS protocol, the hostname must contain no more than 15 characters. If you see a STATUS_BUFFER_OVERFLOW message in the winbind log, odds are the hostname is invalid. Now would also be a good time to clear whatever cache files, if any, Winbind had previously generated. The Winbind cache is located in /var/lib/samba/. Backup this directory to /var/lib/samba.bak/ and delete all the files in the original.

With these housekeeping items taken care of, try joining the domain:[5]

sudo net ads join -U "DOMAINADMIN"

Replace DOMAINADMIN with the name of a user that has privileges to add computers to the domain. If all goes well, you should receive a short message stating that you have successfully joined the domain[6]. If this step fails, chances are that a firewall is restricting access to one or more of the following ports: 88/TCP, 88/UDP, 389/TCP, 464/UDP. Create exceptions for these ports and try again.

9. Start the Services

Now that our workstation is joined to the domain, restart the Samba and Winbind services.

sudo /etc/init.d/samba start

sudo /etc/init.d/winbind start

10. Check Name Resolution

At this point, you should be able to resolve users and groups from the Windows Active Directory domain using getent passwd and getent group. If these commands don’t display your Windows accounts, try to resolve them using wbinfo -u and wbinfo -g. These commands query the Winbind service directly, bypassing the name service switch. If you can resolve users and groups withwbinfo, go back and make sure you configured /etc/nsswitch.conf properly.

If resolution still fails, check if nscd is running. According to the Samba-3 Howto, “if nscd is running on the UNIX/Linux system, then even though NSSWITCH is correctly configured, it will not be possible to resolve domain users and groups for file and directory controls.” Stop this service and retry the steps mentioned above. If nscd is not running, and you are still having problems with resolution, make sure that ports 139/TCP and 445/TCP on the primary domain controller are not blocked by a firewall. Once your workstation is joined to the domain, Winbind uses these ports to perform its lookups.

Also, look out for the following situation. getent passwd lists all of the Active Directory accounts, but getent passwd user fails. Running id user also fails for Active Directory users. If you run into this problem, make sure that the hostname of the workstation you are configuring is unique within the Active Directory domain. If /var/log/samba/log.nmbd reports that nmbd was unable to register the hostname, choose a unique hostname for your workstation and rejoin the domain.

11. Configure PAM

Name resolution is great, but in order to use the Windows accounts for authentication, you need to configure PAM. The Pluggable Authentication Module subsystem (PAM) provides a layer of abstraction for applications that require authentication. Essentially, an application can authenticate against many different backends without having to know anything about the underlying protocols. A PAM-aware application simply requests authentication and then trusts PAM to retrieve it. This layer of abstraction is achieved via PAM modules.

To allow applications to authenticate against Active Directory, you need to register the pam_winbind module with the PAM system. With winbind properly configured, this module provides the logic necessary to retrieve authentication information from Active Directory. Three files in the PAM configuration directory are of interest to us: /etc/pam.d/common-account, /etc/pam.d/common-auth, and /etc/pam.d/common-session. These files merely represent three different stages in the authentication process. Insert a reference to pam_winbind.so in these files:

# /etc/pam.d/common-account
account sufficient pam_winbind.so
account required pam_unix.so
# /etc/pam.d/common-auth
auth sufficient pam_winbind.so
auth required pam_unix.so use_first_pass
# /etc/pam.d/common-session
session sufficient pam_winbind.so
session required pam_unix.so

Normally, PAM will grant authentication only if all of its modules succeed. By changing the priority of a module from required to sufficient, you instruct PAM to grant authentication as soon as that particular module succeeds. In our configuration, PAM won’t bother to check the local files if the user in question resides in Active Directory. This is the desired behavior.

Notice, too, that we added the use_first_pass parameter to modules following pam_winbind in /etc/pam.d/common-auth. In the event that we are logging in as a local user, PAM will check with Active Directory and fail before moving to the next module. Rather than prompt for another password, we tell the pam_unix module to use the password that was previously entered.

Before we can start using our Active Directory accounts, one last change must be made to our PAM configuration. We need to configure our system to create home directories for each of the Active Directory users. Insert a reference to the pam_mkhomedir module in /etc/pam.d/common-session as shown below:

# /etc/pam.d/common-session
session required pam_mkhomedir.so skel=/etc/skel/ umask=0022
session sufficient pam_winbind.so
session required pam_unix.so

When an Active Directory user logs in to our system for the first time, the pam_mkhomedir module will create his home directory, populate it with files from /etc/skel/, and set file permissions according to the umask 0022. This saves us the hassle of having to create each user’s home directory ourselves. Make sure to set the priority of this module to required, as shown above, to prevent a user from logging in should creation of his home directory fail.

12. Conclusion

With Kerberos, Winbind, Nsswitch, and PAM configured, you should now be able to login and access services on your Debian workstation using your Active Directory accounts. Each Active Directory user will have a local home directory on the Debian system, and you will be able to manage your system as if these users were native Unix accounts.

Though sufficient for a network where users typically use only one workstation, this setup is inadequate for an enterprise environment. By default, the Winbind Windows-to-Unix ID mappings are not kept consistent across client workstations. If you were to export user home directories via NFS, as is common in large networks, you would quickly run into trouble.

A more robust solution, better suited for an enterprise environment, would involve keeping the ID mappings consistent across all workstations and automounting shared home directories at login. My next article will present one way of achieving this.

Regardless of which setup you choose, you are now well on your way to achieving a streamlined, heterogenous network of Windows and Linux workstations.


1. Note that these package names are Debian-specific. Look for similarly named packages for your particular package management system.

2. To find the name of your domain, open Control Panel -> Administrative Tools -> Active Directory Domains and Trusts on your primary domain controller. Your domain name will be listed in the side panel.

3. The timezone on your Debian workstation must be correct for the synchronization to be effective. If ntpdate is successful, but the time seems ahead or behind, run tzconfig to select the appropriate timezone.

4. If it does not, the system will be unable to create home directories for the Windows users, which will prevent them from logging in.

5. Many of the articles I read about this procedure suggested to run kinit before joining the domain. I found that this step is unnecessary when net ads is specified.

6. This message may be accompanied by Kerberos preauthentication errors. These may be safely ignored.

Unix/Linux/OSX scripting

Posted 2012/08/18 By mikep345678

{stolen from here…}

Unix Shell Stuff


$# - Contains the number of arguments passed to the 
      program in the form of positional variables.

$* - Contains all the positional parameters passed to 
      the program.

$1-$9 - Positional parameters.

$0 - Contains the program name.

$$ - Contains the process ID of the current process.

$? - Contains the exit status of the last command executed. 

expr - Used to evaluate arguments as mathematical 
        expressions.  ex: CNT=`expr $CNT + 1` 

; - Used to separate commands on command line. 

\ - Used to continue the command on the next line.

\c - Used to suppress newline.
      ex: echo "Enter Name:\c" ; read name

read - Assigns the value entered into the given variable.
         ex: echo "Enter Name:"  
             read name

` ` - A command within grave accent marks returns 
       the output to the variable ex: DATE=`date`.

set - Set the values of the command line argument 
        variables ($1-$9) to it's arguments.

--   - No more options follow.

if expressions:

   :     Null Character (always true)

   ;     Separates commands on one line 

   #!    As the first characters in the first line of shell script,
           tells OS that what follows, is the path to the shell to execute this script with.

   exec  Does not start an new process for the command being execed.
            Over lays the current process instead of starting new one.  Does not return control to shell.

   break Exits "for", "while" and "until" loop.



Testing Character Data:

   =   Equals, string comparisons ex: if [ "$str1" = "str2" ]; then    (equal)

   !=  Not equal, string comparisons ex: if [ "$str1" != "str2" ]; then    (not equal)

   -n string - True if the length of string is greater than 0 (is not null)

   -z string - True if string is null (has a length of 0)

   test - Test strings ex: test "$str" = "abc"



Testing Numeric Data:

     -eq -  Equals                      ex:  if [ $1 -eq 0 ]; then
     -ne -  Not equal                   ex:  if [ $1 -ne 0 ]; then
     -gt -  Greater than                ex:  if [ $1 -gt 0 ]; then
     -ge -  Greater than of equal to    ex:  if [ $1 -ge 0 ]; then
     -lt -  Less than                   ex:  if [ $1 -lt 0 ]; then
     -le -  Less than or equal to       ex:  if [ $1 -le 0 ]; then

Testing For Files:

     True if
        -r    Read permission           ex:  if [ -r $FILE ]; then
        -w    Write permission          ex:  if [ -w $FILE ]; then
        -x    Execute permission        ex:  if [ -x $FILE ]; then
        -f    Regular file              ex:  if [ -f $FILE ]; then
        -d    Directory                 ex:  if [ -d $FILE ]; then
        -c    Character special file    ex:  if [ -c $FILE ]; then
        -b    Block special file        ex:  if [ -b $FILE ]; then
        -s    File not zero             ex:  if [ -s $FILE ]; then
        -t    Terminal device           ex:  if [ -t $FILE ]; then
        -o    True if either (OR)       ex:  if [ ! -r $FILE  -o ! -f $FILE ]; then
        -a    True if both (AND)        ex:  if [ -r $FILE -a -f $FILE ]; then

determine Linux directory sizes

Posted 2012/07/01 By mikep345678

Summarizes size of home folder excluding Dropbox, Downloads, Videos, Trash:
du -sh --exclude "Dropbox" --exclude "Downloads" --exclude "Videos" --exclude "Trash" ~/

Shows size of subfolders (1 deep) with same excludes:
du -h -d 1 --exclude "Dropbox" --exclude "Downloads" --exclude "Videos" --exclude "Trash" ~/

Using NetCat, dd and bzip to copy drives in Linux

Posted 2012/05/02 By mikep345678

Boot from PartedMagic on Source and Target machines.

On terminal of Target machine:

  • use ifconfig to determine ip address (targetip)
  • netcat –l –p 2525 | bzip2 –d | dd of=/dev/sda

On terminal of Source machine:

  • bzip2 -cvv /dev/sda | nc targetip 2525