A repository of useful commands for POSIX systems such as Unix and Linux.

Before, we go into more details about specific Linux or Unix commands, there is a really nice web site that tries to gather most common used commands on various Unix and Linux flavour in a table. It is then easy to find equivalences of commands between all those variants. The web site is called the Rosetta Stone for Unix.

This set of two commands allow to make a difference between two version of a complete file tree and apply it to another tree. This can be really useful let say you are downloading a Blog or Wiki software (version X) and installing it on your personal server. There are a few minor things that annoy you but seems to be OK for a vast majority of person. Therefore, you modify (tune) a few files of the software and use it as is (version Xm). A security update of the software is now available (version Y). If you update to version Y, you loose your change of version Xm. A good solution is to make a difference between version X and Y and apply those same changes to version Xm which become Ym then . The commands are:

$ diff -Naur blog-X blog-Y > patch-blog-X-Y
$ cd blog-Xm
$ patch -Np1 < ../patch-blog-X-Y

The gzip version (by replacing gzip by bzip2 and gz by bz2, you could increase the compression ratio):

$ diff -Naur blog-X blog-Y | gzip --best > patch-blog-X-Y.gz
$ cd blog-Xm
$ gzip -cd ../patch-blog-X-Y.gz | patch -Np1

The diff can also be done between two revision in CVS:

$ cvs rdiff -u -kk -r versionX -r versionY <module-name> > patch-blog-X-Y

• Upgrading to a new stable release

  $ gksudo "update-manager -c"

• Upgrading to the latest development snapshot available

  $ gksudo "update-manager -c -d"

Once your system has been upgraded, you may want to clean your list of installed packages from obsolete packages or residual data.

Here are a dump collection of useful commands with apt/deb system:

• apt-get build-dep inkscape: will retrieve the development libraries that are required to build inkscape
• apt-get update: fetch the latest list of applications from the repositories
• apt-get upgrade: upgrade your applications if some newer release are available in the repositories (once synchronised with apt-get update)
• apt-get dist-upgrade: same as before, but smart upgrade are performed (will explained that later, I'm not sure anymore what exactly smart is)
• apt-get source inkscape: get the source of inkscape in your current local directory, ready to be compiled and packaged.
• apt-get install inkscape: will install the binary package called inkscape.
• apt-get remove inkscape: will uninstall the binary package called inkscape (but possibly not all system configuration files)
• apt-get remove --purge inkscape: will uninstall the binary package called inkscape (and purge all possible files installed by the package)

• aptitude update: fetch the latest list of applications from the repositories
• aptitude safe-upgrade: upgrade your applications if some newer release are available in the repositories (once synchronised with apt-get update)
• aptitude full-upgrade: same as before, but smart upgrade are performed (will explained that later, I'm not sure anymore what exactly smart is)
• aptitude install inkscape: will install the binary package called inkscape.
• aptitude remove inkscape: will uninstall the binary package called inkscape.

• dpkg --get-selections: will list all installed packaged
• dpkg-reconfigure packagename: will reconfigure the package, useful to reconfigure some packages such as apache, mysql, xorg, etc.

• lsof +L1 -R
• Check in the list of files (rightmost column) that the files are under a system folder such as /usr

Synaptic offers many rich possibilities to manage your applications (both installed or available for installation). You can install, update, search for or remove applications. You can also clean your system from obsolete or old packages and even from residual data.

There is the generic rpm command.

• rpm -Uvh package.rpm: install a downloaded RPM package

Then on Fedora (up to release 21) and CentOS/RHEL, there is yum which allows to manage online repository of rpm packages.

• yum upgrade:
• yum install packages: install package(s) from the repositories
• yum remove packages: remove an installed package(s)
• yum list: List packages (installed and available)

Not always present/install are plugins for yum to check which services need to be restarted after an upgrade.

• yum ps based on the yum ps plugin;
• needs-restarting based on ;
• Without using yum, and it works for all Linux systems (whether using RPM or not)
  • lsof +L1 -R
  • Check in the list of files (rightmost column) that the files are under a system folder such as /usr

Fedora 22 and newer have replaced yum by dnf and it is possible that future releases of CentOS/RHEL will follow suit.

• dnf upgrade:
• dnf install packages: install package(s) from the repositories
• dnf remove packages: remove an installed package(s)
• dnf list: List packages (installed and available)

Red Hat-like systems (Red Hat, Fedora, Mandriva, Suse, etc.) offer the 'chkconfig' command to manage services/runlevel. You can configure at which runlevels the service is active or not. For example, activating ssh server only in runlevel 3 and 5 will be done like that:

$ sudo chkconfig --level 35 sshd on

The activation or not of a service will be taken into account at next restart unless you manually start and stop them using the /etc/init.d scripts or the service command.

$ sudo service sshd start

Here are the common parameters for the service (or init.d scripts): start, stop, restart, reload, status

Debian-like systems (incl. Ubuntu) do not provide the chkconfig command. They provide the 'update-rc.d' command that mimic the functionnality of the chkconfig one. They provide also another command, 'invoke-rc.d', that allows to restart, stop and start services.

You can find more resources on my delicious bookmarks about runlevel and services management for Ubuntu.

Now there are at least 2 new breads of system init scripts: systemd and upstart.
The former is the easier to use. Everything is cleanly accessible with one command systemctl:

$ sudo systemctl disable sshd
$ sudo systemctl enable sshd
$ sudo systemctl start sshd
$ sudo systemctl stop sshd

The latter, upstart, is using different commands and requires some manual file editing to do the same thing. The same thing as above can be achieved doing:

$ sudo bash -c 'echo "manual" >> /etc/init/sshd.override'
$ sudo rm -f /etc/init/sshd.override
$ sudo start sshd
$ sudo stop sshd

I am not going to compare technically the pro's and con's of upstart vs systemd here, but clearly for the user space tools to manipulate both systems, systemd has one entry point and that makes thing simpler IMHO.

In recent Linux distribution, the configuration of device does not rely only on device names. One can now make it rely on “label” or “uuid”. A “uuid” is a unique identifier for a device, it can be practical is the device name change often, because its uuid will always be the same.

To find out the UUID of a device, use the following command:

$ sudo vol_id -u <device>

One can replace “<device>” by a partition name like /dev/hda1 or /dev/sda5, etc.

The set of commands below will create a directory readable by root and users belonging to a group, everyone else can't read it. Only root can write into it.

# cd /etc
# mkdir nginx
# chown root:nginx nginx
# chmod 2750 nginx
# setfacl -d --set u::rwx,g::rx,o::- nginx

The two first line are obvious. The third line set the permission as rwx for the user, rx for the group and none for others. In addition the setgid bit (g+s or the '2' in '2750') is used to preserve the group ownership of the parent directory to newly create file under it. The last command (only works if your underlying file system supports ACL) set the default permission of rwx for users, rx for group and none for others.

With the above setup creating a file under this directory will result in preserving the ownership and rights of the parent directory:

# touch nginx/nginx.conf
# mkdir nginx/vhosts.d
# ls -ld nginx/*
-rw-r----- 1 root nginx 0 Feb 28 14:50 nginx/nginx.conf
drwxr-s--- 1 root nginx 0 Feb 28 14:50 nginx/vhosts.d

Simple usage to log in a remote host:

$ ssh [<username>@]<remote host name>

An equivalent (syntax compatible with rlogin):

$ ssh <remote host name> [-l <username>]

To execute a command without logging in first:

$ ssh [<username>@]<remote host name> <command>

However, is the command is a sudo one, the password will be written in clear. It is advise to avoid the form:

$ ssh [<username>@]<remote host name> sudo <cmd>

because it will show the password in clear.
It is better to use a TTY (basically a pseudo-terminal) like this:

$ ssh -t [<username>@]<remote host name> sudo <cmd>

or

$ ssh -o RequestTTY=yes [<username>@]<remote host name> sudo <cmd>

in this case, the password is invisible. Example:

$ ssh -t 192.168.47.107 sudo shutdown -h now

» Main article

Your goal is to access a service on a dedicated host with everything going through SSH. Why? On a remote machine, the only service accessible from the outside World is the SSH service, but you still want to access other services which are behind the firewall of this remote machine.

$ ssh -f -C -N -L <local port>:<remote host name>:<remote service port> [-p <ssh port>] [<username>@]<remote host outside name>

» Guide to Samba manual mount

Mounting a shared drive using smbfs (is deprecating…)

$ sudo apt-get install smbfs
$ sudo mkdir /mnt/share
$ sudo mount -t smbfs //win/Share\ Folder /mnt/share -o username="win\huygens"

It will request you a password. After this the SMB share folder is mapped to the directory /mnt/share. If you go exploring with Nautilus, you go to “File System” → “mnt” → “share” and then you can access and open all of your files.

To make it automatic, you could add a line in the file that manage partitions and disks (/etc/fstab).

//win/Share\040Folder /mnt/share smbfs credentials=/etc/smb-pw/win 0 0

A bit of explanation, the spaces in the share drives have to be replace by '\040', and the login and password are going to be place in a file that is private to the super user (not readable by others). Sample content for the credentials file:

username=win\huygens
password=my_1ncredible_passwd

» Guide to Samba manual mount

Similar to smbfs:

$ sudo apt-get install smbfs
$ sudo mkdir /mnt/share
$ sudo mount -t cifs //host2/share /mnt/share -o user=huygens,domain=lan

In fstab, I have written (added a new line at the end):

//host2/share /mnt/share cifs credentials=/etc/host2-cred,domain=lan 0 0

In /etc/host2-cred I have written:

username=huygens
password=huygens

The next instruction works best when using InnoDB engine at least.

$ mysqldump -uroot -p --single-transaction --databases <name-db1> <name-db2> | xz > backup.sql.xz

Main article

The following list is just a reminder for most used commands. Please see the main article about git for more comprehension.

• 'git status' to get the states of which files are staged (will be 'commit'), which files have been modified but not yet staged and which files aren't tracked
• 'git add <file|directory>' to add a list of files or directories to the staged status
• 'git rm <file|directory>' to remove a list of files or directories from the current local repository, use the option –cached before the list of files/dir if you do not want the files to be removed from the working directory
• 'git commit' will commit the list of files and directories currently staged
• 'git pull origin'
• 'git push origin'
• 'git checkout HEAD – <file>' just in case your deleted <file>, you can restore it with this command

Main article

• ipcs: give all the information on IPC (including shared memory -m, semaphore -s and queues -q)
• netstat: give all information on current network connections.
  • -ltpn or -atpn: all (listening) TCP ports with attached process
  • -aupn: all UDP ports with attached process
  • -i: prints network statistics for all interfaces
  • -s: prints detailed network stack stats (e.g. IP, TCP, UDP)
• iostat: get information on CPU and IO usage. Use the flags -xtc for extended disk statistics.
• top (?): instant CPU and process activity
• perfmeter: displays a small graphic of the CPU activity.
• vmstat: system wide usage (CPU, memory, I/O)
• mpstat -P ALL: per-processor (core) usage
• slabtop: Kernel memory usage
• pidstat -d: disk usage per process

• cscope
• ctags
• silentbob
• synergy2
• quicksynergy
• mkisofs
• cdrecord: sudo cdrecord -v driveropts=burnfree dev=ATA:1,0,0 image.iso ; sudo cdrecord -checkdrive driveropts=help dev=ATA:1,0,0 ; sudo cdrecord -scanbus dev=ATA
• tcpdump: 'tcpdump src host <hostname> and dst port <port-number>' ; 'tcpdump -i lo port 80'
• wireshark
• mkisofs -JlLrRT -v -V “Title” -P “Ozner” -p “Jean-Christophe BERTHON” -A “description” -hide-joliet-trans-tbl -o image.iso directory/
• To use when transforming all WordPress text files from PC to UNIX file format ('\n' instead of '\r\n'), the following command check if the find will be able to select proper file types:

  $ find . -type f ! -name "*.png" -a ! -name "*.gif" -a ! -name "*.jpg" -a ! -name "*.swf" | awk ' BEGIN { FS = "." } { print $NF }' | sort -u

  Then, it is possible to use the following command if all file types are for text files:

  $ find . -type f ! -name "*.png" -a ! -name "*.gif" -a ! -name "*.jpg" -a ! -name "*.swf" | sudo xargs dos2unix -p

• Install src.rpm : cd /usr/src/redhat/SRPMS ; sudo rpmbuild –rebuild synergy-1.3.1-1.src.rpm ; sudo rpm -Uvh ../RPMS/x86_64/synergy-1.3.1-1.x86_64.rpm
• dmidecode
• lsusb, lsmod, lspci
• LVM commands:
  • vgcreate data /dev/sda4
  • lvcreate -L200G –name data data
  • lvcreate -L50G –name vm data
  • mkfs.ext3 -j -l home /dev/data/data
• Backup and restore of partitions
  • dd if=sda.mbr of=/dev/sda ; dd if=/dev/sda of=sda.mbr bs=512 count=1
  • sfdisk /dev/sda < sfdisk.sda ; sfdisk -d /dev/sda > sfdisk.sda
  • partimage -bd restore /dev/sda1 sda1-boot.000 ; partimage -bd save /dev/sda1 sda1-boot
  • partimage -bd restore /dev/sda2 sda2-system.000 ; partimage -bd save /dev/sda2 sda2-system
• find $HOME/workspace -type f -iname “*.o” -print0 | xargs -0 -P 5 rm -f
• zypper (openSUSE/SLES package manager)
  • Search for dar: zypper search dar;
  • List available patches (smaller updates): zypper list-patches;
  • Info about a patch: zypper info -t patch openSUSE-2015-336;
  • Apply available patches: zypper patch;
  • List process(es) that need(s) restart: zypper ps
• iptraf : nice console HMI for monitoring network traffic
• lsof: to verify open files on a filesystem
• lsof +L1 -R: to verify list of open files which have been deleted/changed (e.g. by an update). The owning processes could need a restart.
• netstat -ltpn
• ethtool eth0 (and ethtool -i eth0; etc.)
• Command line monitoring tools:
  • iftop
  • latencytop
• Synology command line for checksums (works with either md5deep, sha1deep or sha256deep):
  • sha1deep -r -x checksum.sha1 .
  • sha1deep -r -l dir > dir.sha1