Unix stuff

Unix cheatsheet

Install a .deb package

dkpg -i /path/to/file.deb

or

apt install /path/to/file.deb

Use --fix-broken or -f directive to install missing dependencies

Bash

Source

History

sudo !!  # run last as sudo

!!                     # The previous command
!-5                    # The fifth last command
!372                   # Command number 372
!foo                   # The last command starting with `foo'
!?bar?                 # The last command containing the string `bar'
^foo^bar               # The previous command, with `foo' replaced by `bar'
!^                     # The first argument of the previous command
!$                     # The last argument of the previous command
!*                     # All the arguments of the previous command
!?foo?:3:h:gs/bar/baz/ # Argument 3 of the last command containing the string `foo' with the trailing pathname component stripped off and all occurrences of `bar' replaced by `baz'

Prompts

Source

• $PS1 This is the main prompt, seen at the command-line.
• $PS2 The secondary prompt, seen when additional input is expected. It displays as ">".
• $PS3 The tertiary prompt, displayed in a select loop.
• $PS4 The quartenary prompt, shown at the beginning of each line of output when invoking a script with the -x [verbose trace] option. It displays as "+".

Globbing

*        # All files (except those starting with `.')
foo*bar  # All files starting with `foo' and ending with `bar'
ba?      # All three-character files starting with `ba'
foo?*bar # All files starting with `foo' and ending with `bar' with at least one character between them
[A-Z]*   # All files starting with a capital letter
*.[ch]   # All files ending in `.c' or `.h'
*[^0-9]  # All files which don't end in a number

a{b,c,d}e                   # Expands to `abe ace ade'
/usr/{bin,lib,man}          # Expands to `/usr/bin /usr/lib /usr/man'
/usr/{,local/}{bin,lib,man} # Expands to `/usr/bin /usr/lib /usr/man /usr/local/bin /usr/local/lib /usr/local/man'

{1..100}     # sequence of numbers 1 2 ... 99 100
{01..100}    # with zero paddings
{1..100..3}  # by steps of 3
{100..1..-1} # by steps if -3

I/O redirection

foo > bar        # Redirects the output of `foo' to the file `bar'
foo 2> bar       # Redirects the error output of `foo' to the file `bar'
foo >> bar       # Appends the output of `foo' to the file `bar', rather than clobbering it
foo < bar        # Causes `foo' to read its input from the file `bar'
foo | snafu      # Takes the output of `foo' and feeds ("pipes") it into `snafu' as input
foo 2>&1         # Sends the error output to the same place as the standard output (useful for sending both to the same place)
foo > bar 2> baz # Redirects the output of `foo' to the file `bar', and the error output to the file `baz'
foo > bar 2>&1   # Redirects the output of `foo' to the file `bar', and then redirects the error output to the same place
foo 2>&1 > bar   # Redirects the error output to where the standard output (was) going, and then redirects the standard output (only) to `bar'
foo &> bar       # Same as `foo > bar 2>&1' (short-cut)
foo 2>&1 | snafu # Pipes both the standard output and the error output of `foo' into the input of `snafu' (the pipe redirection is always done first, so when the error output is sent to the same place as the standard output, the standard output is already being sent to the pipe)

File descriptor

0 # stdin
1 # stdout
2 # stderr

Manipulate text

Display text files

Show line number

cat -n file

Scrollable wall of text

less file

Squeeze blank lines (for cat too)

cat -s file
less -s file

Chomp long lines

less -S file

Open many files, move with :n for next file and :p for previous file

less file1 file2 file3

Show control chars

less -r file

Show only first/last 10 lines

head file
tail file

Last 25 lines

tail -n25 file

Last 25 char

tail -c25 file

Print last 10 lines then monitor for updates

tail -f file

Monitor if file is renamed/recreated/deleted

tail -F file

Counts default -w' words, -llines,-cbytes, also-m` chars wc *.f90

Examine differences

• -q brief, difference only
• -y side by side, 2 columns
• -r recursively for sub-directories
• -s explicitly state if files are identical

Compare binary files

cmp -l file1 file2

Edit lines and text

Source

Extended grep

grep -E "(a*)|(b*)\1"

Search folder (-r) recursively for files with specific text string (-e) matching words only (-w), show line number (-n )

grep -rnw directory -e target_string

Filter and count only lines with regex, -v to invert match, -i ignore case

grep -w 'regex' -c file
grep -w 'regex' -c -v -i file

List, find and sort in reverse

ls | grep 'pattern' | sort -r

Sort third field numerically, comma seperated (default delimiter is space)

sort -k3n -t','

Print out lines 5-8/31st line and first 10 chars only

sed -n '5,8p' tpdf_SLURM
sed -n '31p' tpdf_SLURM | cut -c-10

Extract data from CSV: comma seperated, print first and third columns only

cut -d',' -f1,3 file

Take first field, sort, select only unique entries and count

cat file | cut -d',' -f1 | sort | uniq | wc -l

Count number of identical entries, sequential only so needs to be sorted first

cat file | cut -d',' -f1 | sort | uniq -c

Take 10th/first 10/10th to last/10th to 20th characters per line

cat file | cut -c10
cat file | cut -c-10
cat file | cut -c10
cat file | cut -c10-20

tee

Print to stdout and writes to file(s)

ls | tee –a appendFile
ls | tee multiple files file1

Unix command tools

Basics

Run commands without alias

\ls

Create symbolic link

ln -s /share/share3/austin/tpdf ./tpdf

Shows physical directory path, ignoring symlinks

pwd -P

Shows logical directory with symlinks

pwd -L

Recursively list sub directories; sort by -t time, -S size, -r reverse order

ls -R

Display / after directories, @ for sym link, * for executables etc

ls -F

Copy files, -ppreserving timestamps "last modified" time-stamp and -r recursively for directories

cp -pr src dest

Interactive flag -i for mv and rm asking confirmation for each file.

Create directory with parents

mkdir -p path/to/dir

Rename multiple files with rename or prename

Archiving and compressing

Source

Create archiving create, verbose, filename

tar cvf archive_name.tar dirname/

Create compressed archive. bzip takes more time to compress and decompress than gzip, but higher compression

• z gzip .tgz .tar.gz

• j bzip2 .tbz .tb2 .tar.bz2

  tar cvzf archive_name.tar.gz dirname/ tar cvfj archive_name.tar.bz2 dirname/

Extract whole archive

tar xvf archive_name.tar
tar xvfz archive_name.tar.gz

View without extracting

tar tvf archive_name.tar
tar tvfz archive_name.tar.gz

Extract a single file or dir(s)

tar xvf archive_file.tar /path/to/file
tar xvf archive_file.tar /path/to/dir1/ /path/to/dir2/

Add a file to exisitng tarball (uncompressed only)

tar rvf archive_name.tar newfile

Verify contents (no compression)

tar cvfW file_name.tar dir/

Difference between gzip archive file and file system

tar dfz file_name.tgz

Background tasks

Run a background process

nohup tar cvzf 1401.tar.gz 1401/ > 1401.out &> 1401.err1 &

If you forget the &, temporarily suspend the job with CTRL + Z then run it in the background with bg.

Monitor tasks currently running

jobs -l

Bring jobs suspended bu CTRL + Z back to foreground with fg.

Check running processes

-f for more info, -H process hierarchy, f forest/tree view, a all processes, u user oriented (usage detail)

ps -edHf | grep austin
ps aux
ps xuf

top

Kill processes with

kill -9 [PID]

• SIGHUP (1) – Hangup detected on controlling terminal or death of controlling process. Use SIGHUP to reload configuration files and open/close log files.
• SIGKILL (9) – Kill signal. Use SIGKILL as a last resort to kill process. This will not save data or cleaning kill the process.
• SIGTERM (15) – Termination signal. This is the default and safest way to kill process.

Link

Disk usage

Get total usage only

du -ch | grep total

• -h show file sizes in human readable format
• -s sumarise, display a total only
• -c total, produce a grand total
• -a all, show sizes of files, not just directories

Free space df -h

Check disk quota. "Blocks" is how many disk blocks you are using, in chunks of 1 kB

quota -s

Monitor things

Executes a command repeatedly every 5 sec

watch -n 5 "ps -e | grep php"

Print last 10 lines then monitor for updates and print them

tail -f file

Monitor if file is renamed/recreated/deleted

tail -F file

Mount a partition/iso image

sudo mount -t ext4 /dev/sda3 /mnt/scratch
sudo mount -t iso9600 -o loop /path/to.iso /mnt/iso

Source

Technically a loop device is a block device that writes to a file, rather than a piece of hardware. So you always use/need to use the loop back device when mounting a file. Auto mount drives through /etc/fstab or PySDM (Python Storage Disk Manager) or Disks in menu

/dev/disk/by-uuid/* /scratch ext4 rw,suid,dev,exec,auto,user,async

Unmount

sudo umount /mnt/scratch

List disks/drives

sudo fdisk -l

Search for files

Search for files, kill errors

find / -name '*.f90' 2> /dev/null

Search and execute a command on those found, chained

find .. -name "*.mod" -exec rm {} \; ; find .. -name "*.o" -exec rm {} \; ;

Synchronise files remotely

Syncing files between two locations

rsync -rvuPS -e 'ssh -p 2222' local_folder user@255.255.255.255:~/remote

• -a archive -rlptgoD
• -h human readable
• -r recursive
• -u update
• -P partial&progress
• -S sparse
• -v verbose
• -n dryrun

Copy files

scp -P 2052 ~/.ssh/id_dsa.pub remote:~/

Misc commands

Change grub listing

finger    # lookup a user
who       # is currently logged on
whoami    # my username
id        # group info
uname     # linux OS info
hostname
printenv  # print all defined (environemnt) variables
date

env       # run a program in a modified environment

Locates a command and returns the pathnames to it in the current environment, -a prints all matching pathnames

which -a cat

Update finger info, also see ~/.path, ~/.projects

chfn

Get CPU info and others

/bin/cat /proc/cpuinfo | grep 'processor|model name|chache size|core|sibiling|physical'
lspci     # list pci devices
lsblk

List open files

lsof -u $USER

Link dump

Conquering the command line

Misc

Wierd time zone issues? Windows expects the hardware clock to be local time. Linux expects it to be UTC. Read more.

See timedatectl