LinuxFoundation

1. Linux Filesystem Tree Layout

• one large logical filesystem which can contain one or many distinct filesystems mounted at various points

• Filesystem Hierarchy Standard : shareable (can shared between hosts)

	shareable	unshareable
static	/usr /opt	/etc /boot
-----------	------------------	--------------------
variable	/var/mail	/var/run /var/lock

• Main directories:
  • /bin : contains binary executable files (cat, kill, ps) . Command binaries which are deemed non essential enough are in /usr/bin.
  • /lib: contains libraries needed to execute the binaries in /bin or /sbin . Also kernel module are in /lib/modules/
  • /boot: essential files for booting the system (vmlinuz compressed linux kernel and initrd initial RAM filesystem which is mounted before the real root filesystem becomes available, config to configure kernel compilation)
  • /dev: device nodes aka device files (byte-stream or block I/O devices). Network devices (eth1 , eth2) do not have device nodes
  • /usr: can be thought of as a secondary hierarchy, need not reside in the same partition as the root directory
  • /etc: contains machine-local configuration files/scripts (/etc/systemd contains config scripts for starting, stopping system services using systemd ; also /etc/init.d which contains scripts for System V initialization)
  • /var: logs in /var/log and cron jobs, mail file in /var/spool, lock files in /var/lock or cat /var/log/yum.log
  • /proc: is the mount point for a pseudo-filesystem, where all information resides only in memory, not on disk. The entires in /proc are called virtual files with zero bytes in size.

# List files opened by a process
lsof -p PID 
# equivalent more or less with 
ls -l /proc/PID/fd

---

2. Processes

• Process = executing program and associated resources (open files, signal handlers) and has various states: running, sleeping
• Every process has a PID , PPID(parent PID), pgid(process group ID)
• init usually is the first process run on a system, and is the ancestor of all subsequent processes running on the system (except for thos with [] arounf their name , they are initiated by the kernel)
• orphaned processes (parent process dies before child) are adopted by init (thus the ppgid is set to 1) or (in distributions which use systemd) the ppgid is set to 2 (adopted by kernel thread known as kthreadd)
• zombie process (defunct) is a process which terminates before his parent and released almost all the resources and remained only to convey (communicate) his exit status
• processes are controlled by scheduling
• process context = snapshot of process by trapping the state of the CPU registers
• process permissions: programs marked with s (execute bit on) aka setuid programs run with the user-id of the user who owns the program versus program non-setuid which run with the permissions of the user who starts the program. (setuid program owned by root can be a security issue)
• when process is launched: it runs with the effective user-id and group-id of the user who started it, and with the corresponding privileges. This behaviour can be modified by using special permissions

#setuid programs (e.g.: owned by root)
/etc/passwd
/etc/shadow

# setuid bit is represented by an s in place of x
ls -l /bin/passwd
-rwsr-xr-x. 1 root root 27768 Feb 11  2017 /bin/passwd

• The setuid bit has no effect on directories. setuid or setgid bits are set, but the executable bit is not

• When a process is started it has its own isolated space and it uses system calls to indirectly access the HW (the HW is managed by the kernel)

# ulimit command that displays/resets a number of resource limits associated with processes running under a shell
ulimit -a


# increase no of file descriptors (soft resources)
ulimit -n 1600

# no of leak-ed file descriptors supported by OS
ulimit -n 

• process states : running, sleeping(waiting), stopped(suspended Ctrl+Z sends SIGSTOP), zombie

• fork - original parent process keeps running while the child process starts

• exec - original parent process terminates and child parent inherits the process ID of the parent

• when the user types a command in the shell a new process is created (using fork from the user's login shell then the command is loaded onto child process space via exec system call)

# list all process on the system PID PPID PRI(priority) NI(nice value)
ps -elf

• nice -n 5 command [args] = niceness value can range from -20 (the highest priority) to +19 (the lowest priority) aka set priorities

# start bash with nice value of 10
nice -n 10 bash

# change the value of bash session to 15
renice -n 15 -p (ppid)

---

3. Signals (Inter_Process Communication)

• Are used to emit notifications for processes to take action in response to unpredictable events
• kill -l list signals , basic syntax kill <SIGNAL> <PID>
• pkill -u libby foobar kills process foobar of libby user

---

14. Process monitoring

• process monitoring tools: top-process activity dynamically updated , ps-detailed information about processes , uptime-how long the system is running and average load, mpstat-multiprocessor usage, iostat-CPU utilisation and I/O statistics, sar-display and collect info about system activity, strace-information about sys calls a process makes
• ps is the workhorse for displaying characteristics and it reads information from /proc
• ps possibilities
  1. UNIX-style: options which must be preceded by -
  2. BSD-style: options, which must not be preceded by -
  3. GNU-style long option, each of which must be preceded by --

###  BSD-style: display all processes owned by you and Lift the BSD-style "must have a tty" restriction
## USER  PID %CPU %MEM    VSZ   RSS TTY  STAT START   TIME COMMAND
ps aux 
ps auxf # will show the ancestry tree

# specified user-defined format for the ps output: [parent PID, PID, percentage memory usage
ps -o ppid,pid,etime,command,pmem,pcpu
ps e -o ppid,pid,etime,command,pmem,pcpu

### UNIX-style: display all processes (background ones to) using BSD long format with full format listing
## UID  PID  PPID  F %CPU PRI NI   SZ    RSS WCHAN     S     ADDR TTY    TIME CMD     STIME
ps -elf


# add columns besides the default values (PPID is shown in the default ps -elf)
ps -ef -O etime,pmem,command


# visual description of the process ancestry
pstree -aAp <PID>

• top = interactive use 1 (each CPPU shown separately) , i (only interactive processes are shown), l (show load)

---

4. Package Management Systems

• rpm = Redhat Package Manager (Centos, Fedora) + SUSE (OpenSuse)

• dpkg = Debian Package Manager (Ubuntu, Debian, Mint)

• https://www.digitalocean.com/community/tutorials/package-management-basics-apt-yum-dnf-pkg.

• package types: Binary, Source (rrpmbuild --rebuild -rb p7zip-16.02-16.el8.src.rpm)

• low-level utilities (rpm, dpkg) and high-level utilities (yum,dnf,apt,apt-get)

• EPEL (Extra Packages for Enterprise Linux) - external package repository

  4.1 RPM (RedHat Package Manager)

  • rpm in itself doesn't retrieve packages over the network and installs only from the local machine
  • types of packages: binary (<name>-<version>-<release>.<distro>.<architecture>.rpm) and source (<name>-<version>-<release>.<distro>.src.rpm) also packages with noarch.rpm extension don't depend on CPU architecture
  • /var/lib/rpm = default directory which holds the RPM database (in form of Berkeley DB Hash Files).

# error: rpmdb: damaged header #929 retrieved -- skipping.  
rm -f /var/lib/rpm/__db* # remove lock files
rpm --rebuilddb

# alternative db directory
rpm --dbpath

# repair/rebuild DB
rpm --rebuilddb

# list installed packages 
# yum list installed
rpm -qa

# check file to which package it's belonging to
rpm -qf </path/to/file>

# information about the package (e.g if it is Relocatable)
rpm -qip <package.rpm> 

# list all files from the package aka $ls -lF $(rpm -ql </path/package>)
rpm -qil <package.rpm>

# verify if the files from the package are consistent with RPM database
rpm -Va </path/to/package> # no output means the package is ok

# install pacakage
rpm -ivh <package.rpm>

# upgrade package (also remove the old package)
rpm -Uvh <package.rpm>

# freshening packages in current dir (when download patches and what to upgrade packages already installed)
rpm -Fvh *.rpm

# uninstall a package using --test flag before
rpm -e --test <package> (not path to package)

4.2 YUM (high-level package manager):

• resolves dependencies automatically it also caches information to speed up performance
• repos configuration: /etc/yum.repos.d
• yum configuration: /etc/yum.conf
• OS patches and update: /var/log/yum.log
• toggle a particular repo in /etc/yum.repos.d change value from enabled to 0 or 1
• dnf = next generation replacement for yum

# clear repo cache
yum clean all

# search for package
yum search <package>
yum list

# install package e.g yum install ngnix
yum install <package> 

# install rpm package
yum localinstall <package.rpm>

# check if updates are needed on local system
yum update
yum list updates
yum check-updates

# update package
yum update <package>

# list installed / available packages
yum list installed [installed | updates | available ]

# info about package $rpm -qip package
yum info <package>


# check file to which package it's belonging to $rpm -fq file
yum provides </path/to/file>
yum provides "/logrotate.conf"

# all packages with bash reference/installed and available/package info/dependencies
yum search bash
yum list bash
yum info bash
yum deplist bash

• adding a new yum repo: vim /etc/yum.repos.d/webmin.repo

[Webmin]
name=Webmin Distribution Neutral
baseurl=http://download.webmin.com/download/yum
mirrorlist=http://download.webmin.com/download/yum/
mirrorlist
enabled=1
gpgcheck=0

---

14. I/O monitoring and I/O scheduling

• system I/O bound when the CPU is in IDLE waiting for I/O to complete or the network is waiting to clear buffers
• Network, RAM, CPU can wait for the I/O to complete
• iostat = monitoring I/O devices activity iostat -k [KB] iostat -m [MB] iostat -xk [extended] (utilisation % close to 100 => system I/O bound)
• iotop = display table of current I/O usage dynamically
• ionice = set I/O scheduling and priority for a given process ionice [-c class] [-n priority] [-p pid] [command [args]]
• I/O raporting bonnie and fs_mark
• system performance depends on optimising I/O scheduling strategy
• block layer [I/O scheduler - interface] low-level physical devices. I/O scheduler prioritises requests from Virtual Memory and Virtual File System to block-devices