sysprog_notes.org

Systems Programming - Unofficial Notes

Disclaimer

Hi, I’m Denis, and here I post notes that I am taking in the class. These do not necessarily reflect the lecturer’s exact words; in fact, they are deliberately distorted with my biased ideas. Whoever let me cook, made a huge mistake.

Outline

Order	Project	Approach
1	Device driver I/O	Kernel module
2	File system (FUSE)	User-space
3	Process management	Kernel compile

Week 1 - Introduction and VM installation

• Kernel cannot depend on any external libraries
• Turgut Uyar’s system: Lubuntu
• Dealing with the concept of system call. Specials functions with switch from user space to kernel space.
• Requirements: C compiler, Emacs, and GNU/Linux operating system
• Use virtual machine in order to not brick your computer
• Oracle VM: https://www.virtualbox.org/
• Long Term Support (LTS) versions are the most stable
• Linux, Ubuntu 22.04 LTS (Jammy Jellyfish), English version
• Minimal system: 4 GB RAM, 1 core CPU, disk size 40 GB
• In the last project, we will compile Linux kernel, there should be enough storage
• Minimal installation, without office software, games and other crapware
• Good messaging apps: Signal, Matrix, personal email server
• Apple - devil
• Get new phone - A Greentext Story
• stay off grid 󰋂
• Install virtualbox extensions for shared folders, clipboard, higher window size and resolution On the virtual machine

  sudo ./VBoxLinuxAdditions.run

• vboxusers: Add users to special group for interaction with the virtual box
• If host is Windows and VM is Linux, there may be conflicts between NTFS and ext4 file systems. Use only for copying files, not for compilation.
• Linus Torvalds on Oracle VirtualBox functioning worse than its open source version

  Software is like sex: it’s better when it’s free.

• System Programming slides on Ni nova for extra information
• Not many tutorials on device driver and process management, because the standards change. RTFM.

Week 2 - Kernel Modules

• Some details about rookie virtualbox, shared folder
• Commands on fresh installation

sudo apt update
sudo apt upgrade
sudo apt install build-essential gcc-12

• You have to use the same compiler, the same version when compiling modules of a kernel
• When working with kernel modules, there is still risk of crashing the system
• Hello world module
  • <linux/module.h> library
  • Reason of writing static is to prevent exporting variables or functions, thus polluting global space. Like private.
  • printk - print kernel, configurable print function
  • Makefile, tell the system how to compile the module
  • ls -l (sometimes aliased to ll) - list directory contents with detailed information
  • lsmod - list modules
  • /proc - not an actual directory, current state of the kernel, can be viewed with ls
  • /proc/modules - shows nearly the same information as lsmod
  • modules are located at /lib/modules
  • uname -a: all information about kernel
  • uname -r: show kernel revision number
  • Never ignore compile warnings

    

  • sudo insmod hello.ko - insert module
  • When kernel sees printk, it prints to system logs
  • sudo tail /var/log/kern.log or dmesg - system logs
  • lsmod | grep hello - pipeline output of lsmod to grep (choose lines that contain hello)
  • rmmod <name> - unload module
  • Readable parameter, but cannot be changed: module_param(variable, type, flag), flag S_IRUGO
  • Parameters can be changed while calling insmod:

sudo insmod hello.ko whom=Linus

• Device Types
  • character, block, network
  • A character device is a stream of characters that is read and written. Ex: console, mouse, serial port, sound card. Data cannot be read randomly, only serially.
  • Block device, data transfers are transferred in fixed-sized blocks. Every block is addressable and there is no need for moving the pointer. More complicated.
• Operating systems programming
  • Interrupt driven coding is better for CPU, but harder
  • Blocking is better for user-space programs, easier
  • Operating systems make interrupt look like synchronous
  • Posix interface
  • The problem of operating systems is to receive a function request, find it on the system and execute
• /dev
  • sda - seventh day adventists SCSI (SKUZ-ee) Disk A
  • /dev/console - full screen console, /dev/tty - terminals, /dev/sda scsi disks, disks are emulated as scsi in modern Linux
  • /dev/sdb1 and /dev/sdb2 - first and second partitions
  • ls -l /dev/sd* - view scsi disks
  • The first letter that comes after ls -l command at each row. c is character, - is regular file, l is link, d is directory.
  • major number x (driver), minor number y (actual device)
  • 256 of major number is a very small number for drivers.
  • Subsetting: Drivers can take intervals of major numbers and minor numbers.
• make clean - delete all intermediate files created after build
• Registering major and minor numbers
  • <linux/fs.h> - number allocation and other
  • major number 0 -> give any major number. Otherwise, give the specified number (not a good idea).
  • alloc_chrdev_region(<address of data structure>, <starting of minor numbers>, <number of minor numbers>, <name>)
  • alloc_chrdev_region(&devno, pseudo_minor, 1, “pseudo”)
  • register_chrdev_region -> registering a specific address region. Much higher chance to fail.
  • On exitting give the numbers back: unregister_chrdev_region()
  • grep pseudo /proc/devices: show major numbers of devices
• Device driver interface - next week.
• I/O will not be actually implemented; instead, it will be simulated using memory manipulations. Pseudo-device.

Week 3 - Device Driver

• Device drivers are interacted with using open, close, read and write, functions known as system calls
• For anything that is nor input nor output, ioctl is used. Example: configuration of peripherals in embedded systems.
• Do not confuse open, close (system calls) with fopen, fclose (call system calls)

int open(const char *path, int oflag, ...);
int close(int fd);

• read
  • fd - file descriptor
  • return value - x
  • x = nbyte: successful completion
  • x = 0: end-of-file
  • x < 0: error
  • 0 < x < count: partial transfer. Not a problem, this is a normal operation. Repeat the request for needed bytes.

ssize_t read(int fd, void *buf, size_t nbyte);

• ioctl
  • int ioctl(int fd, unsigned long request, ...);
  • No standard number of arguments
  • ioctl_list

• Normally we would use C functions instead of system calls. System calls are harder.
• strace - list system calls involved in a program
• pseudo2.c - open
  • Check if a driver can be opened: ls -l /dev/foo. If driver does not exist, create a new node: sudo mknod /dev/pseudo c 240 0
  • Create open and close system call implementations
  • pseudo_open, pseudo_release. Fixed function signature, OS sends specific parameters that cannot be changed.
  • struct file_operations: kernel data structure that links open, release, owner etc. properties to our functions.
  • <linux/cdev.h>
  • cdev_init()
  • cdev_add(&pseudo_cdev, devno(major and minor numbers), 1): function to add the event handling to kernel.
  • struct cdev: kernel data structure
  • cdev_del(): call when unloading the module
  • Usage of & in C allows function to modify the variable
  • We can see that the device is opened and closed in dmesg
  • Linux Device Drivers book provides example with multiple devices
• pseudo3.c - read
  • Example of reading one byte from the device
  • If you implement read and make a mistake, reading can crash the system.

    

  • If the function is not implemented, kernel uses a default handler, and the system won’t crush.
  • pseudo_read puts a fixed value buf[0] = ‘A’. Add to pseudo_fops.
  • semaphore up(&pseudo_sem). Prevent multiple programs from reading at the same time. Wait on the down operation.
  • sema_init();
  • od (octal dump) -x (hexadecimal) -N 6 /dev/pseudo - read 6 bytes from the pseudo-device. No need to write test functions.
  • Free A machine, it always returns 1, just like the yes command
• pseudo4.c - monetizing A’s
  • Give one character at a time
  • dummy data is created: pseudo_data, pseudo_capacity
  • copy_to_user, safer function to copy data
  • Endianness is visible in od
• pseudo5.c - give new data continuously
  • f_pos should be updated to read next data portions
  • od -v option

    -v, –output-duplicates do not use * to mark line suppression

  • cat /dev/pseudo - continuously read from the device
  • Device accesses random data because there are no boundaries
• pseudo6.c - limiting f_pos
  • return 0 to indicate end of file
• Tasks (from easy to relatively easy)
  1. make capacity a module parameter
  2. circular read
  3. write (not circular)
  4. llseek
  5. ioctl operation: SHIFT, +n to every byte in the buffer
  6. automatically create node, without finding the major and minor numbers and calling mknod (hint: udev rules).
  7. create an entry in /proc
  8. create a git repository where each commit is one task

Week 4 - File system (FUSE)

• Developing in user-space: safer than in kernel space, external libraries are available.
• VFS - Virtual File System. Software layer in the kernel that provides the filesystem interface to userspace programs.
• Mounting - associating directories with some storage device. We might change partitions as we traverse the file system without realizing.
• hello.c - Creates one directory and one text file.
  • D_FILE_OFFSET_BITS=64 compiler option is related to architecture
  • -lfuse option links with FUSE library (the library must be installed)
  • hello.c creates a text file containg “Hello world”, which actually does not exist.
  • ”/” location means the current mounting point, not the root mounting point.
  • filler function advertises . .. and hello.txt.
  • IFDIR (directory) with 0755 (all permissions except write).
  • IFREG (file) with 0444 (read permissions to everyone).
  • (!) Numbers starting with 0 in C are octal.
  • -ENOENT no such file or directory. EACCES permission denied.
  • Run with ./hello ~/SP24
  • -d flag debug flag for fuse_main.
  • mount, umount shell commands for listing mounted devices and unmounting.
  • Unmount ~/SP24 after testing code.
• /dev/null - Says it has written but does not write anything.

ssize_t pseudo_write(fd, buf, count) {
  return count;
}

• See also WOM (Write-Only Memory)

• /dev/full - Says it is full.

ssize_t pseudo_write(fd, buf, count) {
  return 0;
}

• cats?.c
  • https://catfact.ninja/ - api example. Goal: show api results as file entries.
  • curl -s https://catfact.ninja/breeds:

"data": [
  {
    "breed": "Abyssinian",
    "country": "Ethiopia",
    "origin": "Natural/Standard",
    "coat": "Short",
    "pattern": "Ticked"
  },
  {
    "breed": "Aegean",
    "country": "Greece",
    "origin": "Natural/Standard",
    "coat": "Semi-long",
    "pattern": "Bi- or tri-colored"
  },
... 

• Pipe to jq . to show pretty json and filter results. See man pages for JQ(1), “BASIC FILTERS”.
• We can use libcjson and libcurl, because we are developing in user space.
• -lcurl, -lcjson linker options
• Code using curl library is copied from official examples. https://curl.se/libcurl/c/example.html
• Trick in the C language. malloc(1) to initialize the pointer, then realloc.
• realloc as curl data chunks are coming, which might happen several times.
• Cycle through all pages and fetch data. Ex: breed?page=3
• Result: cat breeds are shown as empty files.

• Assignment
  • Implement a read function to display fetched API data inside the text files.
  • Implement a write function with post API, to upload changes made in the text files back to the server.
  • You can use FUSE with everything you can map to folders and files. https://catfact.ninja/ is just an example.