-
Notifications
You must be signed in to change notification settings - Fork 78
Appendix
- Appendix
[1][]
This is going to be a massive guide to helping you debug your C programs. There are different levels that you can check errors and we will be going through most of them. Feel free to add anything that you found helpful in debugging C programs including but not limited to, debugger usage, recognizing common error types, gotchas, and effective googling tips.
Make your code modular using helper functions. If there is a repeated task (getting the pointers to contiguous blocks in the malloc MP, for example), make them helper functions. And make sure each function does one thing very well, so that you don’t have to debug twice.
Let’s say that we are doing selection sort by finding the minimum element each iteration like so,
void selection_sort(int *a, long len){
for(long i = len-1; i > 0; --i){
long max_index = i;
for(long j = len-1; j >= 0; --j){
if(a[max_index] < a[j]){
max_index = j;
}
}
int temp = a[i];
a[i] = a[max_index];
a[max_index] = temp;
}
}Many can see the bug in the code, but it can help to refactor the above method into
long max_index(int *a, long start, long end);
void swap(int *a, long idx1, long idx2);
void selection_sort(int *a, long len);And the error is specifically in one function. In the end, we are not a class about refactoring/debugging your code. In fact, most kernel code is so atrocious that you don’t want to read it. But for the sake of debugging, it may benefit you in the long run to adopt some of these practices.
Use assertions to make sure your code works up to a certain point – and importantly, to make sure you don’t break it later. For example, if your data structure is a doubly linked list, you can do something like assert(node->size == node->next->prev->size) to assert that the next node has a pointer to the current node. You can also check the pointer is pointing to an expected range of memory address, not null, ->size is reasonable etc. The NDEBUG macro will disable all assertions, so don’t forget to set that once you finish debugging. assert link
Here’s a quick example with assert. Let’s say that I’m writing code using memcpy
assert(!(src < dest+n && dest < src+n)); //Checks overlap
memcpy(dest, src, n);This check can be turned off at compile time, but will save you tons of trouble debugging!
When all else fails, print like crazy! Each of your functions should have an idea of what it is going to do (ie find_min better find the minimum element). You want to test that each of your functions is doing what it set out to do and see exactly where your code breaks. In the case with race conditions, tsan may be able to help, but having each thread print out data at certain times could help you identify the race condition.
Valgrind is a suite of tools designed to provide debugging and profiling tools to make your programs more correct and detect some runtime issues. The most used of these tools is Memcheck, which can detect many memory-related errors that are common in C and C++ programs and that can lead to crashes and unpredictable behaviour (for example, unfreed memory buffers). To run Valgrind on your program:
valgrind --leak-check=full --show-leak-kinds=all myprogram arg1 arg2Arguments are optional and the default tool that will run is Memcheck. The output will be presented in form of number of allocations, number of freed allocations, and the number of errors.Suppose we have a simple program like this:
#include <stdlib.h>
void dummy_function() {
int* x = malloc(10 * sizeof(int));
x[10] = 0; // error 1:as you can see here we write to an out of bound memory address
} // error 2: memory leak the allocated x not freed
int main(void) {
dummy_function();
return 0;
}This program compiles and run with no errors. Let’s see what Valgrind will output.
==29515== Memcheck, a memory error detector
==29515== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==29515== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==29515== Command: ./a
==29515==
==29515== Invalid write of size 4
==29515== at 0x400544: dummy_function (in /home/rafi/projects/exocpp/a)
==29515== by 0x40055A: main (in /home/rafi/projects/exocpp/a)
==29515== Address 0x5203068 is 0 bytes after a block of size 40 alloc'd
==29515== at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==29515== by 0x400537: dummy_function (in /home/rafi/projects/exocpp/a)
==29515== by 0x40055A: main (in /home/rafi/projects/exocpp/a)
==29515==
==29515==
==29515== HEAP SUMMARY:
==29515== in use at exit: 40 bytes in 1 blocks
==29515== total heap usage: 1 allocs, 0 frees, 40 bytes allocated
==29515==
==29515== LEAK SUMMARY:
==29515== definitely lost: 40 bytes in 1 blocks
==29515== indirectly lost: 0 bytes in 0 blocks
==29515== possibly lost: 0 bytes in 0 blocks
==29515== still reachable: 0 bytes in 0 blocks
==29515== suppressed: 0 bytes in 0 blocks
==29515== Rerun with --leak-check=full to see details of leaked memory
==29515==
==29515== For counts of detected and suppressed errors, rerun with: -v
==29515== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)Invalid write: It detected our heap block overrun, writing outside of allocated block.
Definitely lost: Memory leak — you probably forgot to free a memory block.
Valgrind is a very effective tool to check for errors at runtime. C is very special when it comes to such behavior, so after compiling your program you can use Valgrind to fix errors that your compiler may not catch and that usually happen when your program is running.
For more information, you can refer to the valgrind website.
ThreadSanitizer is a tool from Google, built into clang and gcc, to help you detect race conditions in your code. For more information about the tool, see the Github wiki. Note, that running with tsan will slow your code down a bit. Consider the following code.
#include <pthread.h>
#include <stdio.h>
int global;
void *Thread1(void *x) {
global++;
return NULL;
}
int main() {
pthread_t t[2];
pthread_create(&t[0], NULL, Thread1, NULL);
global = 100;
pthread_join(t[0], NULL);
}
// compile with gcc -fsanitize=thread -pie -fPIC -ltsan -g simple_race.cWe can see that there is a race condition on the variable global. Both the main thread and the thread created with will try to change the value at the same time. But, does ThreadSantizer catch it?
$ ./a.out
==================
WARNING: ThreadSanitizer: data race (pid=28888)
Read of size 4 at 0x7f73ed91c078 by thread T1:
#0 Thread1 /home/zmick2/simple_race.c:7 (exe+0x000000000a50)
#1 :0 (libtsan.so.0+0x00000001b459)
Previous write of size 4 at 0x7f73ed91c078 by main thread:
#0 main /home/zmick2/simple_race.c:14 (exe+0x000000000ac8)
Thread T1 (tid=28889, running) created by main thread at:
#0 :0 (libtsan.so.0+0x00000001f6ab)
#1 main /home/zmick2/simple_race.c:13 (exe+0x000000000ab8)
SUMMARY: ThreadSanitizer: data race /home/zmick2/simple_race.c:7 Thread1
==================
ThreadSanitizer: reported 1 warningsIf we compiled with the debug flag, then it would give us the variable name as well.
A very useful trick when debugging complex C programs with GDB is setting breakpoints in the source code.
int main() {
int val = 1;
val = 42;
asm("int $3"); // set a breakpoint here
val = 7;
}$ gcc main.c -g -o main && ./main
(gdb) r
[...]
Program received signal SIGTRAP, Trace/breakpoint trap.
main () at main.c:6
6 val = 7;
(gdb) p val
$1 = 42For example,
int main() {
char bad_string[3] = {'C', 'a', 't'};
printf("%s", bad_string);
}$ gcc main.c -g -o main && ./main
$ Cat ZVQ�� $
(gdb) l
1 #include <stdio.h>
2 int main() {
3 char bad_string[3] = {'C', 'a', 't'};
4 printf("%s", bad_string);
5 }
(gdb) b 4
Breakpoint 1 at 0x100000f57: file main.c, line 4.
(gdb) r
[...]
Breakpoint 1, main () at main.c:4
4 printf("%s", bad_string);
(gdb) x/16xb bad_string
0x7fff5fbff9cd: 0x63 0x61 0x74 0xe0 0xf9 0xbf 0x5f 0xff
0x7fff5fbff9d5: 0x7f 0x00 0x00 0xfd 0xb5 0x23 0x89 0xff
(gdb)
Here, by using the x command with parameters 16xb, we can see that starting at memory address 0x7fff5fbff9c (value of bad_string), printf would actually see the following sequence of bytes as a string because we provided a malformed string without a null terminator.
What do you actually use to run your program? A shell! A shell is a programming language that is running inside your terminal. A terminal is merely a window to input commands. Now, on POSIX we usually have one shell called sh that is linked to a POSIX compliant shell called dash. Most of the time, you use a shell called bash that is not entirely POSIX compliant but has some nifty built in features. If you want to be even more advanced, zsh has some more powerful features like tab complete on programs and fuzzy patterns, but that is neither here nor there.
A shell is actually how you are going to be interacting with the system. Before user friendly operating systems, when a computer started up all you had access to was a shell. This meant that all of your commands and editing had to be done this way. Nowadays, our computers start up in desktop mode, but one can still access a shell using a terminal.
(Stuff) $
It is ready for your next command! You can type in a lot of unix utilities like ls, echo Hello and the shell will execute them and give you the result. Some of these are what are known as shell-builtins meaning that the code is in the shell program itself. Some of these are compiled programs that you run. The shell only looks through a special variable called path which contains a list of : separated paths to search for an executable with your name, here is an example path.
$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:
/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
So when the shell executes ls, it looks through all of those directories, finds /bin/ls and executes that.
$ ls
...
$ /bin/ls
You can always call through the full path. That is always why in past classes if you want to run something on the terminal you’ve had to do ./exe because typically the directory that you are working in is not in the PATH variable. The . expands to your current directory and your shell executes <current_dir>/exe which is a valid command.
-
The up arrow will get you your most recent command
-
ctrl-rwill search commands that you previously ran -
ctrl-cwill interrupt your shell’s process -
Add more!
A terminal is just an application that displays the output form the shell. You can have your default terminal, a quake based terminal, terminator, the options are endless!
-
catconcatenate multiple files. It is regularly used to print out the contents of a file to the terminal but the original use was concatenation.$ cat file.txt ... $ cat shakespeare.txt shakespeare.txt > two_shakes.txt -
difftells you the difference of the two files. If nothing is printed, then zero is returned meaning the files are the same byte for byte. Otherwise, the longest common subsequence difference is printed$ cat prog.txt hello world $ cat adele.txt hello it's me $ diff prog.txt prog.txt $ diff shakespeare.txt shakespeare.txt 2c2 < world --- > it's me -
greptells you which lines in a file or standard in match a POSIX pattern.$ grep it adele.txt it's me -
lstells you which files are in the current directory. -
cdthis is a shell builtin but it changes to a relative or absolute directory$ cd /usr $ cd lib/ $ cd - $ pwd /usr/ -
manevery system programmers favorite command, tells you more about all your favorite functions! -
makeexecutes programs according to a makefile.
Shells have many useful utilities like saving output to a file using redirection >. This overwrites the file from the beginning. If you only meant to append to the file, you can use >>. Unix also allows file descriptor swapping. This means that you can take the output going to one file descriptor and make it seem like its coming out of another. The most common one is 2>&1 which means take the stderr and make it seem like it is coming out of standard out. This is important because when you use > and >> they only write the standard output of the file. There are some examples below.
$ ./program > output.txt # To overwrite
$ ./program >> output.txt # To append
$ ./program 2>&1 > output_all.txt # stderr & stdout
$ ./program 2>&1 > /dev/null # don't care about any output
The pipe operator has a fascinating history. The UNIX philosophy is writing small programs and chaining them together to do new and interesting things. Back in the early days, hard disk space was limited and write times were slow. Brian Kernighan wanted to maintain the philosophy while also not having to write a bunch of intermediate files that take up hard drive space. So, the UNIX pipe was born. A pipe take the stdout of the program on its left and feeds it to the stdin of the program on its write. Consider the command tee. It can be used as a replacement for the redirection operators because tee will both write to a file and output to standard out. It also has the added benefit that it doesn’t need to be the last command in the list. Meaning, that you can write an intermediate result and continue your piping.
$ ./program | tee output.txt # Overwrite
$ ./program | tee -a output.txt # Append
$ head output.txt | wc | head -n 1 # Multi pipes
$ ((head output.txt) | wc) | head -n 1 # Same as above
$ ./program | tee intermediate.txt | wc
The && and || operator are operators that execute a command sequentially. && only executes a command if the previous command succeeds, and || always executes the next command.
$ false && echo "Hello!"
$ true && echo "Hello!"
$ false || echo "Hello!"
Shell Tricks
Shell Tricks
Well each process gets its own dictionary of environment variables that are copied over to the child. Meaning, if the parent changes their environment variables it won’t be transferred to the child and vice versa. This is important in the fork-exec-wait trilogy if you want to exec a program with different environment variables than your parent (or any other process).
For example, you can write a C program that loops through all of the time zones and executes the date command to print out the date and time in all locals. Environment variables are used for all sorts of programs so modifying them is important.
Each thread uses a stack memory. The stack ‘grows downwards’ - if a function calls another function, then the stack is extended to smaller memory addresses. Stack memory includes non-static automatic (temporary) variables, parameter values and the return address. If a buffer is too small some data (e.g. input values from the user), then there is a real possibility that other stack variables and even the return address will be overwritten. The precise layout of the stack’s contents and order of the automatic variables is architecture and compiler dependent. However with a little investigative work we can learn how to deliberately smash the stack for a particular architecture.
The example below demonstrates how the return address is stored on the stack. For a particular 32 bit architecture Live Linux Machine, we determine that the return address is stored at an address two pointers (8 bytes) above the address of the automatic variable. The code deliberately changes the stack value so that when the input function returns, rather than continuing on inside the main method, it jumps to the exploit function instead.
// Overwrites the return address on the following machine:
// http://cs-education.github.io/sys/
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
void breakout() {
puts("Welcome. Have a shell...");
system("/bin/sh");
}
void input() {
void *p;
printf("Address of stack variable: %p\n", &p);
printf("Something that looks like a return address on stack: %p\n", *((&p)+2));
// Let's change it to point to the start of our sneaky function.
*((&p)+2) = breakout;
}
int main() {
printf("main() code starts at %p\n",main);
input();
while (1) {
puts("Hello");
sleep(1);
}
return 0;
}There are a lot of ways that computers tend to get around this.
0x43 0x61 0x74 0xe0 0xf9 0xbf 0x5f 0xff 0x7f 0x00
Warning: Authors are not responsible for any neuro-apoptosis caused by these “jokes.” - Groaners are allowed.
Q. How many system programmers does it take to change a lightbulb?
A. Just one but they keep changing it until it returns zero.
A. None they prefer an empty socket.
A. Well you start with one but actually it waits for a child to do all of the work.
Why did the baby system programmer like their new colorful blankie? It was multithreaded.
Why are your programs so fine and soft? I only use 400-thread-count or higher programs.
Where do bad student shell processes go when they die? Forking Hell.
Why are C programmers so messy? They store everything in one big heap.
A system programmer is…
Someone who knows sleepsort is a bad idea but still dreams of an excuse to use it.
Someone who never lets their code deadlock… but when it does, causes more problems than everyone else combined.
Someone who believes zombies are real.
Someone who doesn’t trust their process to run correctly without testing with the same data, kernel, compiler, RAM, filesystem size,file system format, disk brand, core count, CPU load, weather, magnetic flux, orientation, pixie dust, horoscope sign, wall color, wall gloss and reflectance, motherboard, vibration, illumination, backup battery, time of day, temperature, humidity, lunar position, sun-moon, co-position…
A system program …
Evolves until it can send email.
Evolves until it has the potential to create, connect and kill other programs and consume all possible CPU,memory,network,… resources on all possible devices but chooses not to. Today.
Please do not edit this wiki
This content is licensed under the coursebook licensing scheme. If you find any typos. Please file an issue or make a PR. Thank you!