Howto: Debugging Rumprun with gdb

Martin Lucina edited this page Mar 15, 2016 · 3 revisions
Clone this wiki locally

Generally speaking, use rumprun [platform] -p -D [port] and target remote:[port] in gdb.

Debugging 64-bit unikernels under KVM/QEMU

Debugging a 64-bit rumprun unikernel under KVM/QEMU requires the following workaround due to a GDB issue:

  1. Start the unikernel, leaving it paused and waiting for GDB to connect:
rumprun [kvm|qemu] -p -D 1234 [...]
  1. Run the following GDB command:
gdb -q -ex "target remote:1234" -ex "hbreak x86_boot" -ex "continue" -ex "disconnect" -ex "quit" unikernel.bin

This will cause the unikernel to proceed past the transition from 32-bit mode to long mode and remain in a paused state. 3. Re-launch GDB a second time as you would normally.

Common gdb commands

  • disas or disas startaddr endaddr - disassembles around current PC or specific address
  • info locals/args/variables - prints out local variables, function arguments, all variables...
  • info registers - prints out all registers and their values
  • break file:linenr - sets a breakpoint on specific file/line
  • c - continue execution
  • n - step to next command in current stack frame
  • si - step by single instruction
  • ni - step by single instruction but if it is a function call, step until it returns
  • l - prints out few lines of code around current PC
  • p $reg/variable - prints values of registers or variables
  • x address - print value at some address. Look at docs for more details on formatting

Debugging example

We'll walk through debugging hello_world Go application. Process is the same whether you debug C, Go, Erlang or any other language/setup.

Before running the kernel, start gdb in one terminal, from directory where the binary is located:

cd $HOME/gorump/examples/hello_world
gdb -ex 'target remote:1234' hello.bin

In another terminal, run the kernel:

rumprun xen -D 1234 -p -i hello.bin

gdb prompt should stop on 0x0000 and wait for input:

	info "(gdb)Auto-loading safe path"
Remote debugging using :1234
0x0000000000000000 in _text ()

We can set a breakpoint on runtime1.go:77, which is goenvs_unix function in Go runtime. After that, we use c to run until the breakpoint.

(gdb) break runtime1.go:77
Breakpoint 2 at 0x4ac89: file /usr/local/go/src/runtime/runtime1.go, line 77.
(gdb) c

Breakpoint 1, runtime.goenvs_unix () at /usr/local/go/src/runtime/runtime1.go:76
76		n := int32(0)

Execution of the kernel stops at this point and gdb waits for our input. We can investigate the surrounding code:

(gdb) l
72	func goenvs_unix() {
73		// TODO(austin): ppc64 in dynamic linking mode doesn't
74		// guarantee env[] will immediately follow argv.  Might cause
75		// problems.
76		n := int32(0)
77		for argv_index(argv, argc+1+n) != nil {
78			n++
79		}

argv and argc are global variables in runtime at this point, so we can't get their value with p argc but need to specify exactly:

(gdb) p 'runtime.argc'
$1 = 1
(gdb) p 'runtime.argv'
$2 = (uint8 **) 0x3d5250 <kludge_argv>

We see that argv is a double pointer, so we can unpack that and look at the value:

(gdb) x /1a 'runtime.argv'          # /1a means "one as pointer" so it automatically reads what's on that address
0x3d5250 <kludge_argv>:	0x26d520
(gdb) x /1s 0x26d520                # /1s means "one string on that location"
0x26d520:	"argument"

If we were to debug the stack of main (which is where argc and argv reside) we can use x to get multiple values:

(gdb) x /16w 0x3d5240
0x3d5240 <kludge_argc>:	0x00000001	0x00000000	0x00000000	0x00000000
0x3d5250 <kludge_argv>:	0x0026d520	0x00000000	0x0026d525	0x00000000
0x3d5260 <envp>:	0x0026d52b	0x00000000	0x00000000	0x00000000
0x3d5270:	0x00000000	0x00000000	0x00000000	0x00000000

We can also debug our application of course. HelloWorld has a single .go file, hello.go, so we set a breakpoint:

(gdb) break hello.go:11
Breakpoint 4 at 0x176b7: file /home/ubuntu/gorump/examples/hello_world/hello.go, line 11.

Then, we just c to continue running until the next breakpoint, which should be the one in hello.go

(gdb) c

Breakpoint 4, main.damain () at /home/ubuntu/gorump/examples/hello_world/hello.go:11
11		fmt.Println("Hello, Rumprun.  This is Go.")
(gdb) l
6	func main() {
7	}
9	//export damain
10	func damain() {
11		fmt.Println("Hello, Rumprun.  This is Go.")
12	}

Again, we can investigate variables, stack, code or machine instructions...