Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documenting emulator debugging process with GDB #1415

Merged
212 README.md
@@ -13,6 +13,7 @@ the RISC-V Rocket Core. For more information on Rocket Chip, please consult our
+ [Mapping a Rocket core down to an FPGA](#fpga)
+ [Pushing a Rocket core through the VLSI tools](#vlsi)
+ [How can I parameterize my Rocket chip?](#param)
+ [Debugging with GDB](#debug)
+ [Contributors](#contributors)

## <a name="quick"></a> Quick Instructions
@@ -447,6 +448,217 @@ you can create your own Configuration(s) and compose them with Config's ++ opera

Then you can build as usual with CONFIG=MyConfig.

## <a name="debug"></a> Debugging with GDB

### 1) Generating the Remote Bit-Bang (RBB) Emulator

The objective of this section is to use GNU debugger to debug RISC-V programs running on the emulator in the same fashion as in [Spike](https://github.com/riscv/riscv-isa-sim#debugging-with-gdb).

For that we need to add a Remote Bit-Bang client to the emulator. We can do so by extending our Config with JtagDTMSystem, which will add a DebugTransportModuleJTAG to the DUT and connect a SimJTAG module in the Test Harness. This will allow OpenOCD to interface with the emulator, and GDB can interface with OpenOCD. In the following example we added this Config extension to the DefaultConfig:

class DefaultConfigRBB extends Config(
new WithJtagDTMSystem ++ new WithNBigCores(1) ++ new BaseConfig)

class QuadCoreConfigRBB extends Config(
new WithJtagDTMSystem ++ new WithNBigCores(4) ++ new BaseConfig)

To build the emulator with `DefaultConfigRBB` configuration we use the command:

rocket-chip$ cd emulator
emulator$ CONFIG=DefaultConfigRBB make

We can also build a debug version capable of generating VCD waveforms using the command:

emulator$ CONFIG=DefaultConfigRBB make debug

By default the emulator is generated under the name `emulator-freechips.rocketchip.system-DefaultConfigRBB` in the first case and `emulator-freechips.rocketchip.system-DefaultConfigRBB-debug` in the second.

Please note that generated VCD waveforms can be very voluminous depending on the size of the .elf file (i.e. code size + debugging symbols). If you need only to execute your program and get output, you may need to strip the .elf file. Make sure you keep `tohost` and `fromhost` because these are necessary in the HTIF interface.

### 2) Compiling and executing a custom program using the emulator

We suppose that `helloworld` is our program, you can use `crt.S`, `syscalls.c` and the linker script `test.ld` to construct your own program, check examples stated in [riscv-tests](https://github.com/riscv/riscv-tests).

In our case we will use the following example:

```
char text[] = "Vafgehpgvba frgf jnag gb or serr!";
// Don't use the stack, because sp isn't set up.
volatile int wait = 1;
int main()
{
while (wait)
;
// Doesn't actually go on the stack, because there are lots of GPRs.
int i = 0;
while (text[i]) {
char lower = text[i] | 32;
if (lower >= 'a' && lower <= 'm')
text[i] += 13;
else if (lower > 'm' && lower <= 'z')
text[i] -= 13;
i++;
}
while (!wait)
;
}
```

First we can test if your program executes well in the simple version of emulator before moving to debugging in step 3 :

$ ./emulator-freechips.rocketchip.system-DefaultConfig helloworld

Additional verbose information (clock cycle, pc, instruction being executed) can be printed using the following command:

$ ./emulator-freechips.rocketchip.system-DefaultConfig +verbose helloworld 2>&1 | spike-dasm

Please note that execution time of the program on the emulator depends on executable size. Stripping the .elf executable will unsurprisingly make it run faster. For this you can use `$RISCV/bin/riscv64-unknown-elf-strip` tool to reduce the size. This is good for accelerating execution but not for debugging. Keep in mind that the HTIF communication interface between our system and the emulator relies on `tohost` and `fromhost` symbols to communicate. This is why you may get the following error when you try to run a totally stripped executable on the emulator:

This comment has been minimized.

Copy link
@mwachs5

mwachs5 May 15, 2018

Contributor

Now this note (about the size and stripping) exist in two places. Maybe you should remove the first one and keep this one?

This comment has been minimized.

Copy link
@mwachs5

mwachs5 May 15, 2018

Contributor

Also, minor nit, it's not really the execution time of the program, but the time it takes the emulator to load your program that depends on the size. Minor rewording would be:

Please note that the time it takes the emulator to load your program depends on executable size. Stripping the .elf executable will unsurprisingly make it run faster. For this you can use $RISCV/bin/riscv64-unknown-elf-strip tool to reduce the size. This is good for accelerating your simulation but not for debugging. Keep in mind that the HTIF communication interface between our system and the emulator relies on tohost and fromhost symbols to communicate. This is why you may get the following error when you try to run a totally stripped executable on the emulator:

This comment has been minimized.

Copy link
@mwachs5

mwachs5 May 15, 2018

Contributor

Actually, I re-read your justification for the two notes above, so if you want to keep the other note about VCD waveform size that's fine.

This comment has been minimized.

Copy link
@noureddine-as

noureddine-as May 15, 2018

Author Contributor

Thanks Megan. I rearranged the idea in the 'Compiling and executing' part. I also deleted redundancies. Added also an example of using the -debug version.
I guess the text is now more coherent.
Thank you for the reviews. Glad to participate :)


$ ./emulator-freechips.rocketchip.system-DefaultConfig totally-stripped-helloworld
This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
Listening on port 46529
warning: tohost and fromhost symbols not in ELF; can't communicate with target

To resolve this, we need to strip all the .elf executable but keep `tohost` and `fromhost` symbols using the following command:

$riscv64-unknown-elf-strip -s -Kfromhost -Ktohost helloworld

More details on the GNU strip tool can be found [here](https://www.thegeekstuff.com/2012/09/strip-command-examples/).

The interest of this step is to make sure your program executes well. To perform debugging you need the original unstripped version, as explained in step 3.

### 3) Launch the emulator

First, do not forget to compile your program with `-g -Og` flags to provide debugging support as explained [here](https://github.com/riscv/riscv-isa-sim#debugging-with-gdb).

We can then launch the Remote Bit-Bang enabled emulator with:

./emulator-freechips.rocketchip.system-DefaultConfigRBB +jtag_rbb_enable=1 --rbb-port=9823 helloworld

This comment has been minimized.

Copy link
@mwachs5

mwachs5 May 11, 2018

Contributor

There is no real point to the helloworld argument to the emulator, because you load it later through the debugger. What happens if you leave it off?

This comment has been minimized.

Copy link
@mwachs5

mwachs5 May 11, 2018

Contributor

You should note that if you leave off the --rbb-port argument, a random port will be assigned.

This comment has been minimized.

Copy link
@noureddine-as

noureddine-as May 11, 2018

Author Contributor

Actually without the program argument, the emulator simply shows the help message.

This comment has been minimized.

Copy link
@noureddine-as

noureddine-as May 11, 2018

Author Contributor

Default RBB if no -rbb-port note added.

This comment has been minimized.

Copy link
@mwachs5

mwachs5 May 11, 2018

Contributor

Actually without the program argument, the emulator simply shows the help message.

Good to know. Maybe we should add a note here that it's not going to load your program, or that that argument doesn't matter, or use the term "dummy" ( I believe the latter is what the Debug Regressions actually do). Because it doesn't matter what you put there, since FESVR isn't attached it's not going to load that program.

This comment has been minimized.

Copy link
@noureddine-as

noureddine-as May 11, 2018

Author Contributor

Okay. I prefer to let the helloworld there so that new people don't get confused when they don't use RBB or something, but notified the fact that it is loaded from GDB (in contrast with Spike) and that's it's a dummy arg. Thanks for clarification.

This emulator compiled with JTAG Remote Bitbang client. To enable, use +jtag_rbb_enable=1.
Listening on port 9823
Attempting to accept client socket

You can also use the `emulator-freechips.rocketchip.system-DefaultConfigRBB-debug` version instead if you would like to generate VCD waveforms.

Please note that if the argument `--rbb-port` is not passed, a default free TCP port on your computer will be chosen randomly.

Please note also that when debugging with GDB, the .elf file is not actually loaded by the FESVR. In contrast with Spike, it must be loaded from GDB as explained in step 5. So the `helloworld` argument may be replaced by any dummy name.

### 4) Launch OpenOCD

You will need a RISC-V Enabled OpenOCD binary. This is installed with riscv-tools in `$(RISCV)/bin/openocd`, or can be compiled manually from riscv-openocd. OpenOCD requires a configuration file, in which we define the RBB port we will use, which is in our case `9823`.

$ cat cemulator.cfg
interface remote_bitbang
remote_bitbang_host localhost
remote_bitbang_port 9823

set _CHIPNAME riscv
jtag newtap $_CHIPNAME cpu -irlen 5
set _TARGETNAME $_CHIPNAME.cpu
target create $_TARGETNAME riscv -chain-position $_TARGETNAME
gdb_report_data_abort enable
init
halt
Then we launch OpenOCD in another terminal using the command
$(RISCV)/bin/openocd -f ./cemulator.cfg
Open On-Chip Debugger 0.10.0+dev-00112-g3c1c6e0 (2018-04-12-10:40)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
Warn : Adapter driver 'remote_bitbang' did not declare which transports it allows; assuming legacy JTAG-only
Info : only one transport option; autoselect 'jtag'
Info : Initializing remote_bitbang driver
Info : Connecting to localhost:9823
Info : remote_bitbang driver initialized
Info : This adapter doesn't support configurable speed
Info : JTAG tap: riscv.cpu tap/device found: 0x00000001 (mfg: 0x000 (<invalid>), part: 0x0000, ver: 0x0)
Info : datacount=2 progbufsize=16
Info : Disabling abstract command reads from CSRs.
Info : Disabling abstract command writes to CSRs.
Info : [0] Found 1 triggers
Info : Examined RISC-V core; found 1 harts
Info : hart 0: XLEN=64, 1 triggers
Info : Listening on port 3333 for gdb connections
Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections
A `-d` flag can be added to the command to show further debug information.
### 5) Launch GDB
In another terminal launch GDB and point to the elf file you would like to load then run it with the debugger (in this example, `helloworld`):
$ riscv64-unknown-elf-gdb helloworld
GNU gdb (GDB) 8.0.50.20170724-git
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-pc-linux-gnu --target=riscv64-unknown-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./proj1.out...done.
(gdb)
Compared to Spike, the C Emulator is very slow, so several problems may be encountered due to timeouts between issuing commands and response from the emulator. To solve this problem, we increase the timeout with the GDB `set remotetimeout` command.
After that we load our program by performing a `load` command. This automatically sets the `$PC` to the `_start` symbol in our .elf file.
(gdb) set remotetimeout 2000
(gdb) target remote localhost:3333
Remote debugging using localhost:3333
0x0000000000010050 in ?? ()
(gdb) load
Loading section .text.init, size 0x2cc lma 0x80000000
Loading section .tohost, size 0x48 lma 0x80001000
Loading section .text, size 0x98c lma 0x80001048
Loading section .rodata, size 0x158 lma 0x800019d4
Loading section .rodata.str1.8, size 0x20 lma 0x80001b30
Loading section .data, size 0x22 lma 0x80001b50
Loading section .sdata, size 0x4 lma 0x80001b74
Start address 0x80000000, load size 3646
Transfer rate: 40 bytes/sec, 520 bytes/write.
(gdb)
Now we can proceed as with Spike, debugging works in a similar way:
(gdb) print wait
$1 = 1
(gdb) print wait=0
$2 = 0
(gdb) print text
$3 = "Vafgehpgvba frgf jnag gb or serr!"
(gdb) c
Continuing.
^C
Program received signal SIGINT, Interrupt.
main (argc=0, argv=<optimized out>) at src/main.c:33
33 while (!wait)
(gdb) print wait
$4 = 0
(gdb) print text
$5 = "Instruction sets want to be free!"
(gdb)

This comment has been minimized.

Copy link
@mwachs5

mwachs5 May 11, 2018

Contributor

I think you should add a note either here or at the beginning that will note what to do to make it waveform-enabled (building the -debug emulator target)

This comment has been minimized.

Copy link
@noureddine-as

noureddine-as May 11, 2018

Author Contributor

Added in 1).

  • Pointed out the fact that VCD waveforms can be voluminous depending on program size.
  • I'll add a section for executable stripping may be? I guess it affects the execution time, do you agree @mwachs5 ?.

This comment has been minimized.

Copy link
@mwachs5

mwachs5 May 11, 2018

Contributor

Well, the program size and duration. I don't think it's really that relevant, people already know that VCD waveforms can be big and scale with runtime, so I woudl probably remove both those notes.

This comment has been minimized.

Copy link
@noureddine-as

noureddine-as May 11, 2018

Author Contributor

You're right experienced people could already know this. But when it comes to beginners discovering all these tools, it is not the case. An example is a friend who tried to boot BBL (that has now several MiB of size) and he thought it just doesn't work, but he found out that it just takes hours to run. In these cases gaining more space means gaining hours of waiting.
Anyway, I didn't test it personally. Is it the case normally?

This comment has been minimized.

Copy link
@mwachs5

mwachs5 May 11, 2018

Contributor

I mean, the larger program -> longer runtime is sort of orthogonal to longer runtime -> larger VCD file. So if you want to point out that it's going to be very slow to load a large program this way, I would separate it from the note that the generated waveforms with the -debug version of emulator may be large.

This comment has been minimized.

Copy link
@noureddine-as

noureddine-as May 12, 2018

Author Contributor

I understand your point of view and I agree.
As you said VCD size increase is trivial. Though, I pointed out the relation {larger program -> longer runtime} and how to reduce it as well as how to test the program on the emulator event after beginning debugging.
Again, I know these are basic ideas, but documenting can save new people some time to understand what's going on.
Thanks a lot.

This comment has been minimized.

Copy link
@noureddine-as

noureddine-as May 15, 2018

Author Contributor

I guess it's okay now. Any other recommendations @mwachs5 ?

Further information about GDB debugging is available [here](https://sourceware.org/gdb/onlinedocs/gdb/) and [here](https://sourceware.org/gdb/onlinedocs/gdb/Remote-Debugging.html#Remote-Debugging).
## <a name="contributors"></a> Contributors
Can be found [here](https://github.com/ucb-bar/rocket-chip/graphs/contributors).
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.