Avocado has two different types of GDB support that complement each other:
- Transparent execution of executables inside the GNU Debugger. This takes standard and possibly unmodified tests that uses the :mod:`avocado.utils.process` APIs for running processes. By using a command line option, the executable is run on GDB. This allows the user to interact with GDB, but to the test itself, things are pretty much transparent.
- The :mod:`avocado.utils.gdb` APIs that allows a test to interact with GDB, including setting a executable to be run, setting breakpoints or any other types of commands. This requires a test written with that approach and API in mind.
Tip
Even though this section describes the use of the Avocado GDB
features, which allow live debugging of binaries inside Avocado
tests, it's also possible to debug some application offline by
using tools such as rr. Avocado ships
with an example wrapper script (to be used with --wrapper
) for
that purpose.
This feature adds a few command line options to the Avocado run
command:
$ avocado run --help ... GNU Debugger support: --gdb-run-bin EXECUTABLE[:BREAKPOINT] Run a given executable inside the GNU debugger, pausing at a given breakpoint (defaults to "main") --gdb-prerun-commands EXECUTABLE:COMMANDS After loading an executable in GDB, but before actually running it, execute the GDB commands in the given file. EXECUTABLE is optional, if omitted COMMANDS will apply to all executables --gdb-coredump {on,off} Automatically generate a core dump when the inferior process received a fatal signal such as SIGSEGV or SIGABRT ...
To get started you want to use --gdb-run-bin
, as shown in the example bellow.
The simplest way is to just run
avocado run --gdb-run-bin=doublefree examples/tests/doublefree.py
, which
wraps each executed executable with name doublefree
inside GDB server and
stops at the executable entry point.
Optionally you can specify single breakpoint using
--gdb-run-bin=doublefree:$breakpoint
(eg: doublefree:1
) or just
doublefree:
to stop only when an interruption happens (eg: SIGABRT).
It's worth mentioning that when breakpoint is not reached, the test finishes
without any interruption. This is helpful when you identify regions where you
should never get in your code, or places which interests you and you can run
your code in production and GDB variants. If after a long time you get to this
place, the test notifies you and you can investigate the problem. This is
demonstrated in examples/tests/doublefree_nasty.py
test. To unveil the
power of Avocado, run this test using:
avocado run --gdb-run-bin=doublefree: examples/tests/doublefree_nasty.py --gdb-prerun-commands examples/tests/doublefree_nasty.py.data/gdb_pre --mux-yaml examples/tests/doublefree_nasty.py.data/iterations.yaml
which executes 100 iterations of this test while setting all breakpoints from
the examples/tests/doublefree_nasty.py.data/gdb_pre
file (you can specify
whatever GDB supports, not only breakpoints).
As you can see this test usually passes, but once in a while it gets into the problematic area. Imagine this is very hard to spot (dependent on HW registers, ...) and this is one way to combine regular testing and the possibility of debugging hard-to-get parts of your code.
Currently, when using the Avocado GDB plugin, that is, when using the --gdb-run-bin option, there are some caveats you should be aware of:
- It is not currently compatible with Avocado's --output-check-record feature
- There's no way to perform proper input to the process, that is, manipulate its STDIN
- The process STDERR content is mixed with the content generated by gdbserver on its own STDERR (because they are in fact, the same thing)
But, you can still depend on the process STDOUT, as exemplified by this fictional test:
from avocado import Test from avocado.utils import process class HelloOutputTest(Test): def test(self): result = process.run("/path/to/hello", ignore_status=True) self.assertIn("hello\n", result.stdout)
If run under GDB or not, result.stdout behavior and content is expected to be the same.
There are a two basic reasons for the mentioned caveats:
- The architecture of Avocado's GDB feature
- GDB's own behavior and limitations
When using the Avocado GDB plugin, that is, --gdb-run-bin, Avocado runs a gdbserver instance transparently and controls it by means of a gdb process. When a given event happens, say a breakpoint is reached, it disconnects its own gdb from the server, and allows the user to use a standard gdb to connect to the gdbserver. This provides a natural and seamless user experience.
But, gdbserver has some limitations at this point, including:
- Not being able to set a controlling tty
- Not separating its own STDERR content from the application being run
These limitations are being addressed both on Avocado and GDB, and will be resolved in future Avocado versions.
If the application you're running as part of your test can read input from alternative sources (including devices, files or the network) and generate output likewise, then you should not be further limited.
Another current limitation is the use of avocado-virt and avocado GDB support.
The supported API for transparent debugging is currently limited to :func:`avocado.utils.process.run`, and does not cover advanced uses of the :class:`avocado.utils.process.SubProcess` class. The avocado-virt extension, though, uses :class:`avocado.utils.process.SubProcess` class to execute qemu in the background.
This limitation will be addressed in future versions of avocado and avocado-virt.
Avocado's GDB module, provides three main classes that lets a test writer interact with a gdb process, a gdbserver process and also use the GDB remote protocol for interaction with a remote target.
Please refer to :mod:`avocado.utils.gdb` for more information.
Take a look at examples/tests/modify_variable.py
test:
def test(self): """ Execute 'print_variable'. """ path = os.path.join(self.workdir, 'print_variable') app = gdb.GDB() app.set_file(path) app.set_break(6) app.run() self.log.info("\n".join(app.read_until_break())) app.cmd("set variable a = 0xff") app.cmd("c") out = "\n".join(app.read_until_break()) self.log.info(out) app.exit() self.assertIn("MY VARIABLE 'A' IS: ff", out)
You can see that instead of running the executable using
process.run
we invoke :class:`avocado.utils.gdb.GDB`. This allows
us to automate the interaction with the GDB in means of setting
breakpoints, executing commands and querying for output.
When you check the output (--show-job-log
) you can see that despite
declaring the variable as 0, ff is injected and printed instead.