Use paxtest for validating grsecurity config

[conorsch]

The paxtest package can be used to validate paxctl flags, used as part of the grsecurity configuration. The paxtest package has been implemented in a feature branch via the app-test role (see 247b997), but a better place for it would be in the grsecurity role, since that role applies to both *-staging hosts. There are already serverspec tests written for checking paxtest output, but the tests are currently disabled (see 837d6d3).

As long as there are no concerns with including paxtest in staging and production, the package could be used for a post-validation of a working setup. It's also quite useful for checking for regressions in virtualbox testing during config management changes to the *-staging hosts.

Example output

Running paxtest on a default Ubuntu kernel yields output like this:

Executable anonymous mapping             : Killed
Executable bss                           : Killed
Executable data                          : Killed
Executable heap                          : Killed
Executable stack                         : Killed
Executable shared library bss            : Killed
Executable shared library data           : Killed
Executable anonymous mapping (mprotect)  : Vulnerable
Executable bss (mprotect)                : Vulnerable
Executable data (mprotect)               : Vulnerable
Executable heap (mprotect)               : Vulnerable
Executable stack (mprotect)              : Vulnerable
Executable shared library bss (mprotect) : Vulnerable
Executable shared library data (mprotect): Vulnerable
Writable text segments                   : Vulnerable
Anonymous mapping randomisation test     : 28 bits (guessed)
Heap randomisation test (ET_EXEC)        : 14 bits (guessed)
Heap randomisation test (PIE)            : 28 bits (guessed)
Main executable randomisation (ET_EXEC)  : 28 bits (guessed)
Main executable randomisation (PIE)      : 28 bits (guessed)
Shared library randomisation test        : 28 bits (guessed)
Stack randomisation test (SEGMEXEC)      : 28 bits (guessed)
Stack randomisation test (PAGEEXEC)      : 28 bits (guessed)
Arg/env randomisation test (SEGMEXEC)    : 20 bits (guessed)
Arg/env randomisation test (PAGEEXEC)    : 20 bits (guessed)
Randomization under memory exhaustion @~0: 28 bits (guessed)
Randomization under memory exhaustion @0 : 28 bits (guessed)
Return to function (strcpy)              : paxtest: return address contains a NULL byte.
Return to function (memcpy)              : Killed
Return to function (strcpy, PIE)         : paxtest: return address contains a NULL byte.
Return to function (memcpy, PIE)         : Killed

Whereas on a grsecurity kernel:

Executable anonymous mapping             : Killed
Executable bss                           : Killed
Executable data                          : Killed
Executable heap                          : Killed
Executable stack                         : Killed
Executable shared library bss            : Killed
Executable shared library data           : Killed
Executable anonymous mapping (mprotect)  : Killed
Executable bss (mprotect)                : Killed
Executable data (mprotect)               : Killed
Executable heap (mprotect)               : Killed
Executable stack (mprotect)              : Killed
Executable shared library bss (mprotect) : Killed
Executable shared library data (mprotect): Killed
Writable text segments                   : Killed
Anonymous mapping randomisation test     : 29 bits (guessed)
Heap randomisation test (ET_EXEC)        : 23 bits (guessed)
Heap randomisation test (PIE)            : 35 bits (guessed)
Main executable randomisation (ET_EXEC)  : 29 bits (guessed)
Main executable randomisation (PIE)      : 29 bits (guessed)
Shared library randomisation test        : 29 bits (guessed)
Stack randomisation test (SEGMEXEC)      : 35 bits (guessed)
Stack randomisation test (PAGEEXEC)      : 35 bits (guessed)
Arg/env randomisation test (SEGMEXEC)    : 39 bits (guessed)
Arg/env randomisation test (PAGEEXEC)    : 39 bits (guessed)
Randomization under memory exhaustion @~0: 29 bits (guessed)
Randomization under memory exhaustion @0 : 29 bits (guessed)
Return to function (strcpy)              : paxtest: return address contains a NULL byte.
Return to function (memcpy)              : Killed
Return to function (strcpy, PIE)         : paxtest: return address contains a NULL byte.
Return to function (memcpy, PIE)         : Killed

[redshiftzero]

Looks like we now have this in testinfra in securedrop/testinfra/common/test_grsecurity.py. Closing!

[conorsch]

Unfortunately those checks aren't actually running, they're being skipped conditionally. That's a tiny edit, so I'll knock it out in discrete commit.

Note that the grsecurity checks only run in local staging, not in CI (yet).

[conorsch]

We still have the paxtest checks in the config, but they're still being skipped, since paxtest isn't installed in the server VMs. As soon as we start installing paxtest, the checks should run automatically—and presumably pass. 😄 There's the question of whether we want to install paxtest only on staging VMs, or everywhere—I'd strongly encourage everywhere, which dovetails with work like #5318.

As we work on Focal support, particularly creating a new metapackage, it'd be great to have the additional validation of these tests running. Let's just add the "paxtest" package to the grsec install tasks, so it's available immediately in test environments. We can consider marking it as a dependency of "securedrop-grsec" as well.

[conorsch]

Given that our QA matrix includes running these commands manually, we should indeed click the tests back on soon, to save developers some time, now that we have testinfra on prod (#5318). Some things have changed since the tests were originally written, though, so I'm noting that here. Under a Xenial host with paxtest=1:0.9.11-2, output will be:

Return to function (strcpy)              : paxtest: return address contains a NULL byte.
Return to function (memcpy)              : Killed
Return to function (strcpy, PIE)         : paxtest: return address contains a NULL byte.
Return to function (memcpy, PIE)         : Killed

However, on Foal with paxtest=1:0.9.14-2, you can expect to see:

Return to function (strcpy)              : paxtest: return address contains a NULL byte.
Return to function (memcpy)              : Vulnerable
Return to function (strcpy, PIE)         : paxtest: return address contains a NULL byte.
Return to function (memcpy, PIE)         : Vulnerable

More recent versions of paxtest will show "Vulnerable" on the memcpy tests. You can try downloading the Focal deb onto a Xenial host and rerunning, you'll see the output change. Looks like the change was intentional, as the changelog for 0.9.14 states:

paxtest (0.9.14-1) stable; urgency=low

    Various fixes/cleanups including ensuring an overflowing strcpy/memcpy isn't optimized away, from Mathias Krause

See some relevant (but much older) discussion of these checks in https://forums.grsecurity.net/viewtopic.php?t=1420#p5560. Easy enough to update the tests to match, but stating explicitly here that the "Vulnerable" results on the memcpy and memcpy, PIE tests are "expected" under Focal.