Test failures on 32-bit systems due to differences in NX page protection

[GoogleCodeExporter]

I mentioned this problem on
http://codereview.chromium.org/2074003/show, but I am filing a bug so
that it doesn't get lost.

Currently some of seccomp-sandbox's tests fail on 32-bit systems.

On my netbook, running 32-bit Ubuntu Karmic, two tests failed with SIGSEGV:
test_sa_flags
test_segv_resethand
(NX page protection works on this system.)

On another machine, running 32-bit GHardy, just test_sa_flags failed,
again with SIGSEGV.  (NX page protection doesn't work on this system.)

The tests are fine on the two 64-bit machines I tested on.

From sandbox.cc:

     // Non-executable version of the restorer function. We use this to
     // trigger a SEGV upon returning from the user's signal handler, giving
     // us an ability to clean up prior to returning from the SEGV handler.

I don't think this will work on systems where no-execute page
protection doesn't work, i.e. older kernels and older hardware.  This
restorer function will run and so the signal handler's counter won't
be decremented.  You can verify this by linking the tests with
-Wl,-z,execstack (this option is badly named because it doesn't only
affect the stack).

This explains the test_sa_flags failure.

The test_segv_resethand failure seems a bit odder.  The signal
handler's "ret" instruction jumps to non-executable code which causes
a SIGSEGV.  But when I examined this with "strace -i" and gdb, the
"ret" is shown as the source of the fault, rather than the address
that "ret" jumps to (which is what the code expects).  It looks like
the reported %eip varies between CPUs or kernel versions.

Original issue reported on code.google.com by mseaborn@chromium.org on 27 Sep 2010 at 1:28