test_faulthandler fails under ubsan

[benjaminp]

BPO	35225
Nosy	@gpshead, @benjaminp
PRs	bpo-35225: Cleanup test_faulthandler sanitizer skip logic. #11381 bpo-35225: Cleanup test_faulthandler sanitizer skip logic. #11381 bpo-35225: Cleanup test_faulthandler sanitizer skip logic. #11381 [3.7] bpo-35225: Cleanup test_faulthandler sanitizer skip logic. (GH-11381) #11382 [3.7] bpo-35225: Cleanup test_faulthandler sanitizer skip logic. (GH-11381) #11382 bpo-35225: Fix typos in test_faulthandler skipIfs for ubsan. #11386 bpo-35225: Fix typos in test_faulthandler skipIfs for ubsan. #11386 bpo-35225: Fix typos in test_faulthandler skipIfs for ubsan. #11386 bpo-35225: Fix typos in test_faulthandler skipIfs for ubsan. #11386 [3.7] bpo-35225: Fix typos in test_faulthandler skipIfs for ubsan. (GH-11386) #11387 [3.7] bpo-35225: Fix typos in test_faulthandler skipIfs for ubsan. (GH-11386) #11387 [3.7] bpo-35225: Fix typos in test_faulthandler skipIfs for ubsan. (GH-11386) #11387

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/gpshead'
closed_at = <Date 2018-12-31.04:35:36.572>
created_at = <Date 2018-11-13.03:04:30.670>
labels = ['3.8', 'build', '3.7']
title = 'test_faulthandler fails under ubsan'
updated_at = <Date 2018-12-31.04:35:36.571>
user = 'https://github.com/benjaminp'

bugs.python.org fields:

activity = <Date 2018-12-31.04:35:36.571>
actor = 'gregory.p.smith'
assignee = 'gregory.p.smith'
closed = True
closed_date = <Date 2018-12-31.04:35:36.572>
closer = 'gregory.p.smith'
components = ['Build']
creation = <Date 2018-11-13.03:04:30.670>
creator = 'benjamin.peterson'
dependencies = []
files = []
hgrepos = []
issue_num = 35225
keywords = ['patch', 'patch', 'patch']
message_count = 4.0
messages = ['329797', '332782', '332795', '332798']
nosy_count = 2.0
nosy_names = ['gregory.p.smith', 'benjamin.peterson']
pr_nums = ['11381', '11381', '11381', '11382', '11382', '11386', '11386', '11386', '11386', '11387', '11387', '11387']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue35225'
versions = ['Python 3.7', 'Python 3.8']

[benjaminp]

The UBsan buildbot is failing test_faulthandler. We should either spread some __attribute__((no_sanitize_undefined)) around or make the tests skip.

======================================================================
FAIL: test_enable_fd (test.test_faulthandler.FaultHandlerTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Lib/test/test_faulthandler.py", line 274, in test_enable_fd
    self.check_fatal_error("""
  File "/var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Lib/test/test_faulthandler.py", line 115, in check_fatal_error
    self.check_error(code, line_number, fatal_error, **kw)
  File "/var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Lib/test/test_faulthandler.py", line 108, in check_error
    output, exitcode = self.get_output(code, filename=filename, fd=fd)
  File "/var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Lib/test/test_faulthandler.py", line 73, in get_output
    self.assertEqual(output, '')
AssertionError: 'UndefinedBehaviorSanitizer:DEADLYSIGNAL\n[2619 chars]TING' != ''
Diff is 2693 characters long. Set self.maxDiff to None to see it.

======================================================================
FAIL: test_enable_file (test.test_faulthandler.FaultHandlerTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Lib/test/test_faulthandler.py", line 258, in test_enable_file
    self.check_fatal_error("""
  File "/var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Lib/test/test_faulthandler.py", line 115, in check_fatal_error
    self.check_error(code, line_number, fatal_error, **kw)
  File "/var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Lib/test/test_faulthandler.py", line 108, in check_error
    output, exitcode = self.get_output(code, filename=filename, fd=fd)
  File "/var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Lib/test/test_faulthandler.py", line 68, in get_output
    self.assertEqual(output, '')
AssertionError: 'UndefinedBehaviorSanitizer:DEADLYSIGNAL\n[2619 chars]TING' != ''
Diff is 2693 characters long. Set self.maxDiff to None to see it.

======================================================================
FAIL: test_sigfpe (test.test_faulthandler.FaultHandlerTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Lib/test/test_faulthandler.py", line 176, in test_sigfpe
    self.check_fatal_error("""
  File "/var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Lib/test/test_faulthandler.py", line 115, in check_fatal_error
    self.check_error(code, line_number, fatal_error, **kw)
  File "/var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Lib/test/test_faulthandler.py", line 110, in check_error
    self.assertRegex(output, regex)
AssertionError: Regex didn't match: '^Fatal Python error: Floating point exception\n\nCurrent thread 0x[0-9a-f]+ \\(most recent call first\\):\n  File "<string>", line 3 in <module>' not found in 'Modules/faulthandler.c:1059:11: runtime error: division by zero\nFatal Python error: Floating point exception\n\nCurrent thread 0x00007f14c5e81740 (most recent call first):\n  File "<string>", line 3 in <module>\nUndefinedBehaviorSanitizer:DEADLYSIGNAL\n==5027==ERROR: UndefinedBehaviorSanitizer: FPE on unknown address 0x006d000013a3 (pc 0x7f14c5355f9f bp 0x000002f485a0 sp 0x000002f698e8 T5027)\n    #0 0x7f14c5355f9e in gsignal (/lib/x86_64-linux-gnu/libpthread.so.0+0x10f9e)\n    #1 0x7f14c53560bf  (/lib/x86_64-linux-gnu/libpthread.so.0+0x110bf)\n    #2 0x829906 in faulthandler_sigfpe /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/./Modules/faulthandler.c:1059:11\n    #3 0x4707c3 in _PyMethodDef_RawFastCallKeywords /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Objects/call.c\n    #4 0x46e5d1 in _PyCFunction_FastCallKeywords /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Objects/call.c:730:14\n    #5 0x656805 in call_function /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Python/ceval.c:4570:9\n    #6 0x64b0d6 in _PyEval_EvalFrameDefault /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Python/ceval.c:3164:23\n    #7 0x6596be in _PyEval_EvalCodeWithName /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Python/ceval.c:3946:14\n    #8 0x6384ee in PyEval_EvalCodeEx /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Python/ceval.c:3975:12\n    #9 0x6384ee in PyEval_EvalCode /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Python/ceval.c:508\n    #10 0x6e816e in run_mod /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Python/pythonrun.c:1030:9\n    #11 0x6e816e in PyRun_StringFlags /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Python/pythonrun.c:954\n    #12 0x6e80bc in PyRun_SimpleStringFlags /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Python/pythonrun.c:450:9\n    #13 0x44db44 in pymain_run_command /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Modules/main.c:293:11\n    #14 0x44db44 in pymain_run_python /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Modules/main.c:1600\n    #15 0x44db44 in pymain_main /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Modules/main.c:1758\n    #16 0x44f29b in _Py_UnixMain /var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/Modules/main.c:1795:12\n    #17 0x7f14c4ba72e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)\n    #18 0x425499 in _start (/var/lib/buildbot/clang-ubsan/3.x.gps-clang-ubsan/build/python+0x425499)\n\nUndefinedBehaviorSanitizer can not provide additional info.\n==5027==ABORTING'

---

[gpshead]

These are all related to the output of the child process containing sanitizer stuff so the stdout/stderr based tests are failing.

i skipped two of these for memory sanitizers, the third looks easy enough to fix via the regex. PR attached.

[gpshead]

my test skipping logic is somehow not detecting my ubsan buildbot

https://buildbot.python.org/all/#/builders/135/builds/1108/steps/4/logs/stdio

TODO: figure that out.

otherwise the regex change fixed test_sigfpe.

[gpshead]

https://buildbot.python.org/all/#/builders/135/builds/1110

== Tests result: SUCCESS ==
404 tests OK.
10 slowest tests:

• test_tokenize: 3 min 43 sec
• test_concurrent_futures: 3 min 28 sec
• test_multiprocessing_spawn: 3 min 20 sec
• test_tools: 2 min 59 sec
• test_lib2to3: 2 min 23 sec
• test_multiprocessing_forkserver: 2 min 2 sec
• test_asyncio: 1 min 55 sec
• test_multiprocessing_fork: 1 min 28 sec
• test_subprocess: 1 min 15 sec
• test_io: 58 sec 708 ms
  14 tests skipped:
  test_devpoll test_gdb test_ioctl test_kqueue test_msilib
  test_ossaudiodev test_startfile test_tix test_tk test_ttk_guionly
  test_winconsoleio test_winreg test_winsound test_zipfile64