test_dtrace failed

[sayno996]

BPO	36702
Nosy	@vstinner, @willrogers3
Files	test.log: The log file for test_dtrace test failure in make test make_test-clipped.txt test_dtrace_stap_bug.txt

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-04-23.06:48:53.045>
labels = ['3.8', 'build', '3.7']
title = 'test_dtrace failed'
updated_at = <Date 2020-12-23.12:17:15.401>
user = 'https://bugs.python.org/sayno996'

bugs.python.org fields:

activity = <Date 2020-12-23.12:17:15.401>
actor = 'brian.costlow'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build']
creation = <Date 2019-04-23.06:48:53.045>
creator = 'sayno996'
dependencies = []
files = ['48284', '48687', '49696']
hgrepos = []
issue_num = 36702
keywords = []
message_count = 3.0
messages = ['340692', '355677', '383645']
nosy_count = 5.0
nosy_names = ['vstinner', 'sayno996', 'willrogers3', 'tchan', 'brian.costlow']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue36702'
versions = ['Python 3.7', 'Python 3.8']

[sayno996]

I install Python 3.7.3 on CentOS 7.6. However, when I run "make test", I got a failure on test_dtrace as:

Ran 8 tests in 4.752s

FAILED (failures=6, skipped=2)
test test_dtrace failed
test_dtrace failed

== Tests result: FAILURE ==

1 test failed:
test_dtrace

Total duration: 4 sec 771 ms
Tests result: FAILURE

[willrogers3]

I also encountered this on Scientific Linux 7.4 (Linux 3.10.0-693.11.6.el7.x86_64) while trying to build Python 3.8.0.

I think it is due to differences in versions of 'dtrace' on different platforms.
On my MacOS-X v10.14.6 (Mojave) system, the 'dtrace' command has a -q (quiet the output) option.
On SL 7.4, it does not. The options are quite different.

This is output and versions from my test run:

./python  ./Tools/scripts/run_tests.py 
== CPython 3.8.0 (default, Oct 29 2019, 14:37:09) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]
== Linux-3.10.0-693.11.6.el7.x86_64-x86_64-with-glibc2.17 little-endian
== cwd: /opt/Python-3.8.0/build/test_python_30482
== CPU count: 2
== encodings: locale=UTF-8, FS=utf-8

Below is an error from the test log that shows the 'q' option being rejected from 'dtrace' (linux'):

0:10:16 load avg: 1.79 Re-running test_dtrace in verbose mode
skipped "dtrace(1) failed: ('/usr/bin/dtrace', 'invalid option', '-q')\nUsage /usr/bin/dtrace [--help] [-h | -G] [-C [-I<Path>]] -s File.d [-o <File>]"

Assuming 'dtrace' is run as a sub-process, the different versions could be producing very different results.

I attached a (clipped) version of my 'make test' output, it is essentially the same as sayno996's results.
There are also some error from 'test_nis" which can be ignoed for this issue.

I hope this helps.

[briancostlow]

There are actually two different issues here.

dtrace -q will not work on Fedora-based linux (haven't tried elsewhere) and that probably should be corrected, but that is NOT what causes the test fail.

The tests' setup checks whether dtrace is usuable, and since it is not, those tests are skipped.

However, stap IS usable, so those tests run.

test.test_dtrace.SystemTapOptimizedTests.test_line will always fail because it expects files in dtracedata (line.stp and line.stp.expected) that are not there.

I've attached a file showing isolated runs of test_dtrace on a newly built Python 3.8.6 on two Centos 7 systems.

The first is against the official Centos 7 Docker container, and stap fails because Linuxkit kernel modules are not installed. The test_dtrace check for a working stap fails, and all 4 tests are skipped.

The second is against a virtualized Centos 7 where the kernel modules are properly installed and stap works.

I don't see how test_dtrace ever passes on a system with a working /bin/stap command.

[kumaraditya303]

Closing as 3.8.0 is not supported anymore (security fixes only).

[vstinner]

Python has RHEL7 buildbot workers on test_dtrace pass on them, even on the 3.8 branch. Recent build ("Finished 14 days ago"):
https://buildbot.python.org/all/#/builders/415/builds/72

0:00:45 load avg: 5.00 [210/423] test_dtrace passed

On Fedora 36, test_dtrace seems to skip most tests:

setUpClass (test.test_dtrace.DTraceNormalTests) ... skipped 'dtrace(1) failed: /usr/bin/dtrace invalid option -q\nUsage /usr/bin/dtrace [--help] [-h | -G] [-C [-I<Path>]] -s File.d [-o <File>]'
setUpClass (test.test_dtrace.DTraceOptimizedTests) ... skipped 'dtrace(1) failed: /usr/bin/dtrace invalid option -q\nUsage /usr/bin/dtrace [--help] [-h | -G] [-C [-I<Path>]] -s File.d [-o <File>]'
setUpClass (test.test_dtrace.SystemTapNormalTests) ... skipped 'stap(1) failed: You are trying to run systemtap as a normal user.\nYou should either be root, or be part of the group "stapusr" and possibly one of the groups "stapsys" or "stapdev".  [man stap]\nAlternatively, you may specify --runtime=dyninst for userspace probing.\nTry \'--help\' for more information. [man stap]\nA message like [man foo] means for more info, run % man foo'
setUpClass (test.test_dtrace.SystemTapOptimizedTests) ... skipped 'stap(1) failed: You are trying to run systemtap as a normal user.\nYou should either be root, or be part of the group "stapusr" and possibly one of the groups "stapsys" or "stapdev".  [man stap]\nAlternatively, you may specify --runtime=dyninst for userspace probing.\nTry \'--help\' for more information. [man stap]\nA message like [man foo] means for more info, run % man foo'

If you can still reproduce the issue on a recent CentOS Stream on a recent Python version, please open a new issue referring to this one ;-)