-
-
Notifications
You must be signed in to change notification settings - Fork 31.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test_zlib: test_pair() and test_speech128() fail with s390x hardware accelerator #90781
Comments
test_pair() and test_speech128() tests of test_zlib fail on the s390x architecture if zlib uses the s390x hardware accelerator. RHEL8 downstream issues (with most details): Fedora downstream issues: The s390x has a hardware accelerator for zlib. Depending if the hardware accelerator is used or not, the output (compress data) is different. Also, test_zlib compress data in two different ways and then expect the same output. test_zlib pass with the software implementation which creates a single (final) compressed block. test_zlib fails with the hardware implementation which creates multiple compressed blocks (the last one is a final block). Another reason the output differs is the FHT/DHT heuristic. The zlib deflate algorithm can analyze the data distribution and decide whether it wants to use a fixed-Huffman table (FHT) or a dynamic-Huffman table (DHT) for the next block, but the accelerator can't. Furthermore, looking at data in software would kill the accelerator performance. Therefore the following heuristic is used on s390x: the first 4k are compressed with FHT and the rest of the data with DHT. So, compress() creates a single FHT block. compressobj() creates a FHT block, a DHT block and a trailing block. It is *not a bug* in zlib: the decompression gives back the original content as expected in all cases. The issue is that Python test_zlib makes too many assumptions on how "streamed" data should be compressed. The test expected that compressed data using different ways to call zlib would return the exact same compressed data. If an accelarator is used, it's not always the case. |
I checked os.uname() value on two buildbots, by looking at their test.pythoninfo: os.uname: posix.uname_result(sysname='Linux', nodename='ztcpip3.pok.ibm.com', release='3.10.0-1160.53.1.el7.s390x', version='#1 SMP Thu Dec 16 04:33:52 EST 2021', machine='s390x') os.uname: posix.uname_result(sysname='Linux', nodename='dje', release='5.10.0-11-s390x', version='#1 SMP Debian 5.10.92-1 (2022-01-18)', machine='s390x') So the machine is 's390x'. |
Even if Python 3.9 and 3.10 are also affected, I prefer to not backport the change since it's not ideal. |
The Python 3.7 package on Fedora also skips these two tests using --ignore option of regrtest:
|
The current fix skips all of the tests, rather than just the parts that fail. |
Right. In short, I copied the Fedora fix to Python as 9475dc0 because I know that it works on s390x. But I don't have access to a s390x machine with hardware accelerator, so I cannot develop a change which only skips the tests if hardware accelerator is present, or only skip the part which is affected by the hardware accelerator.
The initial scope of the issue is fixed, so I just close the issue. If you manage to write and test a better fix, please open a separated issue/PR. |
Code is originally from gh#python/cpython!31096, it was released upstream in 3.11.0. Fixes: gh#python#90781 Fixes: bpo-46623
Code is originally from gh#python/cpython!31096, it was released upstream in 3.11.0. Fixes: gh#python#90781 Fixes: bpo-46623 Patch: bpo-46623-skip-zlib-s390x.patch
Code is originally from gh#python/cpython!31096, it was released upstream in 3.11.0. Fixes: gh#python#90781 Fixes: bpo-46623 Patch: bpo-46623-skip-zlib-s390x.patch
Code is originally from gh#python/cpython!31096, it was released upstream in 3.11.0. Fixes: gh#python#90781 Fixes: bpo-46623 Patch: bpo-46623-skip-zlib-s390x.patch
Code is originally from gh#python/cpython!31096, it was released upstream in 3.11.0. Fixes: gh#python#90781 Fixes: bpo-46623 Patch: bpo-46623-skip-zlib-s390x.patch
Code is originally from gh#python/cpython!31096, it was released upstream in 3.11.0. Fixes: gh#python#90781 Fixes: bpo-46623 Patch: bpo-46623-skip-zlib-s390x.patch
Code is originally from gh#python/cpython!31096, it was released upstream in 3.11.0. Fixes: gh#python#90781 Fixes: bpo-46623 Patch: bpo-46623-skip-zlib-s390x.patch
Code is originally from gh#python/cpython!31096, it was released upstream in 3.11.0. Fixes: gh#python#90781 Fixes: bpo-46623 Patch: bpo-46623-skip-zlib-s390x.patch
Skip test_pair() and test_speech128() of test_zlib on s390x since they fail if zlib uses the s390x hardware accelerator. Code is originally from gh#python/cpython!31096, it was released upstream in 3.11.0. Fixes: gh#python#90781 Fixes: bpo-46623 Patch: bpo-46623-skip-zlib-s390x.patch
…, skip checking the compressed bytes (pythonGH-125042) This backports two commits: - pythonGH-31096 skipped the tests unconditionally - pythonGH-125042 skips only the possibly-failing assertion (cherry picked from commit cc5a225)
…ration, skip checking the compressed bytes (pythonGH-125042) This backports two commits: - pythonGH-31096 skipped the tests unconditionally - pythonGH-125042 skips only the possibly-failing assertion (cherry picked from commit d522856) Co-authored-by: Petr Viktorin <encukou@gmail.com>
…p checking the compressed bytes (GH-125042) (#125585) gh-125041: gh-90781: test_zlib: For s390x HW acceleration, skip checking the compressed bytes (GH-125042) This backports two commits: - GH-31096 skipped the tests unconditionally - GH-125042 skips only the possibly-failing assertion (cherry picked from commit cc5a225)
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: