get_peer_creds failed in two tests for ppc64le

[michelmno]

❓ I'm submitting a ...

• 🐞 bug report
• 🐣 feature request
• ❓ question about the decisions made in the repository

🐞 Describe the bug. What is the current behavior?

cheroot has two failing tests with same reported error when executing on ppc64le.

[  171s] cheroot/test/test_server.py::test_peercreds_unix_sock FAILED             [ 88%]
[  171s] cheroot/test/test_server.py::test_peercreds_unix_sock_with_lookup FAILED [ 89%]
...
[  177s] ___________________________ test_peercreds_unix_sock ___________________________
...
[  177s] E           HTTPError: 500 Server Error: Internal Server Error for url: http+unix://%2Ftmp%2Ftmp3WSNw9/peer_creds/ids
...
[  177s]   File "/home/abuild/rpmbuild/BUILD/cheroot-6.5.5/cheroot/server.py", line 1399, in get_peer_creds
[  177s]     pid, uid, gid = struct.unpack(PEERCRED_STRUCT_DEF, peer_creds)
[  177s] error: unpack requires a string argument of length 12
[  177s] _____________________ test_peercreds_unix_sock_with_lookup _____________________
...
[  177s] E           HTTPError: 500 Server Error: Internal Server Error for url: http+unix://%2Ftmp%2FtmpVe2REm/peer_creds/texts
...
[  177s]   File "/home/abuild/rpmbuild/BUILD/cheroot-6.5.5/cheroot/server.py", line 1399, in get_peer_creds
[  177s]     pid, uid, gid = struct.unpack(PEERCRED_STRUCT_DEF, peer_creds)
[  177s] error: unpack requires a string argument of length 12
[  177s] == 2 failed, 70 passed, 1 skipped, 30 deselected, 3 xfailed in 43.27 seconds ===
[  177s] error: Bad exit status from /var/tmp/rpm-tmp.C8h9wf (%check)

❓ What is the motivation / use case for changing the behavior?

💡 To Reproduce

Steps to reproduce the behavior:
The error is reported in OBS tool while building python-cheroot package for openSUSE ppc64le at https://build.opensuse.org/package/live_build_log/devel:languages:python/python-cheroot/openSUSE_Factory_PowerPC/ppc64le

💡 Expected behavior

📋 Details

📋 Environment

• Cheroot version: 3.5.5

• CherryPy version: X.X.X (if applicable)

• Python version: 3.6.7

• OS: openSUSE

• Browser: [all | Chrome XX | Firefox XX | IE XX | Safari XX | Mobile Chrome XX | Android X.X Web Browser | iOS XX Safari | iOS XX UIWebView | iOS XX WKWebView ]

📋 Additional context

python_cheroot_standard_ppc64le_201908231005.log

[webknjaz]

@michelmno thanks for the report! I don't have any env with that arch for testing.
If you can debug what are the values of local vars there, it'd be highly appreciated.
If not, at least figure out how to put an xfail marker for those tests with a proper conditional. Is this only reproducible under Python 2.7? Is it possible to emulate that arch under a normal x86_64 laptop?

The interesting part of the traceback is:

error('unpack requires a string argument of length 12',)
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/cheroot-6.5.5/cheroot/server.py", line 1273, in communicate
    req.respond()
  File "/home/abuild/rpmbuild/BUILD/cheroot-6.5.5/cheroot/server.py", line 1077, in respond
    self.server.gateway(self).respond()
  File "/home/abuild/rpmbuild/BUILD/cheroot-6.5.5/cheroot/test/test_server.py", line 137, in respond
    peer_creds = conn.peer_pid, conn.peer_uid, conn.peer_gid
  File "/home/abuild/rpmbuild/BUILD/cheroot-6.5.5/cheroot/server.py", line 1405, in peer_pid
    pid, _, _ = self.get_peer_creds()
  File "/usr/lib/python2.7/site-packages/backports/functools_lru_cache.py", line 137, in wrapper
    result = user_function(*args, **kwds)
  File "/home/abuild/rpmbuild/BUILD/cheroot-6.5.5/cheroot/server.py", line 1399, in get_peer_creds
    pid, uid, gid = struct.unpack(PEERCRED_STRUCT_DEF, peer_creds)
error: unpack requires a string argument of length 12

[michelmno]

I verified that failure only with python2.7, no failure with python3.7

I tried to modify source file for python2.7; I am not fluent with python, so my 2 added lines are probably wrong, as output seems to be garbage:

$grep debugmno -A1 -B1 cheroot/server.py
        else:
            print('debugmno PEERCRED_STRUCT_DEF: '.join(PEERCRED_STRUCT_DEF))
            print('debugmno peer_creds: '.join(peer_creds))
            pid, uid, gid = struct.unpack(PEERCRED_STRUCT_DEF, peer_creds)

gives following output:

---- Captured stdout call ------
3debugmno PEERCRED_STRUCT_DEF: i
debugmno peer_creds: debugmno peer_creds: debugmno peer_creds:

[webknjaz]

You may want to add -vs -n 0 --fulltrace to the test run, this will probably prevent mangled output.

PEERCRED_STRUCT_DEF is a constant, declared once like 15 lines above and not reassigned since so its value isn't of interest to me.

str.join() accepts an iterable with strings that it'll concatenate using the delimiter, that's why you've got that output it iterated over chars in '3i' and put that string ('debugmno PEERCRED_STRUCT_DEF: ') in between.
What you want is something like

print('debugmno peer_creds: {!r}'.format(peer_creds))
print(
    'PEERCRED_STRUCT_DEF size: {!s}'.
    format(struct.calcsize(PEERCRED_STRUCT_DEF))
)

[webknjaz]

Also, could you please log

python2.7 -c 'import sys; print("byte order: {}".format(sys.byteorder))'

in that env

[michelmno]

with your lines:

debugmno peer_creds: '\x01\x00\x00\x00'
PEERCRED_STRUCT_DEF size: 12

yes ppc64le is little endian:

python2.7 -c 'import sys; print("byte order: {}".format(sys.byteorder))'
byte order: little

[webknjaz]

Interesting, it's supposed to return 3 integers... Do you know what is an integer size under this architecture?

[michelmno]

int is 4 bytes:

gcc -dM -E - < /dev/null |grep -i size
#define __SIZE_MAX__ 0xffffffffffffffffUL
#define __SIZEOF_INT__ 4
#define __SIZEOF_POINTER__ 8
#define __SIZEOF_LONG__ 8
#define __SIZEOF_LONG_DOUBLE__ 16
#define __SIZEOF_SIZE_T__ 8
#define __SIZEOF_WINT_T__ 4
#define __SIZE_TYPE__ long unsigned int
#define __SIZEOF_PTRDIFF_T__ 8
#define __SIZE_WIDTH__ 64
#define __SIZEOF_FLOAT__ 4
#define __SIZEOF_SHORT__ 2
#define __SIZEOF_INT128__ 16
#define __SIZEOF_WCHAR_T__ 4
#define __SIZEOF_DOUBLE__ 8
#define __SIZEOF_LONG_LONG__ 8

[webknjaz]

Can you try the program at the end of http://welz.org.za/notes/on-peer-cred.html to see whether it works (to see whether it's kernel's fault or so and exclude Python from the equation)? Also grep -rn 'struct ucred' /usr/include should lead you to the struct definition, could you plz check it too?

Oh, and did you run Python 3.7 and 2.7 tests under exactly same env?

[webknjaz]

Aha! It looks like this: https://bugzilla.redhat.com/show_bug.cgi?id=1733967 / https://www.mail-archive.com/devel@lists.fedoraproject.org/msg136822.html.

[webknjaz]

Could you please grep the headers for SO_PEERCRED constant definition? Supposedly, somewhere in socket.h.

[webknjaz]

And also grep for LOCAL_PEERCRED just in case.

[webknjaz]

Please post the output of these:

$ python2.7 -c 'import socket; print("SO_PEERCRED: {}".format(socket.SO_PEERCRED))'
$ python3.7 -c 'import socket; print("SO_PEERCRED: {}".format(socket.SO_PEERCRED))'

[webknjaz]

If I'm right, 621f4ee should fix this.

[webknjaz]

@michelmno once you confirm that it's fine, I'll cut a new release.

[michelmno]

@michelmno once you confirm that it's fine, I'll cut a new release.

I confirm that with the patch applied to my 3.5.5 version, the two tests passed on python2.7 (and still on python3.7)
Thank you for the correction.

[webknjaz]

@michelmno released under v6.5.8