expose API for writing perf map files #103295
Labels
interpreter-core
(Objects, Python, Grammar, and Parser dirs)
type-feature
A feature request or enhancement
Comments
arhadthedev
added
the
interpreter-core
pablogsal
pushed a commit
that referenced
this issue
May 21, 2023
Co-authored-by: Aniket Panse <aniketpanse@fb.com> Co-authored-by: Gregory P. Smith <greg@krypto.org> Co-authored-by: Carl Meyer <carl@oddbird.net>
|
Hey folks, we experience this failure on Fedora 37, 38 and 39 on i686 (32bit): We are yet to investigate, but is this thing supposed to work on i686 as well? |
This seems to have crashed due to some hardening stuff. Can you share your hardening flags? |
The test is just writing to a file, not running |
|
CC: @gsallam |
CFLAGS='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m32 -march=i686 -mtune=generic -msse2 -mfpmath=sse -mstackrealign -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' https://src.fedoraproject.org/rpms/redhat-rpm-config/blob/rawhide/f/redhat-hardened-cc1 |
|
How easy is for you to get a native backtrace? |
|
I'm building it locally on my x86_64 machine and will try to salvage a coredump but I have never done this before with i686, hopefully, I'll get it right. It takes a while. Observation: The debug build passed the tests, only the optimized one did not. |
That's certainly a bad omen 😓 |
|
I was unable to find a coredump, but this is what I get when I run it trough gdb: Apparently, I am really bad at this :D |
|
I am currently trying to figure out how to reproduce this issue locally and will investigate it further. |
|
@hroncok Can you give any more details about how you built Python on an |
|
I am building the package without running the tests in https://copr.fedorainfracloud.org/coprs/churchyard/python3.12b1/builds/ -- that should hopefully make it easier to install it in a Fedora docker container and reproduce the issue there. I've built the package using mock on Fedora: Alternatively, the SRPM can be obtained from https://src.fedoraproject.org/rpms/python3.12/pull-request/40 Fedora CI scratch build result, currently https://kojipkgs.fedoraproject.org//work/tasks/3880/101493880/python3.12-3.12.0~b1-1.fc39.src.rpm and just use the mock commands. |
|
|
Also, in the same container: |
|
I may know what's going on. We are parsing the code address using the "K" format specified into a |
|
Heh, yes, I just reached the same conclusion as @pablogsal and came here to report it :) I will work on a PR (unless @gsallam gets there first.) |
Ah but I am still the fastest gun in the west 🔫🤠 |
carljm
added a commit
to carljm/cpython
that referenced
this issue
May 23, 2023
miss-islington
pushed a commit
to miss-islington/cpython
that referenced
this issue
May 23, 2023
…ss (pythonGH-104811) (cherry picked from commit e0b3078) Co-authored-by: Carl Meyer <carl@oddbird.net>
carljm
added a commit
that referenced
this issue
May 23, 2023
carljm
added a commit
that referenced
this issue
May 23, 2023
hrnciar
added a commit
to fedora-python/cpython
that referenced
this issue
May 24, 2023
…GH-104811) (python#104823) pythongh-103295: fix stack overwrite on 32-bit in perf map test harness (pythonGH-104811) (cherry picked from commit e0b3078) Co-authored-by: Carl Meyer <carl@oddbird.net>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
#96123 added support for CPython to write
/tmp/perf-<pid>.mapfiles, associating instruction address ranges with a human-readable frame name for the Linuxperfprofiler.Two external Python JIT compilers, Cinder and Pyston, both also independently write to perf map files.
Since perf map files are one-per-process, multiple separate libraries trying to write perf map entries independently can lead to file corruption from simultaneous writes.
It's unlikely for both Cinder and Pyston JITs to be used in the same process, but it's quite reasonable to use one of these JITs along with CPython's native perf trampoline support.
In order for this to be safe, CPython should expose a thread-safe API for writing perf map entries that all these clients can use.
(We've backported the 3.12 perf trampolines feature to Cinder 3.10 and experimented with using it, and we've seen this write corruption problem occur in practice; it's not just a theoretical risk we identified.)
cc @pablogsal , @kmod
Linked PRs
The text was updated successfully, but these errors were encountered: