Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
adds USDT trace points for supported distros #366
Python supports user-level statically defined tracing (USDT) for platforms that support it. This has historically been not compiled-in by default in Linux distributions because only hosts with
The upstream implementation is a little weird (ref https://docs.python.org/3/howto/instrumentation.html). The
This package or one like it is not available for Alpine. I have another branch with an experiment to vendor the header files and the mock
Just to put some stats down so we have a couple comparisons:
Overall, it might be worth the extra build time and small bump in size, tracing is nice to have. Is there a comparison of python performance before and after this flag?
On the other branch, I'm not sure vendoring in the Debian scripts for Alpine is the right way; that could dramatically increase the maintenance in this repo and distracts from its simplicity. Maybe we just skip Alpine for now and work to get the package into Alpine proper?
Seems generally sane, although I'm definitely curious to see the results of that benchmark.
This is definitely supported for Python 2.x and 3.x? (https://docs.python.org/2/howto/instrumentation.html is a 404, so I want to make sure Python 2 isn't just quietly ignoring the
More concretely: https://travis-ci.org/docker-library/python/jobs/481912534#L1050
(So I think this should definitely be removed from all the
Sorry about the delay in replying.
I looked into the Python2 thing and it looks like it's not supported in upstream for Linux, only as patches for other Unixes. So we can remove that. I'm going to admit I'm not sure what the
I'm currently running Python's own benchmarking suite against both
Not sure how long this benchmark takes to run but I'm just running it on a machine overnight and I'll post again here tomorrow with results.
The python benchmark suite I ran exercises 60 real-world applications. It takes a bit over 20 min to run on my lab machine, so I ran this a few times to verify the results I got here.
Bottom line: although many of the benchmarks don't show a significant difference between
Needless to say I'm surprised and disappointed by this. Keep in mind this benchmark was run without an active trace, so I would not have expected the no-op function wrapping to create such a high level of overhead. I'm definitely glad I did the benchmarks, because accepting this PR would result in a significant community-wide hit. I'm going to withdraw this PR.
There's another approach worth exploring for eBPF tracing of the interpreter without compiling-in usdt, so I may try to tackle publishing a reproducible method for that for folks to use. But that's not specific to Docker.
#!/bin/bash set -e cat <<EOF > Dockerfile FROM python:3.7-stretch RUN python3 -m pip install performance EOF cat <<EOF > Dockerfile-with-dtrace FROM 0x74696d/python:3.7-stretch RUN python3 -m pip install performance EOF docker build -t="python-bench-no-dtrace" . docker build -t="python-bench-with-dtrace" -f Dockerfile-with-dtrace . echo 'Running benchmark for Python3 baseline' docker run -d \ --name bench1 \ -w /tmp \ -v $(pwd):/tmp \ python-bench-no-dtrace \ pyperformance run --python=python3 -o python-no-dtrace.json time docker wait bench1 echo 'Running benchmark for Python3 with dtrace, no traces enabled' docker run -d \ --name bench2 \ -w /tmp \ -v $(pwd):/tmp \ python-bench-with-dtrace \ pyperformance run --python=python3 -o python-dtrace-not-tracing.json time docker wait bench2 pyperformance compare python-no-dtrace.json python-dtrace-not-tracing.json > results.txt