Using --bug-report from PyJulia

[MilesCranmer]

Hey all,

I'm trying to add the --bug-report flag to PyJulia because I'm encountering some nasty data races that I think only rr could track down (x-ref JuliaPy/PyCall.jl#1072, python/cpython#113591). The error only occurs in a PyJulia context, so I am working to enable the --bug-report flag in PyJulia here: JuliaPy/pyjulia#548.

However, it seems this flag doesn't actually do anything when passed, unlike every other option I've added. Basically it seems like the Julia runtime that Python gets linked to (by this method) is not actually aware of this flag.

I looked into the code here:

julia/base/client.jl

Lines 246 to 247 in 1b183b9

	# If we're doing a bug report, don't load anything else. We will
	# spawn a child in which to execute these options.

and it seems that --bug-report works by launching a child process, so I'm not sure what the next steps are.

Do you have any tips for how we could get --bug-report working within PyJulia?

Thanks!
Miles

[maleadt]

Basically it seems like the Julia runtime that Python gets linked to (by this method) is not actually aware of this flag.

--bug-report=rr works by re-launching the process under rr, so this cannot be made to work when only dlopening/initializing the runtime library. Instead, you'll have to manually launch python under rr in the first place. Since rr records all spawned processes, you may be able to conveniently do so by first starting Julia with --bug-report=rr and starting Python from that environment, but doing so may of course introduce other issues.

[MilesCranmer]

Thanks @maleadt. I tried running rr on python which called PyJulia but I found the debugging info about Julia to not be very useful. Running rr on Julia itself is much more useful but the segfault I am seeing is specific to the PyJulia interface and how the garbage collections interact with each other. So I’m just wondering if there are any options for getting this working better... maybe there just aren’t any?

[maleadt]

I tried running rr on python which called PyJulia but I found the debugging info about Julia to not be very useful.

BugReporting.jl goes to some lengths to find additional source information (i.e., identifying the Julia version, downloading the git repository, registering paths for GDB to look-up paths), but if you use a local source build (doesn't even have to be a debug build) rr should be able most without being invoked through BugReporting.jl.