RFC: Don't leave precompile-time tempdirs to be cleaned-up in binaries (#60078)

[deroulers]

I think that #60078 is due to the fact that Julia binaries inherit the dictionary TEMP_CLEANUP (used in base/file.jl) defined during build (more precisely, by function generate_precompile_statements in contrib/generate_precompile.jl), which is wrong because the temporary files and directories at runtime may reside in quite a different directory than at build time.

In particular, in the case of MacPorts on a recent Mac OS (e.g. Mac OS 15), where each user has a specific tempdir not accessible to others, building Julia is performed by the user macports, then Julia's binary hardcodes that some directories inside the tempdir of user macports have to be cleaned on exit. When running Julia under the identity of another user, who has another tempdir and no read/write access to the tempdir of macports, the reported failure happens.

The proposed fix is to set up temporary files created by function generate_precompile_statements in contrib/generate_precompile.jl to have no automatic cleanup. This won't do harm since they will be removed by the cleanup of the temporary directory created by the call to mktempdir in function generate_precompile_statements (see the call rm(tmpdir, recursive=true) in base/file.jl). On the other hand, the benefit of this is that the dictionary TEMP_CLEANUP at runtime will be free of irrelevant compile-time directories to clean.

To test this issue, one can do:
$ mkdir $TMPDIR/JuliaBuild
$ OLDTMPDIR=$TMPDIR
$ export TMPDIR=$TMPDIR/JuliaBuild
$ cd <julia's build root>
$ <build julia>
$ strings usr/lib/julia/sys.dylib|less # shows that the temp. dir. JuliaBuild is hardcoded in the binaries
$ TMPDIR=$OLDTMPDIR
$ chmod u-x $TMPDIR/JuliaBuild
$ <run the built julia with ./julia and exit> # should try to use $TMPDIR/JuliaBuild which is now inaccessible

Without the fix, the last step triggers the infinite loop of ``Failed to clean up temporary path''. With the fix, one can see that the binaries (usr/lib/julia/sys.dylib) do not contain the temp. dir. anymore, and the infinite loop of errors is not triggered.

Any comment/help appreciated.

[vchuravy]

Please use a descriptive title rather than "Fix issue", that greatly aids in discover ability

[Keno]

Seems like a better solution would be a manual Base.temp_cleanup_purge_all at the bottom where it's resetting all the global state pre-output? Otherwise we're littering these temp files. Alternatively, we could add support a syntax for these cli argument to write to a numeric fd and just read the output into an IOBuffer.

[deroulers]

Otherwise we're littering these temp files.

In my understanding, these temp files are removed because they are created in a temp dir created by the call to mktempdir and this function cleans afterwards.

[vtjnash]

manual Base.temp_cleanup_purge_all at the bottom where it's resetting all the global state pre-output

We attempt to explicitly do this automatically, but we forgot to call Base.__init__ (and particularly Filesystem.__postinit__) to re-enable it for this process:

$ ./julia -E 'Base.Filesystem.TEMP_CLEANUP'
Dict{String, Bool}("/var/folders/r7/k49rwnbx0w15d55pw31jlrrr0000gn/T/jl_95271P/jl_bkBIBeruRi" => 0, "/var/folders/r7/k49rwnbx0w15d55pw31jlrrr0000gn/T/jl_95271P/jl_JU9YJQNcRm" => 0)

[vtjnash]

This might not be the ideal fix (figuring out how to run the right atexit hooks), but it appears to be all of sufficient, correct, and minimal, so I've gone ahead and merged it and marked for backporting to all affected versions (this bug does not appear to be present in v1.10)