relocation: account for trailing path separator in depot paths #55355
Conversation
fatteneder
force-pushed
the
fa/fix-55340
branch
from
fb19c5c
to
42eb383
Compare
| @@ -3131,7 +3131,7 @@ function replace_depot_path(path::AbstractString) | |||
| depot = dirname(depot) | |||
| end | |||
|
|
|||
| if startswith(path, depot) | |||
| if startswith(path, string(depot, Filesystem.pathsep())) || path == depot | |||
There was a problem hiding this comment.
In many cases, Windows accepts / as a path separator - is there any risk of that appearing here?
There was a problem hiding this comment.
oh, I did not know that.
I pushed a fix with an accompanying test.
There was a problem hiding this comment.
Thanks! The replace isn't safe for UNC paths (it destroys double slashes \\), but maybe we don't support those?
https://learn.microsoft.com/en-us/dotnet/standard/io/file-path-formats#example-ways-to-refer-to-the-same-file shows some exotic ways to build paths on Windows, in case we do. The Base path functionality uses splitdrive to get the UNC, etc. prefixes out of the way.
There was a problem hiding this comment.
I now settled to just use abspath to normalize both.
That should also increase the accuracy in matching depots in cases where one is an absolute and the other a relative one.
There was a problem hiding this comment.
I traced back the use of replace_depot_path and found that these issues should already be taken care of.
In particular,
- the
pathargument will be absolute and normalized when it arrives, which happens atbase/loading.jl:_include_dependency, - the elements of
DEPOT_PATHalso went throughabspathbefore they are handed over to precompilation, seebase/loading.jl:create_expr_cache.
So I will revert changes related to this and make the tests pass again.
There was a problem hiding this comment.
Right before spawning the precompilation process, here
Line 2812 in f2f76d8
Shouldn't --output-ji hit that too?
There was a problem hiding this comment.
That's something that spawns a process with --output-ji not the other way around
There was a problem hiding this comment.
I am sorry, I don't understand.
Where would be the entry point for what you describe?
There was a problem hiding this comment.
Ah, my point is that it's legal to spawn Julia with --output-ji outside of the spawning that precompilation does, and in that case the DEPOT_PATH may not be normalized
There was a problem hiding this comment.
Ok, I see. Indeed, that's problematic.
fatteneder
force-pushed
the
fa/fix-55340
branch
2 times, most recently
from
fd6ec91
to
6a7b91f
Compare
|
Seems like something is not good right now: |
fatteneder
force-pushed
the
fa/fix-55340
branch
from
6a7b91f
to
fc8a17e
Compare
|
Windows now chokes on Any idea what that could be? |
|
abspath does not always preserve being a path to the same file (except usually on Windows), so generally is not correct to use compared to other alternatives |
fatteneder
force-pushed
the
fa/fix-55340
branch
from
79afe76
to
b7dc53d
Compare
fatteneder
force-pushed
the
fa/fix-55340
branch
from
b7dc53d
to
79246a4
Compare
|
this should be good to go now |
Backported PRs: - [x] #54962 <!-- Add timing to precompile trace compile --> - [x] #55180 <!-- compress jit debuginfo for easy memory savings --> - [x] #54919 <!-- Fix annotated join with non-concrete eltype iters --> - [x] #55013 <!-- [docs] change docstring to match code --> - [x] #55017 <!-- TOML: Make `Dates` a type parameter --> - [x] #54033 <!-- Fix a bug in `stack`'s DimensionMismatch error message --> - [x] #55242 <!-- fix at-main docstring to not code quote a compat box --> - [x] #55261 <!-- Make `jl_*affinity` tests more portable --> - [x] #54736 <!-- specificity: ensure fast-path in `sub/eq_msp` handle missing `UnionAll` wrapper correctly. --> - [x] #55299 <!-- typeintersect: fix bounds merging during inner `intersect_all`. --> - [x] #55302 <!-- Add `lbt_forwarded_funcs()` to debug LBT forwarding issues --> - [x] #55148 <!-- Random: Mark unexported public symbols as public --> - [x] #55303 <!-- avoid overflowing show for OffsetArrays around typemax --> - [x] #55317 <!-- Restrict argument to `isleapyear(::Integer)` --> - [x] #55327 <!-- Profile: Fix stdlib paths --> - [x] #55330 <!-- [libblastrampoline] Bump to v5.11.0 --> - [x] #55310 <!-- Preserve structure in scaling triangular matrices by NaN --> - [x] #55329 <!-- mapreduce: don't inbounds unknown functions --> - [x] #55356 <!-- Profile: close files when assembling heap snapshot --> - [x] #55371 <!-- Fix tr for block SymTridiagonal --> - [x] #55307 <!-- Make REPL.TerminalMenus public --> - [x] #55362 <!-- inference: fix missing LimitedAccuracy markers --> - [x] #55306 <!-- AllocOpt: Fix stack lowering where alloca continas boxed and unboxed data --> - [x] #55395 <!-- fix #55389: type-unstable `join` --> - [x] #55226 <!-- re-add `unsafe_convert` for Reinterpret and Reshaped array --> - [x] #55405 <!-- handle unbound vars in NTuple fields --> - [x] #55365 <!-- ml-matches: ensure all methods are included --> - [x] #55428 <!-- codegen: move undef freeze before promotion point --> - [x] #55419 <!-- `stale_cachefile`: handle if the expected cache file is missing --> - [x] #55470 <!-- Add push! implementation for AbstractArray depending only on resize! --> - [x] #55483 <!-- fix hierarchy level of "API reference" in `Dates` documentation --> - [x] #55268 <!-- simplify complex atanh and remove singularity perturbation --> - [x] #55441 <!-- fix Event to use normal Condition variable --> - [x] #55413 <!-- subtyping: fast path for lhs union and rhs typevar --> - [x] #55492 <!-- build: add missing dependencies for expmap --> - [x] #55507 <!-- Fix fast getptls ccall lowering. --> - [x] #55424 <!-- add missing clamp function for IOBuffer --> - [x] #55504 <!-- Update symmetric docstring to reflect the type of uplo --> - [x] #55107 <!-- Make the memory GEP an inbounds GEP since the bounds check has happened somewhere else --> - [x] #55411 <!-- Vendor the terminfo database for use with base/terminfo.jl --> - [x] #55452 <!-- Do not load `ScopedValues` with `using` --> - [x] #55407 <!-- Remove deprecated non string API for LLVM pass pipeline and parse all options --> - [x] #55461 <!-- 🤖 [master] Bump the StyledStrings stdlib from d7496d2 to f6035eb --> - [x] #55433 <!-- Backport #55407 to 1.11 --> - [x] #55225 <!-- [1.11 backport] trace-compile: don't generate `precompile` statements for OpaqueClosure methods (#55072) --> - [x] #55212 <!-- Make `Base.depwarn()` public --> - [x] #552 - [x] #55052 <!-- Fix `(l/r)mul!` with `Diagonal`/`Bidiagonal` --> - [x] #55251 <!-- Restrict binary ops for Diagonal and Symmetric to Number eltypes -->95 <!-- LAPACK: Aggressive constprop to concretely infer syev!/syevd! --> - [x] #55522 <!-- Fix tr for Symmetric/Hermitian block matrices --> Need manual backport: - [x] #55342 <!-- Ensure bidiagonal setindex! does not read indices in error message --> Contains multiple commits, manual intervention needed: - [ ] #55336 <!-- codegen: take gc roots (and alloca alignment) more seriously --> Non-merged PRs with backport label: - [ ] #55506 <!-- Fix indexing in _mapreducedim for OffsetArrays --> - [ ] #55500 <!-- make jl_thread_suspend_and_get_state safe --> - [ ] #55499 <!-- propagate the terminal's `displaysize` to the `IOContext` used by the REPL --> - [ ] #55458 <!-- Allow for generically extracting unannotated string --> - [ ] #55457 <!-- Make AnnotateChar equality consider annotations --> - [ ] #55453 <!-- Privatise the annotations API, for StyledStrings --> - [ ] #55443 <!-- Add test for upper/lower/titlecase and fix call --> - [ ] #55355 <!-- relocation: account for trailing path separator in depot paths --> - [ ] #55220 <!-- `isfile_casesensitive` fixes on Windows --> - [ ] #55169 <!-- `propertynames` for SVD respects private argument --> - [ ] #54457 <!-- Make `String(::Memory)` copy --> - [ ] #53957 <!-- tweak how filtering is done for what packages should be precompiled --> - [ ] #51479 <!-- prevent code loading from lookin in the versioned environment when building Julia --> - [ ] #50813 <!-- More doctests for Sockets and capitalization fix --> - [ ] #50157 <!-- improve docs for `@inbounds` and `Base.@propagate_inbounds` --> - [ ] #41244 <!-- Fix shell `cd` error when working dir has been deleted -->
Fixes #55340