Symbol caches: Larger download batching. Don't ask for non-General pkgs #263
Conversation
IanButterworth
force-pushed
the
ib/cache_download_tweaks
branch
from
c872a91
to
b674b12
Compare
IanButterworth
changed the title
IanButterworth
force-pushed
the
ib/cache_download_tweaks
branch
from
14e653f
to
17f657c
Compare
|
This has proven quite painful to get working on older julia versions. I went as far as 1.1, and for 1.0 rely on a try-catch to default to just generated caches locally. (Perhaps someone else can see a fix for 1.0 where |
IanButterworth
requested review from
davidanthoff and
fredrikekre
and removed request for
davidanthoff
IanButterworth
force-pushed
the
ib/cache_download_tweaks
branch
2 times, most recently
from
4ee8a56
to
d98c479
Compare
|
@davidanthoff it would be great, if you approve, to get this trialed on the pre-release |
IanButterworth
changed the title
src/SymbolServer.jl
Outdated
| regs = Pkg.Types.Context().registries | ||
| i = findfirst(r -> r.name == "General" && r.uuid == UUID("23338594-aafe-5451-b93e-139f81909106"), regs) | ||
| i === nothing && return Dict{UUID, PkgEntry}() | ||
| return regs[i].pkgs |
There was a problem hiding this comment.
This isn't working in the vscode extension because Pkg.Types.Context().registries is empty
┌ Info: regs
└ regs = Pkg.Registry.RegistryInstance[]
┌ Error: Symbol cache downloading: Failed to identify which packages to omit based on the General registry.
│ All packages will be processsed locally
│ err = Could not find the General registry
└ @ SymbolServer ~/Documents/GitHub/julia-vscode/scripts/packages/SymbolServer/src/
There was a problem hiding this comment.
What's the right way to get the General registry from here?
There was a problem hiding this comment.
That's kinda tricky. We're overwriting setting the JULIA_DEPOT_PATH env variable to a depot vendored in the extension, but you could probably temporarily change the DEPOT_PATH before this?
There was a problem hiding this comment.
Sorry, I left this behind. I think it would be great to get this finished up. Does that sound like a good plan @davidanthoff ?
There was a problem hiding this comment.
Ok. I've tried to implement that. I'd appreciate trying to get this merged to stop sending out non-General package name & uuids
Co-authored-by: Eric Hanson <5846501+ericphanson@users.noreply.github.com>
IanButterworth
force-pushed
the
ib/cache_download_tweaks
branch
from
f78bbfb
to
d3f34d3
Compare
|
Will review/merge this tomorrow. Feel free to ping me somewhere if I don't :) |
|
Will do, thanks |
|
Bump 🙏 |
|
Thanks for the PR! I'll make a few follow-up tweaks, but none of that should hold this up any further :) |
|
Just checking, is this used on the extension now? I wasn't sure if the extension uses a non-tagged commit |
|
It is. You can check here which commit the extension uses. |
Given non-General packages don't have symbol caches, don't request them, to avoid requesting URLs that include the UUID & name of private packages.
Also the download batching didn't make sense to me. It seems better to limit the concurrent number of downloads, rather than the total number of batches. Update: Now batches of 100
Fixes #.
For every PR, please check the following: