Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SystemError: opening file "C:\\Users\\justi\\.julia\\registries\\General\\Registry.toml": No such file or directory #1667

Closed
justintong97 opened this issue Feb 12, 2020 · 29 comments

Comments

@justintong97
Copy link

If you have a question please search or post to our Discourse site: https://discourse.julialang.org.
We use the GitHub issue tracker for bug reports and feature requests only.

If you're submitting a bug report, be sure to include as much relevant information as
possible, including a minimal reproducible example and the output of versioninfo().
If you're experiencing a problem with a particular package, open an issue on that
package's repository instead.
When i type Pkg.add("IJulia")
it returns: SystemError: opening file "C:\Users\justi\.julia\registries\General\Registry.toml": No such file or directory

Thanks for contributing to the Julia project!

@KristofferC KristofferC transferred this issue from JuliaLang/julia Feb 12, 2020
@fredrikekre
Copy link
Member

Can you try to delete C:\Users\justi\.julia\registries\General and try again?

@justintong97
Copy link
Author

justintong97 commented Feb 12, 2020 via email

@00vareladavid
Copy link
Contributor

Perhaps we can detect this ourselves and re-download General if this is the case?

@AndyAbok
Copy link

Can you try to delete C:\Users\justi\.julia\registries\General and try again?

when you say delete how do you mean facing the same issue over here when tried to add the general after removing it since DataFrames was having issues with versioning

julia> Pkg.Registry.add("General")
Cloning registry from "https://github.com/JuliaRegistries/General.git"
ERROR: SystemError: opening file "C:\Users\andrewa\.julia\registries\General\Registry.toml": No such file or directory
Stacktrace:
[1] systemerror(::String, ::Int32; extrainfo::Nothing) at .\error.jl:168
[2] #systemerror#50 at .\error.jl:167 [inlined]
[3] systemerror at .\error.jl:167 [inlined]
[4] open(::String; read::Bool, write::Nothing, create::Nothing, truncate::Nothing, append::Nothing) at .\iostream.jl:254
[5] open(::String, ::String) at .\iostream.jl:310
[6] open(::typeof(Pkg.TOML.parse), ::String, ::Vararg{String,N} where N; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at .\io.jl:296
[7] open at .\io.jl:296 [inlined]
[8] parsefile at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\ext\TOML\src\TOML.jl:50 [inlined]
[9] read_registry(::String; cache::Bool) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Types.jl:974
[10] read_registry at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Types.jl:969 [inlined]
[11] (::Pkg.Types.var"#92#95"{Pkg.Types.Context,String,Pkg.Types.RegistrySpec})(::String) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Types.jl:946
[12] mktempdir(::Pkg.Types.var"#92#95"{Pkg.Types.Context,String,Pkg.Types.RegistrySpec}, ::String; prefix::String) at .\file.jl:682
[13] mktempdir at .\file.jl:680 [inlined] (repeats 2 times)
[14] clone_or_cp_registries(::Pkg.Types.Context, ::Array{Pkg.Types.RegistrySpec,1}, ::String) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Types.jl:911
[15] clone_or_cp_registries at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Types.jl:905 [inlined]
[16] #add#6 at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Registry.jl:29 [inlined]
[17] add at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Registry.jl:28 [inlined]
[18] #add#5 at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Registry.jl:26 [inlined]
[19] add at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Registry.jl:26 [inlined]
[20] add(::Array{String,1}; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Registry.jl:25
[21] add at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Registry.jl:25 [inlined]
[22] #add#1 at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Registry.jl:24 [inlined]
[23] add(::String) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Registry.jl:24
[24] top-level scope at REPL[173]:1

@Alexander-Barth
Copy link

I see this error too from time to time on Windows computers (last time on a Windows computer with julia 1.5.1 from a student). Removing C:\Users\<name>\.julia\registries\General always resolved the problem, but it would be nice to know the root cause of it. Unfortunately, this issue is very difficult to reproduce.

Here are other reports:
https://discourse.julialang.org/t/unable-to-install-ijulia/47160/2
JuliaLang/julia#37521

Is there any information that would be helpful to collect the next time this issue shows up? Maybe Linux and Mac handle better transient network issues when doing a git checkout than Windows?

@fredrikekre
Copy link
Member

it would be nice to know the root cause of it.

Windows or some program on your computer deletes it.

@fredrikekre
Copy link
Member

fredrikekre commented Sep 24, 2020

Here is another example where Windows deleted folders even: #2017 (comment), and here are a bunch more https://discourse.julialang.org/search?q=Registry.toml%22%3A%20No%20such%20file%20or%20directory

@StefanKarpinski
Copy link
Sponsor Member

It's absolutely bananas that Windows just deletes stuff, but that seems to be what happens.

@Alexander-Barth
Copy link

Yesterday, I had the first time seen this error (/Users/admin/.julia/registries/General/Registry.toml": No such file or directory while installing the first julia package; julia 1.5.1, I think) on a student's Mac. I thought that this problem was Windows specific.

I had a look at the code, and it is really strange:

  • At line 1004 the file is there (as the error message is not printed),
    if !isfile(joinpath(tmp, "Registry.toml"))
  • At line 1007, Registry.toml is missing
    registry = read_registry(joinpath(tmp, "Registry.toml"); cache=false) # don't cache this tmp registry

@fredrikekre
Copy link
Member

The error is from a later stage after the registry has been copied to the correct place.

@j-fu
Copy link

j-fu commented Nov 12, 2020

Hi, such things seem to happen also outside of Julia...

https://www.tenforums.com/general-support/121122-empty-folders-disappearing.html
https://www.tenforums.com/general-support/121891-downloads-folder-windows-10-1809-a.html
https://techcommunity.microsoft.com/t5/storage-at-microsoft/windows-10-and-storage-sense/ba-p/428270

There seems to be a thing like StorageSense which may be involved. Also some people report about
deactivating AV software like avast. No clear picture though.

Disclaimer: I am not a windows user, just do it occasionally in a VM - for the odd .docx and for testing Julia packges, so all of this was just googled.

@fonsp
Copy link
Member

fonsp commented Nov 13, 2020

I hear about this problem very often from new Pluto users. I think the most common cause is:

  1. Do a pkg> command
  2. It starts updating the registry without a progress bar
  3. After X minutes, I assume that something is broken and I Ctrl+C

The result is either this issue, or an incomplete registry ("package not found" errors) - both have unhelpful error messages.

But using the tarball directly in future Pkg version will also solve this issue, right?

@j-fu
Copy link

j-fu commented Nov 13, 2020

So it seems that there are possibly two problems here: windows possibly deleting files, and slow communication regarding packages. @fonsp: did you experiment with switching of the package server comunication by unsetting JULIA_PKG_SERVER ?

EDIT: May be also related to #2175 - it seems that the default for windows is being reset to git cloning.

@musm
Copy link
Contributor

musm commented Dec 4, 2020

I'm also getting this

@SimonEnsemble
Copy link

one of my students got this error on Mac:

SystemError: opening file "/Users/sarahutton/.julia/registries/General/Registry.toml": No such file or directory

@Manas2030
Copy link

Hi, I got the same error message even though I had my StorageSense turned off. But I was able to succesfully run the Pkg.add() command after adding, the "/.julia/registries/General" folder to my windows defender exclusions.

@SimonEnsemble
Copy link

to give an idea of prevalence: in my class of 86, ~6 students had this problem, where they needed to delete the General folder.

@IanButterworth
Copy link
Sponsor Member

IanButterworth commented Jan 20, 2021

Is what's happening here something like this?

  1. User initializes the registry, either by their first pkg> add or explicitly pkg> registry add
  2. Sees things going slowly so ctrl-c's, which may occur during the cp here (which is a mv on more recent julia)
    cp(tmp, regpath)
  3. The cp didn't get as far as copying Registry.toml over, which on windows happens alphabetically after the R dir has copied
  4. The user tries again to add etc.
  5. The logic here checks that the registry dir exists but not the Registry.toml file specifically
    if Pkg.isdir_nothrow(regpath)
  6. Goes on to open the nonexistent Registry.toml file

Full section (this is from Pkg that shipped with 1.5.1 so I could line up with the trace in https://discourse.julialang.org/t/unable-to-install-ijulia/47160)

Pkg.jl/src/Types.jl

Lines 1013 to 1029 in aaf4e6e

regpath = joinpath(depot, "registries", registry["name"]#=, slug=#)
ispath(dirname(regpath)) || mkpath(dirname(regpath))
if Pkg.isdir_nothrow(regpath)
existing_registry = read_registry(joinpath(regpath, "Registry.toml"))
if registry["uuid"] == existing_registry["uuid"]
println(ctx.io,
"registry `$(registry["name"])` already exist in `$(Base.contractuser(regpath))`.")
else
throw(PkgError("registry `$(registry["name"])=\"$(registry["uuid"])\"` conflicts with " *
"existing registry `$(existing_registry["name"])=\"$(existing_registry["uuid"])\"`. " *
"To install it you can clone it manually into e.g. " *
"`$(Base.contractuser(joinpath(depot, "registries", registry["name"]*"-2")))`."))
end
else
cp(tmp, regpath)
printpkgstyle(ctx, :Added, "registry `$(registry["name"])` to `$(Base.contractuser(regpath))`")
end

If that is what's happening, this might be a fix: #2349

@IanButterworth
Copy link
Sponsor Member

This appears to be fixed in 1.6 already, by changing from cp-ing to mv-ing the registry from temp

@Rapsodia86
Copy link

Just to let you all know, It is not only Windows/Mac issue. I just had it on Linux as well.

@StefanKarpinski
Copy link
Sponsor Member

That supports the theory that it's the cp being slow and getting interrupted half way through since that isn't Windows-specific (although it's worse on Windows since file operations are so slow there).

@IanButterworth
Copy link
Sponsor Member

Nightly will still do this if a registry dir is present but missing a Registry.toml, which may happen if people use older and newer julia versions together.
This might be a better check to make #2356

@Flourish38
Copy link

This also happened to me after Ctrl-C on a pkg> add command, I'm on Windows. Seems you all pretty much have it figured out, and the workaround isn't so bad in the meantime. Thanks for the help!

@bughht
Copy link

bughht commented Jan 27, 2021

The issue also happened to me after I shut down installing packages, deleting the General folder actually works for me, thanks.

@fredrikekre
Copy link
Member

Likely fixed by #2356

@sudhakar086
Copy link

Can you try to delete C:\Users\justi\.julia\registries\General and try again?

wonderfull

@cveintimillat
Copy link

Can you try to delete C:\Users\justi\.julia\registries\General and try again?

This did the trick

@Republic1024
Copy link

Can you try to delete C:\Users\justi\.julia\registries\General and try again?

May be function

@ron-wolf
Copy link

Hello all, I think I am just now seeing this issue crop up in September 2023. I am using Julia 1.9.2 on Windows 10, build 19045.3448. I am also a Pluto user. Anything I can do to help reproduce this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests