Can't add packages in v0.5 on 'new' macOS installation #19873

cormullion · 2017-01-05T11:06:59Z

An apparently empty directory, and a newly downloaded copy of Julia DMG on macOS:

[the-iMac:~/.julia] $ ls -alt
total 0
drwxr-xr-x   5 me  staff   170  5 Jan 10:55 .
drwxr-xr-x   4 me  staff   136  5 Jan 10:27 lib
drwxr-xr-x@ 85 me  staff  2890  5 Jan 09:59 ..
drwxr-xr-x  45 me  staff  1530  4 Jan 16:14 v0.4
drwxr-xr-x  99 me  staff  3366 16 Aug 13:23 v0.3
[the-iMac:~/.julia] $
[the-iMac:~/.julia] $ julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.5.0 (2016-09-19 18:14 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-apple-darwin13.4.0

julia> Pkg.add("Images")
INFO: Initializing package repository /Users/me/.julia/v0.5
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: SystemError (with /Users/me/.julia/v0.5/tmpWCKLoI): rmdir: Directory not empty
 in #systemerror#51 at ./error.jl:34 [inlined]
 in (::Base.#kw##systemerror)(::Array{Any,1}, ::Base.#systemerror, ::Symbol, ::Bool) at ./<missing>:0
 in #rm#7(::Bool, ::Bool, ::Function, ::String) at ./file.jl:125
 in init(::String, ::String) at ./pkg/dir.jl:60
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::String, ::Vararg{Any,N}) at ./pkg/dir.jl:28
 in add(::String) at ./pkg/pkg.jl:100

Trying again gives different error of course, because things are no longer empty:

julia> Pkg.add("Images")
INFO: Initializing package repository /Users/me/.julia/v0.5
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: ArgumentError: '/Users/me/.julia/v0.5/REQUIRE' exists. `remove_destination=true` is required to remove '/Users/me/.julia/v0.5/REQUIRE' before moving.
 in #checkfor_mv_cp_cptree#8(::Bool, ::Function, ::String, ::String, ::String) at ./file.jl:147
 in #checkfor_mv_cp_cptree#8(::Bool, ::Function, ::String, ::String, ::String) at /Applications/Julia-0.5.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 in (::Base.Filesystem.#kw##checkfor_mv_cp_cptree)(::Array{Any,1}, ::Base.Filesystem.#checkfor_mv_cp_cptree, ::String, ::String, ::String) at ./<missing>:0
 in #mv#11 at ./file.jl:184 [inlined]
 in mv at ./file.jl:184 [inlined]
 in init(::String, ::String) at ./pkg/dir.jl:58
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::String, ::Vararg{Any,N}) at ./pkg/dir.jl:28
 in add(::String) at ./pkg/pkg.jl:100

The text was updated successfully, but these errors were encountered:

tkelman · 2017-01-05T16:21:56Z

Relevant code is here:

julia/base/pkg/dir.jl

Lines 43 to 65 in b53cd8b

    
           local temp_dir = "" 
        
           try 
        
               mkpath(dir) 
        
               temp_dir = mktempdir(dir) 
        
               Base.cd(temp_dir) do 
        
                   info("Cloning METADATA from $meta") 
        
                   with(LibGit2.clone(meta, "METADATA", branch = branch)) do metadata_repo 
        
                       LibGit2.set_remote_url(metadata_repo, meta) 
        
                   end 
        
                   touch("REQUIRE") 
        
                   touch("META_BRANCH") 
        
                   write("META_BRANCH", branch) 
        
               end 
        
               #Move TEMP to METADATA 
        
               Base.mv(joinpath(temp_dir,"METADATA"), metadata_dir) 
        
               Base.mv(joinpath(temp_dir,"REQUIRE"), joinpath(dir,"REQUIRE")) 
        
               Base.mv(joinpath(temp_dir,"META_BRANCH"), joinpath(dir,"META_BRANCH")) 
        
               rm(temp_dir) 
        
           catch err 
        
               ispath(metadata_dir) && rm(metadata_dir, recursive=true) 
        
               ispath(temp_dir) && rm(temp_dir, recursive=true) 
        
               rethrow(err) 
        
           end

What's left inside /Users/me/.julia/v0.5/tmpWCKLoI in the first case?

cormullion · 2017-01-05T17:12:28Z

Trouble is, there's nothing there. Running it again:

julia> Pkg.add("Images")
INFO: Initializing package repository /Users/me/.julia/v0.5
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: SystemError (with /Users/me/.julia/v0.5/tmpkw6rui): rmdir: Directory not empty
 in #systemerror#51 at ./error.jl:34 [inlined]
 in (::Base.#kw##systemerror)(::Array{Any,1}, ::Base.#systemerror, ::Symbol, ::Bool) at ./<missing>:0
 in #rm#7(::Bool, ::Bool, ::Function, ::String) at ./file.jl:125
 in init(::String, ::String) at ./pkg/dir.jl:60
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::String, ::Vararg{Any,N}) at ./pkg/dir.jl:28
 in add(::String) at ./pkg/pkg.jl:100
shell> ls -alt v0.5
total 24
-rw-r--r--@ 1 me  staff  6148  5 Jan 17:07 .DS_Store
drwxr-xr-x  5 me  staff   170  5 Jan 17:07 .
-rw-r--r--  1 me  staff    11  5 Jan 17:07 META_BRANCH
-rw-r--r--  1 me  staff     0  5 Jan 17:07 REQUIRE
drwxr-xr-x  7 me  staff   238  5 Jan 17:07 ..

shell> ls /Users/me/.julia/v0.5/tmpkw6rui
ls: /Users/me/.julia/v0.5/tmpkw6rui: No such file or directory

Presumably the temp dir got deleted after the error...

tkelman · 2017-01-05T18:54:42Z

maybe add a @show readdir(temp_dir) right before the rm(temp_dir) and see what's there?

cormullion · 2017-01-05T21:14:05Z

Do I have to re-build Julia or something first? (There's a first time for everything, I suppose, but I've not done that before...)

tkelman · 2017-01-05T21:21:34Z

At least the system image (I'm actually moderately amazed this question doesn't get asked more often). See https://github.com/JuliaLang/julia/blob/d7625dd7dccab548025fa8b5b9d4c7941336fb24/doc/src/devdocs/sysimg.md#building-the-julia-system-image and https://github.com/JuliaLang/julia/blob/d7625dd7dccab548025fa8b5b9d4c7941336fb24/contrib/build_sysimg.jl

cormullion · 2017-01-05T21:30:49Z

Oh, cool, I didn't know about this! I'll try it first thing tomorrow.

cormullion · 2017-01-06T09:16:31Z

I modified this bit of base/pkg/dir.jl (line 56)

#Move TEMP to METADATA
Base.mv(joinpath(temp_dir,"METADATA"), metadata_dir)
Base.mv(joinpath(temp_dir,"REQUIRE"), joinpath(dir,"REQUIRE"))
Base.mv(joinpath(temp_dir,"META_BRANCH"), joinpath(dir,"META_BRANCH"))
rm(temp_dir)

to

#Move TEMP to METADATA
@show readdir(temp_dir)
Base.mv(joinpath(temp_dir,"METADATA"), metadata_dir)
@show readdir(temp_dir)
Base.mv(joinpath(temp_dir,"REQUIRE"), joinpath(dir,"REQUIRE"))
@show readdir(temp_dir)
Base.mv(joinpath(temp_dir,"META_BRANCH"), joinpath(dir,"META_BRANCH"))
@show readdir(temp_dir)
rm(temp_dir)

and I got this:

readdir(temp_dir) = String[".DS_Store","METADATA","META_BRANCH","REQUIRE"]
readdir(temp_dir) = String[".DS_Store","META_BRANCH","REQUIRE"]
readdir(temp_dir) = String[".DS_Store","META_BRANCH"]
readdir(temp_dir) = String[".DS_Store"]
ERROR: SystemError (with /Users/me/.julia/v0.5/METADATA): rmdir: Directory not empty

so I'm guessing that the final rm(temp_dir) fails because the directory contains a random .DS_Store file in there? (A problem specific to macOS users I expect.)

I don't think I'm quite up to submitting a PR that modifies code at the heart of the base Julia system, though...

tkelman · 2017-01-06T09:39:27Z

ah, I agree with your assessment. all it should need is making that last rm recursive then I believe

cormullion · 2017-01-06T12:03:43Z

Well I had a go at a PR (#19894) after saying I wouldn't... :)

tkelman · 2017-01-16T17:20:14Z

fixed by #19894, right?

Issue #19873. If the macOS Finder inserts a .DS_Store file while Pkg.update/Pkg.add operation is in progress, a plain `rm(temp_dir)` will not succeed.

cormullion · 2017-01-16T17:25:29Z

@tkelman yes it will be—or rather it was! Thanks.

Issue #19873. If the macOS Finder inserts a .DS_Store file while Pkg.update/Pkg.add operation is in progress, a plain `rm(temp_dir)` will not succeed. (cherry picked from commit ca3b06e)

tkelman added the domain:packages Package management and loading label Jan 5, 2017

cormullion mentioned this issue Jan 6, 2017

Avoid error if temp_dir acquires .DS_Store file (#19873) #19894

Merged

tkelman closed this as completed Jan 16, 2017

tkelman pushed a commit that referenced this issue Jan 16, 2017

Avoid error if temp_dir acquires .DS_Store file (#19894)

ca3b06e

Issue #19873. If the macOS Finder inserts a .DS_Store file while Pkg.update/Pkg.add operation is in progress, a plain `rm(temp_dir)` will not succeed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't add packages in v0.5 on 'new' macOS installation #19873

Can't add packages in v0.5 on 'new' macOS installation #19873

cormullion commented Jan 5, 2017

tkelman commented Jan 5, 2017

cormullion commented Jan 5, 2017

tkelman commented Jan 5, 2017

cormullion commented Jan 5, 2017

tkelman commented Jan 5, 2017

cormullion commented Jan 5, 2017

cormullion commented Jan 6, 2017 •

edited

tkelman commented Jan 6, 2017

cormullion commented Jan 6, 2017 •

edited

tkelman commented Jan 16, 2017

cormullion commented Jan 16, 2017

Can't add packages in v0.5 on 'new' macOS installation #19873

Can't add packages in v0.5 on 'new' macOS installation #19873

Comments

cormullion commented Jan 5, 2017

tkelman commented Jan 5, 2017

cormullion commented Jan 5, 2017

tkelman commented Jan 5, 2017

cormullion commented Jan 5, 2017

tkelman commented Jan 5, 2017

cormullion commented Jan 5, 2017

cormullion commented Jan 6, 2017 • edited

tkelman commented Jan 6, 2017

cormullion commented Jan 6, 2017 • edited

tkelman commented Jan 16, 2017

cormullion commented Jan 16, 2017

cormullion commented Jan 6, 2017 •

edited

cormullion commented Jan 6, 2017 •

edited