Registry installation failure with 1.6 on Windows 10

[PetrKryslUCSD]

After the .Julia folder was completely removed:

               _                                                                              
   _       _ _(_)_     |  Documentation: https://docs.julialang.org                           
  (_)     | (_) (_)    |                                                                      
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.                               
  | | | | | | |/ _` |  |                                                                      
  | | |_| | | | (_| |  |  Version 1.6.0-DEV.1647 (2020-12-03)                                 
 _/ |\__'_|_|_|\__'_|  |  Commit 49b8e61a80* (0 days old master)                              
|__/                   |                                                                      
                                                                                              
julia> using Pkg; Pkg.activate("."); Pkg.instantiate();                                       
  Activating environment at `C:\Users\PetrKrysl\Documents\work\BoneShaker.jl\Project.toml`    
  Installing known registries into `C:\Users\PetrKrysl\.julia`                                
     Cloning registry from "https://github.com/JuliaRegistries/General.git"                   
ERROR: expected package `Arpack [7d9fca2a]` to be registered                                  
Stacktrace:                                                                                   
 [1] pkgerror(msg::String)                                                                    
   @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:52                                                                                       
 [2] check_registered(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec})            
   @ Pkg.Operations C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Operations.jl:1130                                                                           
 [3] instantiate(ctx::Pkg.Types.Context; manifest::Nothing, update_registry::Bool, verbose::Bool, platform::Base.BinaryPlatforms.Platform, allow_autoprecomp::Bool, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})                                     
   @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1312                                                                                         
 [4] instantiate                                                                              
   @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1265 [inlined]                                                                                       
 [5] #instantiate#246                                                                         
   @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1261 [inlined]                                                                                       
 [6] instantiate()                                                                            
   @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1261                                                                                         
 [7] top-level scope                                                                          
   @ REPL[1]:1                                                                                
                                                                                              
caused by: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:SSL, TLS backend doesn't support certificate locations)                   
Stacktrace:                                                                                   
  [1] pkgerror(msg::String)                                                                   
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:52                                                                                      
  [2] clone(ctx::Pkg.Types.Context, url::String, source_path::String; header::String, credentials::Nothing, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})                                                                                             
    @ Pkg.GitTools C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\GitTools.jl:119                                                                               
  [3] (::Pkg.Types.var"#81#84"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec})(tmp::String)                                                                                             
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:985                                                                                     
  [4] mktempdir(fn::Pkg.Types.var"#81#84"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec}, 
parent::String; prefix::String)                                                               
    @ Base.Filesystem .\file.jl:729                                                           
  [5] mktempdir                                                                               
    @ .\file.jl:727 [inlined]                                                                 
  [6] clone_or_cp_registries(ctx::Pkg.Types.Context, regs::Vector{Pkg.Types.RegistrySpec}, depot::String)                                                                                   
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:964                                                                                     
  [7] clone_or_cp_registries                                                                  
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:957 
[inlined]                                                                                     
  [8] clone_default_registries(ctx::Pkg.Types.Context; only_if_empty::Bool)                   
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:860                                                                                     
  [9] clone_default_registries                                                                
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:848 
[inlined]                                                                                     
 [10] find_registered!(ctx::Pkg.Types.Context, names::Vector{String}, uuids::Vector{Base.UUID})                                                                                             
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:1228                                                                                    
 [11] find_registered!                                                                        
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:1189 [inlined]                                                                                    
 [12] check_registered(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec})           
    @ Pkg.Operations C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Operations.jl:1127                                                                          
 [13] instantiate(ctx::Pkg.Types.Context; manifest::Nothing, update_registry::Bool, verbose::Bool, platform::Base.BinaryPlatforms.Platform, allow_autoprecomp::Bool, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})                                    
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1306                                                                                        
 [14] instantiate                                                                             
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1265 [inlined]                                                                                      
 [15] #instantiate#246                                                                        
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1261 [inlined]                                                                                      
 [16] instantiate()                                                                           
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1261                                                                                        
 [17] top-level scope                                                                         
    @ REPL[1]:1                                                                               
                                                                                              
caused by: GitError(Code:ERROR, Class:SSL, TLS backend doesn't support certificate locations) 
Stacktrace:                                                                                   
  [1] macro expansion                                                                         
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\error.jl:110 [inlined]                                                                                 
  [2] set_ssl_cert_locations(cert_loc::String)                                                
    @ LibGit2 C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\LibGit2.jl:997                                                                                 
  [3] initialize()                                                                            
    @ LibGit2 C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\LibGit2.jl:990                                                                                 
  [4] ensure_initialized                                                                      
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\LibGit2.jl:970 [inlined]                                                                               
  [5] LibGit2.GitConfig(level::LibGit2.Consts.GIT_CONFIG)                                     
    @ LibGit2 C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\config.jl:50                                                                                   
  [6] CredentialPayload                                                                       
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\types.jl:1365 [inlined]                                                                                
  [7] #CredentialPayload#24                                                                     
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\types.jl:1375 [inlined]      
  [8] CredentialPayload                                                                                           
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\types.jl:1375 [inlined]      
  [9] clone(repo_url::SubString{String}, repo_path::String; branch::String, isbare::Bool, remote_cb::Ptr{Nothing}, credentials::LibGit2.CachedCredentials, callbacks::Dict{Symbol, Tuple{Ptr{Nothing}, Any}})                       
    @ LibGit2 C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\LibGit2.jl:562       
 [10] clone(ctx::Pkg.Types.Context, url::String, source_path::String; header::String, credentials::Nothing, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})                                       
    @ Pkg.GitTools C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\GitTools.jl:109     
 [11] (::Pkg.Types.var"#81#84"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec})(tmp::String)                   
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:985           
 [12] mktempdir(fn::Pkg.Types.var"#81#84"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec}, parent::String; prefix::String)                                                                                                       
    @ Base.Filesystem .\file.jl:729                                                                               
 [13] mktempdir                                                                                                   
    @ .\file.jl:727 [inlined]                                                                                     
 [14] clone_or_cp_registries(ctx::Pkg.Types.Context, regs::Vector{Pkg.Types.RegistrySpec}, depot::String)         
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:964           
 [15] clone_or_cp_registries                                                                                      
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:957 [inlined]           
 [16] clone_default_registries(ctx::Pkg.Types.Context; only_if_empty::Bool)                                       
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:860           
 [17] clone_default_registries                                                                                    
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:848 [inlined]           
 [18] find_registered!(ctx::Pkg.Types.Context, names::Vector{String}, uuids::Vector{Base.UUID})                   
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:1228          
 [19] find_registered!                                                                                            
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:1189 [inlined]          
 [20] check_registered(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec})                               
    @ Pkg.Operations C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Operations.jl:1127                                                                                                                  
 [21] instantiate(ctx::Pkg.Types.Context; manifest::Nothing, update_registry::Bool, verbose::Bool, platform::Base.BinaryPlatforms.Platform, allow_autoprecomp::Bool, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})                                                                                                
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1306              
 [22] instantiate                                                                                                 
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1265 [inlined]            
 [23] #instantiate#246                                                                                            
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1261 [inlined]            
 [24] instantiate()                                                                                               
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:1261              
 [25] top-level scope                                                                                             
    @ REPL[1]:1                                                                                                   
                                                                                                                  
julia> 

This result for

julia> versioninfo()                                                                                              
Julia Version 1.6.0-DEV.1647                                                                                      
Commit 49b8e61a80* (2020-12-03 17:00 UTC)                                                                         
Platform Info:                                                                                                    
  OS: Windows (x86_64-w64-mingw32)                                                                                
  CPU: Intel(R) Core(TM) i7-8705G CPU @ 3.10GHz                                                                   
  WORD_SIZE: 64                                                                                                   
  LIBM: libopenlibm                                                                                               
  LLVM: libLLVM-11.0.0 (ORCJIT, skylake)                                                                          
Environment:                                                                                                      
  JULIA_NUM_THREADS = 4      

Curiously, if the registry is first built with 1.5.3 and then the environment activation/instantiation is started with 1.6, everything goes through smoothly. So I guess it is just the registry, not the packages.

[PetrKryslUCSD]

Actually errors still occur in 1.6. The above information about smooth sailing after initial registry update was too optimistic.

Cloning into 'FinEtoolsVibInFluids.jl'...                                                                                                                                                               
remote: Enumerating objects: 367, done.                                                                                                                                                                 
remote: Counting objects: 100% (367/367), done.                                                                                                                                                         
remote: Compressing objects: 100% (226/226), done.                                                                                                                                                      
remote: Total 367 (delta 196), reused 292 (delta 125), pack-reused 0                                                                                                                                    
Receiving objects: 100% (367/367), 9.68 MiB | 4.62 MiB/s, done.                                                                                                                                         
Resolving deltas: 100% (196/196), done.                                                                                                                                                                 
Current folder: C:\Users\PetrKrysl\Documents\work\FinEtoolsTestAll.jl\tests\FinEtoolsVibInFluids.jl                                                                                                     
  Activating environment at `C:\Users\PetrKrysl\Documents\work\FinEtoolsTestAll.jl\tests\FinEtoolsVibInFluids.jl\Project.toml`                                                                          
   Installed NIfTI ─ v0.4.1                                                                                                                                                                             
Precompiling project...                                                                                                                                                                                 
  Progress [========================================>]  18/18                                                                                                                                           
18 dependencies successfully precompiled in 17 seconds (14 already precompiled)                                                                                                                         
    Updating registry at `C:\Users\PetrKrysl\.julia\registries\General`                                                                                                                                 
ERROR: LoadError: GitError(Code:ERROR, Class:SSL, TLS backend doesn't support certificate locations)                                                                                                    
Stacktrace:                                                                                                                                                                                             
  [1] macro expansion                                                                                                                                                                                   
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\error.jl:110 [inlined]                                                                                             
  [2] set_ssl_cert_locations(cert_loc::String)                                                                                                                                                          
    @ LibGit2 C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\LibGit2.jl:997                                                                                             
  [3] initialize()                                                                                                                                                                                      
    @ LibGit2 C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\LibGit2.jl:990                                                                                             
  [4] ensure_initialized                                                                                                                                                                                
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\LibGit2.jl:970 [inlined]                                                                                           
  [5] LibGit2.GitRepo(path::String)                                                                                                                                                                     
    @ LibGit2 C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\LibGit2\src\repository.jl:9                                                                                            
  [6] #ensure_clone#2                                                                                                                                                                                   
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\GitTools.jl:70 [inlined]                                                                                               
  [7] handle_repo_add!(ctx::Pkg.Types.Context, pkg::Pkg.Types.PackageSpec)                                                                                                                              
    @ Pkg.Types C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Types.jl:580                                                                                                 
  [8] up_load_versions!(ctx::Pkg.Types.Context, pkg::Pkg.Types.PackageSpec, entry::Pkg.Types.PackageEntry, level::Pkg.Types.UpgradeLevel)                                                               
    @ Pkg.Operations C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Operations.jl:1251                                                                                      
  [9] up(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, level::Pkg.Types.UpgradeLevel)                                                                                                    
    @ Pkg.Operations C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Operations.jl:1286                                                                                      
 [10] up(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; level::Pkg.Types.UpgradeLevel, mode::Pkg.Types.PackageMode, update_registry::Bool, kwargs::Base.Iterators.Pairs{Union{}, Union{}, 
Tuple{}, NamedTuple{(), Tuple{}}})                                                                                                                                                                      
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:254                                                                                                     
 [11] up                                                                                                                                                                                                
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:230 [inlined]                                                                                                   
 [12] up(pkgs::Vector{Pkg.Types.PackageSpec}; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})                                                                         
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:73                                                                                                      
 [13] up                                                                                                                                                                                                
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:72 [inlined]                                                                                                    
 [14] up(; name::Nothing, uuid::Nothing, version::Nothing, url::Nothing, rev::Nothing, path::Nothing, mode::Pkg.Types.PackageMode, subdir::Nothing, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})                                                                                                                                                                           
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:89                                                                                                      
 [15] up                                                                                                                                                                                                
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:81 [inlined]                                                                                                    
 [16] top-level scope                                                                                                                                                                                   
    @ ~\Documents\work\FinEtoolsTestAll.jl\testall.jl:44                                                                                                                                                
 [17] include(fname::String)                                                                                                                                                                            
    @ Base.MainInclude .\client.jl:444                                                                                                                                                                  
 [18] top-level scope                                                                                                                                                                                   
    @ REPL[1]:1                                                                                                                                                                                         
in expression starting at C:\Users\PetrKrysl\Documents\work\FinEtoolsTestAll.jl\testall.jl:38  

[fredrikekre]

caused by: failed to clone from https://github.com/JuliaRegistries/General.git, error: GitError(Code:ERROR, Class:SSL, TLS backend doesn't support certificate locations)

^ seems related to your recent work @StefanKarpinski

[StefanKarpinski]

@PetrKryslUCSD, can you try the following in a REPL session post what Julia outputs?

import NetworkOptions
NetworkOptions.ca_roots()
import Downloads
Downloads.get_ca_roots()
Downloads.Curl.SSL_VERSION

[PetrKryslUCSD]

julia> import NetworkOptions                                                                                                                          
                                                                                                                                                      
julia> NetworkOptions.ca_roots()                                                                                                                      
"C:\\Users\\PetrKrysl\\Documents\\SublimeText4_portable\\Lib\\python3\\certifi\\cacert.pem"                                                           
                                                                                                                                                      
julia> import Downloads                                                                                                                               
                                                                                                                                                      
julia> Downloads.get_ca_roots()                                                                                                                       
"C:\\Users\\PetrKrysl\\Documents\\SublimeText4_portable\\Lib\\python3\\certifi\\cacert.pem"                                                           
                                                                                                                                                      
julia> Downloads.Curl.SSL_VERSION
"Schannel"                                                                                                                                            
                                                                                                                                                      

[StefanKarpinski]

Do you have SSL_CERT_DIR or SSL_CERT_FILE set in your environment?

[PetrKryslUCSD]

Yes.

julia> ENV["SSL_CERT_FILE"]                                                                                                                           
"C:\\Users\\PetrKrysl\\Documents\\SublimeText4_portable\\Lib\\python3\\certifi\\cacert.pem"  

[PetrKryslUCSD]

When Julia 1.6 is run outside of the SublimeText editor (which has its own Python installation), everything is fine.
Well, except for some MbedTLS_jll errors.

[PetrKryslUCSD]

Sublime Text defines the variable SSL_CERT_FILE to access package control.

[PetrKryslUCSD]

Discussion of the Open SSL as implemented in ST:
https://discord.com/channels/280102180189634562/280157083356233728

[DilumAluthge]

If this is specific to when you run Julia in Sublime Text, maybe you can do this in your Julia startup file:

delete!(ENV, "SSL_CERT_FILE")

[PetrKryslUCSD]

Yes, I set "env": {"JULIA_NUM_THREADS":"4", "SSL_CERT_FILE": ""}, and that fixed the problem for me. However, there still may be an issue with Julia use of the certificate in general.

[StefanKarpinski]

That explains why this is happening. We switched to using the native SSL/TLS implementation on Windows and macOS because that's the least friction approach on those platforms and most likely to be correct and secure (the OS should keep these up-to-date and there's no way to do certificate revocation lists otherwise). However, if one of those variables or JULIA_SSL_CA_ROOTS_PATH are set then we use that as the CA root certificate path, but it seems that the native TLS system on Windows at least doesn't know how to use a CA root cert file, hence the error.

[PetrKryslUCSD]

OK. I guess that solves it then for me. But I believe other Python installations (conda?) will also try to provide that certification file. in that case you might not be seeing this error for the last time.