Update problems with corporate proxy with user and pass #20948

Milou314 · 2017-03-08T14:28:55Z

Hello,

I know this question was asked a hundred times in the web, but I did'nt find any solution. So I ask again for my special situation:

I istalled a fresh Julia 0.5.1 64-bit on my windows 7 computer. There is no program "git" or "git.exe" on the computer (which is part of several soutions) and it did not come with Julia.

If I call e.g. Pkg.status() (or any other Pkg related command) then I get:

INFO: Initializing package repository C:\Users\di29394\.julia\v0.5
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: GitError(Code:ERROR, Class:OS, Failed to set proxy: Falscher Parameter.
)
 in macro expansion at .\libgit2\error.jl:99 [inlined]
 in clone(::String, ::String, ::Base.LibGit2.CloneOptions) at .\libgit2\repository.jl
:191
 in #clone#112(::String, ::Bool, ::Ptr{Void}, ::Nullable{Base.LibGit2.AbstractCredent
ials}, ::Function, ::String, ::String) at .\libgit2\libgit2.jl:327
 in (::Base.LibGit2.#kw##clone)(::Array{Any,1}, ::Base.LibGit2.#clone, ::String, ::St
ring) at .\<missing>:0
 in (::Base.Pkg.Dir.##4#6{String,String})() at .\pkg\dir.jl:49
 in cd(::Base.Pkg.Dir.##4#6{String,String}, ::String) at .\file.jl:48
 in init(::String, ::String) at .\pkg\dir.jl:47
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::Base.TTY, ::Vararg{Base.TTY,N}) a
t .\pkg\dir.jl:28
 in status(::Base.TTY) at .\pkg\pkg.jl:136 (repeats 2 times)

The environment variables HTTP_PROXY, HTTPS_PROXY, FTP_PROXY,http_proxy,https_proxy and ftp_proxy are set in a proper way, i.e. http://user:pass@proxy:port and work well e.g. for all of my Python installations.
I also set C:/users/user/.gitconfig to:

[http]
    	proxy = proxy:port
[https]
    	proxy = proxy:port
[url "https://"]
    	insteadof = git://

and also tried to set

[http]
    	proxy = user:pass@proxy:port
[https]
    	proxy = user:pass@proxy:port
[url "https://"]
    	insteadof = git://

The error is always the same.

What to do now?

P.S.: user, pass, proxy and port were set to the actual values :-)

The text was updated successfully, but these errors were encountered:

ararslan · 2017-03-08T19:36:40Z

Hi @Milou314! We use the GitHub issue tracker for bug reports and feature request. Since this is more of a question, it should be posted to Discourse. Thanks!

lobingera · 2017-03-09T15:48:55Z

@ararslan, i can clearly see a bug report here. You don't?

nalimilan · 2017-03-09T16:41:18Z

Well, the ending question makes it sound more like a request for help. The OP does not say explicitly whether the steps he followed are those recommended in the manual, or whether this issue is precisely about the fact that no documentation exists about proxies.

Anyway somebody familiar with the libgit2 code will tell whether it's a bug or not.

lobingera · 2017-03-09T16:59:30Z

@nalimilan, maybe re-opening increases the visibility for libgit2 experts?

fredrikekre · 2017-03-09T17:07:52Z

Ref: https://discourse.julialang.org/t/update-problems-behind-a-corporate-proxy-with-user-and-pass/2560

lobingera · 2017-03-09T17:11:55Z

@fredrikekre If i'm not mistaken, some of the experts regularly write, they are better reachable on github.
And second, this thread on discourse didn't acquire many answers until now.
Do you have one?

fredrikekre · 2017-03-09T19:44:15Z

No, I just cross-referenced.

simonbyrne · 2017-03-13T15:08:08Z

I suspect it may have something to do with this issue: libgit2/libgit2#2106. Could you try the latest nightly build (which has a newer version of libgit2) and see if that fixes it?

Milou314 · 2017-03-14T07:48:10Z

Hi, I downloaded the nightly build. This is what happened:

C:\Julia\bin>set HTTPS_PROXY=http://...:...@...:...

C:\Julia\bin>set HTTP_PROXY=http://...:...@...:...

C:\Julia\bin>set FTP_PROXY=http://...:...@...:...

C:\Julia\bin>julia.exe
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0-pre.alpha.140 (2017-03-14 00:06 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 58ed252* (0 days old master)
|__/                   |  x86_64-w64-mingw32

julia> Pkg.status()
INFO: Initializing package repository C:\Users\di29394\.julia\v0.6
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: GitError(Code:ERROR, Class:OS, failed to send request: Das Zeitlimit für den Vorgang wurde erreicht.)
Stacktrace:
 [1] macro expansion at .\libgit2\error.jl:99 [inlined]
 [2] clone(::String, ::String, ::Base.LibGit2.CloneOptions) at .\libgit2\repository.j
l:287
 [3] #clone#98(::String, ::Bool, ::Ptr{Void}, ::Nullable{Base.LibGit2.AbstractCredent
ials}, ::Function, ::String, ::String) at .\libgit2\libgit2.jl:474
 [4] (::Base.LibGit2.#kw##clone)(::Array{Any,1}, ::Base.LibGit2.#clone, ::String, ::S
tring) at .\<missing>:0
 [5] (::Base.Pkg.Dir.##8#10{String,String})() at .\pkg\dir.jl:55
 [6] cd(::Base.Pkg.Dir.##8#10{String,String}, ::String) at .\file.jl:58
 [7] init(::String, ::String) at .\pkg\dir.jl:53
 [8] #cd#1(::Array{Any,1}, ::Function, ::Function, ::Base.TTY, ::Vararg{Base.TTY,N} w
here N) at .\pkg\dir.jl:28
 [9] status(::Base.TTY) at .\pkg\pkg.jl:143 (repeats 2 times)

julia>

Das Zeitlimit für den Vorgang wurde erreicht means, that the time limit for the current task was exeeded. The same proxy settings for Python work well.

Well, at least the error is now different than before.

Gretings from Germany!

lobingera · 2017-03-14T09:00:56Z

@Milou314, afaiu even the 'new' libgit2 would read configuration form a .gitconfig or from ENV?

simonbyrne · 2017-03-14T11:47:42Z

Hmm, it seems that 0.6 is completely ignoring the proxy settings (i.e. giving an invalid proxy doesn't throw any error on my machine).

Unfortunately this is too far out of my experience. If anyone else wants to take it up you can use poxyproxy, which is what libgit2 use for testing.

lobingera · 2017-03-14T11:52:28Z

@simonbyrne; looks like this is a OS-specific problem. I just installed (actually only unpacked) the linux nightly 0.6 and Pkg.init() could correctly fetch METADATA via http proxy set via export in shell.

simonbyrne · 2017-03-14T11:54:36Z

Are you behind a proxy? My concern is that it's completely ignoring the proxy settings.

lobingera · 2017-03-14T12:03:43Z

@simonbyrne; i'm currently at my workplace, which is firewalled and proxied and the linux i tested is even further away from the internet connection (cloud-instance far away...).

xxxxxxxx@euca-10-211-211-211:~$ ./julia-58ed252f6c/bin/julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0-pre.alpha.140 (2017-03-14 00:06 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 58ed252 (0 days old master)
|__/                   |  x86_64-pc-linux-gnu

julia> ENV
Base.EnvHash with 24 entries:
  "XDG_SESSION_ID"     => "2052"
  "TERM"               => "xterm"
  "SHELL"              => "/bin/bash"
  "SSH_CLIENT"         => "10.948.199.964 55847 22"
  "SSH_TTY"            => "/dev/pts/0"
  "http_proxy"         => "http://10.944.1.90:8080/"
  "USER"               => "xxxxxxxx"
  "LS_COLORS"          => "rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01…
  "ftp_proxy"          => "http://10.944.1.90:8080/"
  "MAIL"               => "/var/mail/xxxxxxxx"
  "PATH"               => "/home/xxxxxxxx/bin:/home/xxxxxxxx/.local/bin:/usr/lo…
  "PWD"                => "/home/xxxxxxxxx"
  "LANG"               => "en_US.UTF-8"
  "https_proxy"        => "http://10.944.1.90:8080/"
  "SHLVL"              => "1"
  "HOME"               => "/home/xxxxxxxx"
  "no_proxy"           => "localhost,127.0.0.1,instance-data,169.254.169.254,.n…
  "LOGNAME"            => "xxxxxxxx"
  "SSH_CONNECTION"     => "10.948.199.964 55847 10.211.211.211 22"
  "LESSOPEN"           => "| /usr/bin/lesspipe %s"
  "XDG_RUNTIME_DIR"    => "/run/user/1000"
  "LESSCLOSE"          => "/usr/bin/lesspipe %s %s"
  "_"                  => "./julia-58ed252f6c/bin/julia"
  "OPENBLAS_MAIN_FREE" => "1"

julia> Pkg.init()
INFO: Initializing package repository /home/xxxxxxxx/.julia/v0.6
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
julia> Pkg.status()
No packages installed

lobingera · 2017-03-14T12:55:03Z

Now on a windows box in the same network i get:

julia> Pkg.init()
INFO: Initializing package repository C:\Users\xxxxxxxx\.julia\v0.6
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: GitError(Code:ERROR, Class:OS, failed to send request: The operation time
d out
)
Stacktrace:
 [1] macro expansion at .\libgit2\error.jl:99 [inlined]
 [2] clone(::String, ::String, ::Base.LibGit2.CloneOptions) at .\libgit2\reposit
ory.jl:287
 [3] #clone#98(::String, ::Bool, ::Ptr{Void}, ::Nullable{Base.LibGit2.AbstractCr
edentials}, ::Function, ::String, ::String) at .\libgit2\libgit2.jl:474
 [4] (::Base.LibGit2.#kw##clone)(::Array{Any,1}, ::Base.LibGit2.#clone, ::String
, ::String) at .\<missing>:0
 [5] (::Base.Pkg.Dir.##8#10{String,String})() at .\pkg\dir.jl:55
 [6] cd(::Base.Pkg.Dir.##8#10{String,String}, ::String) at .\file.jl:58
 [7] init(::String, ::String) at .\pkg\dir.jl:53
 [8] init() at .\pkg\pkg.jl:74

(inlc. setting of HTTPS_PROXY)

lobingera · 2017-03-14T12:59:38Z

My concern is that it's completely ignoring the proxy settings.

should they be visible in Libgit2 like ProxyOptions or getconfig?

mpastell · 2017-03-15T08:09:22Z

I have the same issue. This also affects proxies without user name and password and make 0.6 unusable for me at work. I see this as something that is critical to fix before 0.6 release.

simonbyrne · 2017-03-15T10:03:03Z

@mpastell What OS are you using? And what errors does, say,

LibGit2.clone("https://github.com/JuliaLang/Example.jl","example")

give in both 0.5 and 0.6 nightly?

simonbyrne · 2017-03-15T11:33:17Z

I can confirm that passwordless proxy settings (both set via ENV and .gitconfig) are ignored on the Windows 0.6 build. This is indeed a regression.

lobingera · 2017-03-15T13:13:10Z

@simonbyrne, so which windows build (post 0.5) should be able to read proxy setting correctly?

simonbyrne · 2017-03-15T13:20:25Z

I'm not sure I understand your question: from what I can tell, the 0.6 Windows nightlies do not read the proxy settings at all.

lobingera · 2017-03-15T13:35:56Z

If you call this a regression, then there should be windows version in the past that does Pkg via LibGit2 that worked with proxy settings.

simonbyrne · 2017-03-15T15:59:38Z

Well, 0.5 does. I don't know which commit broke it, but I would suspect it was the upgrade of LibGit2 from 0.24 to 0.25.

lobingera · 2017-03-15T16:06:00Z

I'm confused. But why then is it working with a 0.6 linux?

simonbyrne · 2017-03-15T16:09:42Z

I think libgit2 uses different libraries for the http stuff on Windows and Linux.

lobingera · 2017-03-15T16:10:27Z

Great.

mpastell · 2017-03-15T17:06:45Z

The release notes of libgit2 0.25 say:

Add git_proxy_options which is used by the different networking
implementations to let the caller specify the proxy settings instead of
relying on the environment variables.

simonbyrne · 2017-03-15T21:12:53Z

Ah, so that's what's going on (I should have remembered, I made the changes to the structs). This might explain the linux behaviour:

 * Do not attempt to connect through a proxy
 *
 * If built against libcurl, it itself may attempt to connect
 * to a proxy if the environment variables specify it.

tkelman · 2017-03-16T00:36:32Z

maybe enough to change the default value of that parameter then?

lobingera · 2017-03-17T06:20:40Z

@ararslan Has this been tested sucessfully?

ararslan · 2017-03-17T06:25:18Z

GitHub auto-closed this since I merged Simon's PR that had the text "fix #20948" in it. This shouldn't be closed until we can verify it's working as expected.

mpastell · 2017-03-17T07:31:27Z

For me the issue is fixed with the current nightly (8f97e6e, Windows 7 x64 ). I just tested and it works behind proxy with the same settings as in 0.5.1 (no authentication).

Milou314 · 2017-03-17T07:33:38Z

I changed all as proposed by #21044 - well this results in:


C:\Julia\bin>Julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0-pre.alpha.140 (2017-03-14 00:06 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 58ed252* (3 days old master)
|__/                   |  x86_64-w64-mingw32

julia> Pkg.status()
INFO: Initializing package repository C:\Users\di29394\.julia\v0.6
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: GitError(Code:ERROR, Class:OS, failed to send request: Das Zeitlimit für den V
organg wurde erreicht.
)
Stacktrace:
 [1] macro expansion at .\libgit2\error.jl:99 [inlined]
 [2] clone(::String, ::String, ::Base.LibGit2.CloneOptions) at .\libgit2\repository.j
l:287
 [3] #clone#98(::String, ::Bool, ::Ptr{Void}, ::Nullable{Base.LibGit2.AbstractCredent
ials}, ::Function, ::String, ::String) at .\libgit2\libgit2.jl:474
 [4] (::Base.LibGit2.#kw##clone)(::Array{Any,1}, ::Base.LibGit2.#clone, ::String, ::S
tring) at .\<missing>:0
 [5] (::Base.Pkg.Dir.##8#10{String,String})() at .\pkg\dir.jl:55
 [6] cd(::Base.Pkg.Dir.##8#10{String,String}, ::String) at .\file.jl:58
 [7] init(::String, ::String) at .\pkg\dir.jl:53
 [8] #cd#1(::Array{Any,1}, ::Function, ::Function, ::Base.TTY, ::Vararg{Base.TTY,N} w
here N) at .\pkg\dir.jl:28
 [9] status(::Base.TTY) at .\pkg\pkg.jl:143 (repeats 2 times)

julia>

AnacondaPython conda list still works well, i.e. no issue with proxy settings in environment variables.
Any suggestions?

nalimilan · 2017-03-17T07:41:01Z

@Milou314 The version you tested is too old: Version 0.6.0-pre.alpha.140 (2017-03-14 00:06 UTC). The possible fix was only merged yesterday.

Milou314 · 2017-03-17T08:02:48Z

Oh, sorry. I just changed the lines myself. So there are other changes not reported in #21044 ??
I'll try the new release asap.

Milou314 · 2017-03-17T08:10:22Z

C:\Julia\bin>julia.exe
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0-pre.alpha.161 (2017-03-17 02:27 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 8f97e6e* (0 days old master)
|__/                   |  x86_64-w64-mingw32

julia> Pkg.status()
INFO: Initializing package repository C:\Users\di29394\.julia\v0.6
INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl
ERROR: GitError(Code:ERROR, Class:Net, invalid URL: 'myname:mypass@myproxy:myport'
)
Stacktrace:
 [1] macro expansion at .\libgit2\error.jl:99 [inlined]
 [2] clone(::String, ::String, ::Base.LibGit2.CloneOptions) at .\libgit2\repository.j
l:287
 [3] #clone#98(::String, ::Bool, ::Ptr{Void}, ::Nullable{Base.LibGit2.AbstractCredent
ials}, ::Function, ::String, ::String) at .\libgit2\libgit2.jl:474
 [4] (::Base.LibGit2.#kw##clone)(::Array{Any,1}, ::Base.LibGit2.#clone, ::String, ::S
tring) at .\<missing>:0
 [5] (::Base.Pkg.Dir.##8#10{String,String})() at .\pkg\dir.jl:55
 [6] cd(::Base.Pkg.Dir.##8#10{String,String}, ::String) at .\file.jl:58
 [7] init(::String, ::String) at .\pkg\dir.jl:53
 [8] #cd#1(::Array{Any,1}, ::Function, ::Function, ::Base.TTY, ::Vararg{Base.TTY,N} w
here N) at .\pkg\dir.jl:28
 [9] status(::Base.TTY) at .\pkg\pkg.jl:143 (repeats 2 times)

julia>

myname:mypass@myproxy:myport I just changed for the post from the right values to dummies.

mpastell · 2017-03-17T08:25:29Z

I would check the format for proxy url in .gitconfig as you now get an error from invalid URL instead of a timeout.

simonbyrne · 2017-03-17T09:07:21Z

@Milou314 Are you using environment variables or .gitconfig?

Milou314 · 2017-03-17T09:21:28Z

Well, this was (finally) the right question! For Python I use environment variables. But I found somewehre in the web the proposal to generate .gitconfig. I did this, but obviously with wrong parameters (missing leading http://). I never got hints with preceeding Julia versions, that this is a problem. Now I corrected the settings and all runs well (just successfully called Pkg.add("IJulia")).

I also checked, wether the new version works without .gitconfig and just with environment variables. This works now well too.

Thank you all for fixing the mixed software and human bug ;)

simonbyrne · 2017-03-17T09:25:04Z

Fantastic. I was also successful in emulating it in a VM via both a .gitconfig with:

[http]
proxy = http://user:pass@proxyhost:port

[https]
proxy = http://user:pass@proxyhost:port

and with the environment variables

ENV["HTTP_PROXY"] = "http://user:pass@proxyhost:port"
ENV["HTTPS_PROXY"] = "http://user:pass@proxyhost:port"

lobingera · 2017-03-17T09:43:49Z

@simonbyrne; you ran windows in VM? Or linux?

tkelman · 2017-03-17T10:06:42Z

yay, an easy fix for once. could use a doc note too now that we have a known good "how it should work" confirmation

(@Milou314 just editing .jl files in base isn't quite enough to change behavior since the standard library is precompiled, see the devdocs on building a system image for the step you were missing)

simonbyrne · 2017-03-17T10:13:46Z

Windows VM only.

jzazo · 2018-12-02T23:11:08Z

I am experiencing building error in Julia v1.0.2 on both packages ZipFile and LibCURL. I am in a corporate proxy that requires user and pass.

When I move my connection behind a VPN, I can build both packages without error. It seems the problem I get is similar to this old problem. Is there a regression?

wsshin · 2019-01-07T20:36:14Z

I am experiencing this issue on Julia version ≥ 1.0. It seems that my proxy settings are correct because I can git clone the packages directly from the terminal, but for some reason Julia does not recognize the proxy settings.

See the report in the discourse at https://discourse.julialang.org/t/cannot-add-packages-maybe-due-to-corporate-firewall/19352.

nalimilan · 2019-01-07T20:47:29Z

@wsshin Does it work if you apply the configuration changes described at #20948 (comment)? If not, please file a new issue.

wsshin · 2019-01-07T21:13:50Z

No, the configuration does not work; our proxy server does not require login.

I will file a new issue.

ararslan closed this as completed Mar 8, 2017

nalimilan added the libgit2 The libgit2 library or the LibGit2 stdlib module label Mar 9, 2017

StefanKarpinski reopened this Mar 10, 2017

simonbyrne mentioned this issue Mar 16, 2017

Change default proxy settings to auto #21044

Merged

ararslan closed this as completed in #21044 Mar 16, 2017

ararslan reopened this Mar 17, 2017

simonbyrne closed this as completed Mar 17, 2017

jzazo mentioned this issue Dec 2, 2018

Building error fhs/ZipFile.jl#48

Closed

wsshin mentioned this issue Jan 7, 2019

Julia does not recognize proxy settings #30635

Closed

Update problems with corporate proxy with user and pass #20948

Update problems with corporate proxy with user and pass #20948

Comments

Milou314 commented Mar 8, 2017 • edited

ararslan commented Mar 8, 2017

lobingera commented Mar 9, 2017

nalimilan commented Mar 9, 2017

lobingera commented Mar 9, 2017

fredrikekre commented Mar 9, 2017

lobingera commented Mar 9, 2017

fredrikekre commented Mar 9, 2017

simonbyrne commented Mar 13, 2017

Milou314 commented Mar 14, 2017 • edited

lobingera commented Mar 14, 2017

simonbyrne commented Mar 14, 2017 • edited

lobingera commented Mar 14, 2017

simonbyrne commented Mar 14, 2017

lobingera commented Mar 14, 2017 • edited

lobingera commented Mar 14, 2017

lobingera commented Mar 14, 2017

mpastell commented Mar 15, 2017

simonbyrne commented Mar 15, 2017 • edited

simonbyrne commented Mar 15, 2017

lobingera commented Mar 15, 2017

simonbyrne commented Mar 15, 2017

lobingera commented Mar 15, 2017

simonbyrne commented Mar 15, 2017

lobingera commented Mar 15, 2017

simonbyrne commented Mar 15, 2017

lobingera commented Mar 15, 2017

mpastell commented Mar 15, 2017

simonbyrne commented Mar 15, 2017 • edited

tkelman commented Mar 16, 2017

lobingera commented Mar 17, 2017

ararslan commented Mar 17, 2017

mpastell commented Mar 17, 2017

Milou314 commented Mar 17, 2017

nalimilan commented Mar 17, 2017

Milou314 commented Mar 17, 2017

Milou314 commented Mar 17, 2017

mpastell commented Mar 17, 2017

simonbyrne commented Mar 17, 2017

Milou314 commented Mar 17, 2017

simonbyrne commented Mar 17, 2017

lobingera commented Mar 17, 2017

tkelman commented Mar 17, 2017

simonbyrne commented Mar 17, 2017

jzazo commented Dec 2, 2018

wsshin commented Jan 7, 2019 • edited

nalimilan commented Jan 7, 2019

wsshin commented Jan 7, 2019

Milou314 commented Mar 8, 2017 •

edited

Milou314 commented Mar 14, 2017 •

edited

simonbyrne commented Mar 14, 2017 •

edited

lobingera commented Mar 14, 2017 •

edited

simonbyrne commented Mar 15, 2017 •

edited

simonbyrne commented Mar 15, 2017 •

edited

wsshin commented Jan 7, 2019 •

edited