Spacemacs ignores url-proxy-services

[chasinglogic]

Hello!

I used to use Spacemacs behind a proxy however recently it's stopped using my proxy settings. I used to keep the proxy in user-init however it would try to check the various repositories (melpa,elpa,gnu) before user-init seemed to run.

I added my proxy settings to init.el however spacemacs still ignores them when contacting the repos. I have tried using emacs --insecure and issue persists. I verified that it's completely ignoring the proxy settings by leaving the proxy set but connecting to another network with no proxy on it and Spacemacs installed. (it should have timed out looking for the proxy)

I can provide any additional info requested. Thanks for reading and thanks for making Spacemacs!

EDIT: I am running OSX

[pullmonkey]

EDIT: Actually, it works for me if I do emacs --insecure.

I'm having the same problem. I'm using ubuntu 14.04. For now, I took all the ipv6 lines out of /etc/hosts (I read that was a possible issue). These are the lines I've put into init.el at the top.

(setq url-proxy-services
   '(("http" . "<proxy>:80")
    ("https" . "<proxy>:80")))

In messages, I see these lines:

Contacting host: <proxy>:80
Importing package-keyring.gpg...done
Failed to download `melpa' archive.
Importing package-keyring.gpg...done
Contacting host: <proxy>:80 [2 times]
Importing package-keyring.gpg...done
Failed to download `gnu' archive.
Spacemacs is ready.

I'm not sure what else to try

[xoltar]

I have this problem on Mac OS X as well. I get around the update problem by launching emacs from the terminal (where I have http[s]_proxy set) when I want to do updates.

When it's launched as an app, it seems to ignore system proxy settings.

[dstcruz]

I am having the same issue on Windows, using spacemacs-0.105.9. The proxy settings seem to be ignored when running the app. Passing the --insecure flag works. If it makes any difference, I have the proxy settings in .emacs.d/init.el.

[adbrown101]

The only way I can use spacemacs behind our proxy at work is to install cntlm. Then putting my proxy settings on the first line of my .emacs.d/init.el.

(setq url-proxy-services
      '(("http" . "127.0.0.1:3128")
        ("https" . "127.0.0.1:3128")))

We could do with a dotspacemacs proxy variable which comes into play before emacs contacts for updates layers etc. I'm not quite sure where this would be, but I am guessing before the dotspacemacs-configuration-layers, so that we do not have to mess with the .emacs.d/init.el. System win 7, emacs 24.5.1, spacemacs-0.105.9

[chasinglogic]

For me this issue is actually related to the emacs for mac build. When I use the official emacs build everything works as normal (except for the known problems with spacemacs + official build for macs)

[syl20bnr]

Do you all have your http_proxy and https_proxy environment variables setup and visible by Emacs ?

Emacs should pickup the proxy settings automatically if those variables are set.

[sroeschus]

I have the same problem on linux where I connect to the internet through a proxy. The http_proxy and https_proxy is set. However installing spacemacs is only possible if i specify the --insecure flag. After restarting emacs, I get the same error messages as above and it does not matter if --insecure is specified or not when emacs is invoked.

[xoltar]

@syl20bnr Yes, those variables are exported in my .zprofile. When I run from the terminal, it works fine. It's only when launched as an app (from spotlight actually) that the problem happens.

[pfmoore]

Is there any progress on this? I am having the same problem (Windows system using cntlm as a proxy). Using the --insecure option lets spacemacs install, but I don't know if I need to include that flag permanently (and I don't know how I would do that, as I want to start Emacs in daemon mode using emacsclient -a ""). Also, I've not found any documentation on the --insecure flag - what does it do, and does it have any other effects (I assume so!) that I should be aware of?

+1 for some means of setting proxy values in the spacemacs config (preferably on a per-system basis, so that I can share my config across machines and only have the proxy enabled on the machine that uses it).

[pfmoore]

... ah, I see that --insecure is a spacemacs-specific option (which is why emacs --help doesn't show it!) So from the help for that option, I guess my issue is that my proxy isn't being invoked for https, for some reason? (That and there's still a need to be able to set url-proxy-services in spacemacs - for reasons it's not worth going into details on, environment variables aren't a reliable solution for me)

[jtbm37]

@xoltar It makes sense as your .zprofile always applies to your terminal, not the gui version.

I don't think anybody picked up on what sylv20bnr said. You need to make sure those env variables are set. To do so, evaluate this in emacs: (getenv "http_proxy")

If you wish to set an env variable from within emacs: (setenv "http_proxy" "foo")

[TheBB]

(preferably on a per-system basis, so that I can share my config across machines and only have the proxy enabled on the machine that uses it).

Your config is just a bunch of functions that can execute arbitrary code, so there's nothing preventing you from using conditionals to set variables. They don't have to be constant values.

[pfmoore]

Looks like there might be an issue with HTTPS on Windows.

From https://melpa.org/#/getting-started:

Note: There are some problems using the https location with Emacs on Windows. There is currently no know easy fix for this. You can still use MELPA by using the non-SSL location by replacing https with http.

So, to summarise:

1. On Windows, there are issues with https (but it's unclear what they are), so users need to set dotspacemacs-elpa-https to nil (and this should probably be the default on that platform...) It would be nice to know what those unspecified issues might be - it's hardly a good solution to just abandon https :-( It's worth pointing out that (gnutls-availble-p) is t on my system, so it's not as simple as "not all Windows installations have https support".
2. To set a proxy, you need to set the http_proxy environment variable. Unfortunately, on Windows, it's hard to do this in a simple way if your proxy is not always available, so it is better to set url-proxy-services in Emacs. It's not obvious how to do this "sufficiently early" in the spacemacs config - but that may simply be a matter of documentation.
3. There's an installation issue, in that you can't set anything in the config till you've initialised. You can reliably install using --insecure and a https_proxy environment variable, but it's far from obvious how to do that for a naive user.

Hopefully, this is of some use to others encountering this issue (on Windows, at least).

[pfmoore]

Your config is just a bunch of functions that can execute arbitrary code, so there's nothing preventing you from using conditionals to set variables. They don't have to be constant values.

Well, yes and no. Things like dotspacemacs-elpa-https are variables, and it's not immediately obvious how (or where) to use a conditional to set it. You certainly can do it, but you need a way to run the conditional code early enough in the startup sequence to be done before the variable gets used. And there's no documentation at that level of detail, so you're left with diving into the source.

[TheBB]

I think it's fairly obvious that you have to set it in the same function where it's being set in the template. In the template, dotspacemacs-elpa-https is set in dotspacemacs/user-init so your code for setting it should be in that function, too. The final result of dotspacemacs/user-init should be that dotspacemacs-elpa-https is set to whatever you want it to be.

[pfmoore]

I'm confused. dotspacemacs-elpa-https is set in dotspacemacs/init but the docstring of that function says "You should not put any user code in there besides modifying the variable values". I took that as meaning that you should only set values here, not put arbitrary code. But I guess you could. What I'd want to do in practice though is put configuration like that in a separate file, not have a single monolithic config file - and I don't know how I'd structure that file (for example, if I understand the concept, I can't make it a "layer" because the layer system isn't set up yet).

But I need to learn the basics before over-complicating things. At the moment, I've only just got spacemacs installed for the first time (the install broke because of the proxy issues) so it's a bit early for me to be looking at complex config. Emacs always feels a bit like that to me, though - basic behaviour doesn't quite work, so before I know what's happened, I'm writing complicated elisp rather than actually doing the editing I planned on :-)

[syl20bnr]

For user code running just after dotspacemacs/init there is the function dotspacemacs/user-init that you should find right after the init function.

[TheBB]

Right, sorry. It does say that. Anyway it won't stop you. :-P

[taqtiqa-mark]

The challenge is when you are in a Windows context and you install spacemacs from behind an authenticated proxy.
The workaround is to:

1. git clone ..... spacemacs to emacs.d
2. Run <path to gnu emacs for windows>\runemacs.exe --insecure
3. Choose some defaults
4. quit emacs
5. delete emacs.d
6. In the .spacemacs that was created, add the following in the dotspacemacs/user-init section

(setq url-proxy-services
   '(("no_proxy" . "^\\(localhost\\|10.*\\)")
     ("http" . "your.proxy.com.cn:80")
     ("https" . "your.proxy.com.cn:80")))

(setq url-http-proxy-basic-auth-storage
    (list (list "your.proxy.com.cn:80"
                (cons "Input your LDAP UID !"
                      (base64-encode-string "your-user-name:your-password")))))

7. git clone ..... spacemacs to emacs.d
8. Run <path to gnu emacs for windows>\runemacs.exe --insecure

Now things should proceed correctly.

I think the correct solution is to have spacemacs:

1. Check if http is available
2. If not, prompt for:
   • proxy address
   • proxy port
   • proxy username
   • proxy password
3. Ask if you want to save the user name and password to .spacemacs or enter if each time it is required.
4. Add the gathered information to .spacemacs
5. Check if http is available
6. If not ask if you want to continue - explaining the consequences of saying yes, and how to run the skipped stages later.

[hackrole]

@taqtiqa-mark, thank you. you save my life.

[chasinglogic]

Closing this, I'm no longer behind a proxy at work so I have no way to test or verify that this is still occuring.

[taqtiqa-mark]

I think @hackrole and other reports here confirm this is still an issue. Is there a PR or commit that might indicate the issue has been resolved?

[TBusen]

+1 behind a corporate proxy and I'm seeing this. Spacemacs launched from terminal with --insecure flag works, but when launched as a desktop app it doesn't work. Running on Ubuntu 18. My http_proxy and https_proxy are set on the os

[TBusen]

(setq url-proxy-services
("http" . "your.proxy.com.cn:80")
("https" . "your.proxy.com.cn:80"))) in user-init worked for me. emacs 25.2, latest spacemacs and ubuntu 18

[d12frosted]

I am reopening this issue as people are still affected. Let's leave it open until workaround or fix is found.

@taqtiqa-mark can you please try the suggestion made by @TBusen ?

[taqtiqa-mark]

@d12frosted, I believe @TBusen repeats some part of what I describe here. I gave an begin-to-end description that worked for me and the others voting thumbs up.

Have I misunderstood what you want me to try?

[d12frosted]

@taqtiqa-mark my bad, I missed that you are the author of that long comment describing a work around.

[HerbM]

I don't know if this will be of any help but on the off chance:

Git on Windows works for me from behind a proxy.

For the AUTHENTICATION to work seamlessly and securely (not clear text passwords) the environment variable "GIT_credential_helper" is set to 'wincred'

In cmd.exe this would look like this (before running Git):

set GIT_credential_helper=wincred

or for PowerShell

$Env:GIT_credential_helper = 'wincred'  

If someone were trying to provide a full fix, taking a look at the Git code might be useful.

[taqtiqa-mark]

@HerbM my understanding: the issue here is to configure spacemacs such that spacemacs package installs and updates 'just-work'. The git clone ... step is assumed to work.....

Are you saying that by setting GIT_credential_helper=wincred you are able to add a new package to .spacemacs and the package install and/or updates proceed smoothly?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please let us know if this issue is still valid!

[HerbM]

@HerbM my understanding: the issue here is to configure spacemacs such that spacemacs package installs and updates 'just-work'. The git clone ... step is assumed to work.....

Are you saying that by setting GIT_credential_helper=wincred you are able to add a new package to .spacemacs and the package install and/or updates proceed smoothly?

Yes.

@taqtiqa-mark (apologies, this went stale and old, my fault for not answering but for completeness here it is.)

Yes, if "git clone" just works (proving the paths and credentials are correct then it would be very nice if spaceemacs did this same.

Better if it read or used the .gitconfig that made the rest work (or a better method) but even having to configure the settings again in spaceemacs would be "good enough".

Thanks.

[mosquito-magnet]

There is a bug in url-http with the function url-https-proxy-connect: It will not send a
Proxy-Authorization header, so all https calls through an authenticating proxy will fail.

See https://debbugs.gnu.org/cgi/bugreport.cgi?bug=42422, fixed in Emacs 28.1 apparently.

With orgmode.org/elpa and melpa changing to https only, this problem became more visible.

As a workaround till 28.1 arrives, override the function with fixed version:

(with-eval-after-load 'url-http
  (defun url-https-proxy-connect (connection)
    (setq url-http-after-change-function 'url-https-proxy-after-change-function)
    (process-send-string connection (format (concat "CONNECT %s:%d HTTP/1.1\r\n"
						    "Host: %s\r\n"
						    (let ((proxy-auth (let ((url-basic-auth-storage
									     'url-http-proxy-basic-auth-storage))
									(url-get-authentication url-http-proxy nil 'any nil))))
						      (if proxy-auth (concat "Proxy-Authorization: " proxy-auth "\r\n")))
						    "\r\n")
					    (url-host url-current-object)
					    (or (url-port url-current-object)
						url-https-default-port)
					    (url-host url-current-object)))))