Set Proxy Behavior is Misleading with Malformed URLs

[bailejl]

Recently, tried to set proxies for HTTP and HTTPS. However, the URLs were malformed, due to the password in the URL containing the British pound sign. The character was encoded with Windows-1252 and not UTF-8, which is a malformed URL to NPM. I set the HTTPS proxy first and it accepted the command without error, then I went to do the HTTP proxy and it showed an error. The two URLs only differed by one letter, due to the protocol. This led me to think the problem was only with the HTTP URL, which tainted my troubleshooting and delayed me resolving the issue.

The preferred behavior from my perspective, is to check the URL structure when the "set proxy" commands are used. This provides the user with a consistent feedback from NPM.

[smikes]

Unfortunately, npm does not actually do anything with the proxies except pass them (as opaque-to-npm values) to the request module, which actually handles the HTTP requests, proxy support, etc.

I don't think npm knows enough about proxy validation to do this, but if request (https://github.com/request/request) produced more informative error messages, npm would display them.

[bailejl-two]

I should have put up the error when I filled it out the request. I was able to recreate it, so I put it below. The %A3 is causing it to fail. However, it will not fail until you run it the second time. If the %A3 is in the domain portion it will not fail. However, if I change it to %C2%A3 it works. I hope this helps understand what I am trying to say.

C:\Users\jlb>npm config set proxy http://user:password%A3@proxy.company.com:8080
URIError: URI malformed
at decodeURIComponent (native)
at Url.parse (url.js:215:19)
at Object.urlParse as parse
at Object.validateUrl [as validate](C:Program Filesnodejsnode_modulesnpmnode_modulesnoptlibnopt.js:164:13)
at validate (C:\Program Files\nodejs\node_modules\npm\node_modules\nopt\lib\nopt.js:212:24)
at validate (C:\Program Files\nodejs\node_modules\npm\node_modules\nopt\lib\nopt.js:179:11)
at C:\Program Files\nodejs\node_modules\npm\node_modules\nopt\lib\nopt.js:101:12
at Array.map (native)
at C:\Program Files\nodejs\node_modules\npm\node_modules\nopt\lib\nopt.js:67:15
at Array.forEach (native)

C:\Program Files\nodejs\node_modules\npm\lib\npm.js:31
throw new Error('npm.load() required')
^
Error: npm.load() required
at Object.npm.config.get (C:\Program Files\nodejs\node_modules\npm\lib\npm.js:31:11)
at exit (C:\Program Files\nodejs\node_modules\npm\lib\utils\error-handler.js:57:40)
at process.errorHandler (C:\Program Files\nodejs\node_modules\npm\lib\utils\error-handler.js:358:3)
at process.emit (events.js:107:17)
at process._fatalException (node.js:236:26)

[smikes]

Thanks - the additional context is useful /cc @othiym23 Windows issue

[bailejl]

Same behavior on a mac.

Joes-MacBook-Pro:~ jlb$ npm config set proxy http://user:password%A3@proxy.company.com:8080
URIError: URI malformed
at decodeURIComponent (native)
at Url.parse (url.js:185:19)
at Object.urlParse as parse
at Object.validateUrl as validate
at validate (/usr/local/lib/node_modules/npm/node_modules/nopt/lib/nopt.js:212:24)
at validate (/usr/local/lib/node_modules/npm/node_modules/nopt/lib/nopt.js:179:11)
at /usr/local/lib/node_modules/npm/node_modules/nopt/lib/nopt.js:101:12
at Array.map (native)
at /usr/local/lib/node_modules/npm/node_modules/nopt/lib/nopt.js:67:15
at Array.forEach (native)

/usr/local/lib/node_modules/npm/lib/npm.js:31
throw new Error('npm.load() required')
^
Error: npm.load() required
at Object.npm.config.get (/usr/local/lib/node_modules/npm/lib/npm.js:31:11)
at exit (/usr/local/lib/node_modules/npm/lib/utils/error-handler.js:57:40)
at process.errorHandler (/usr/local/lib/node_modules/npm/lib/utils/error-handler.js:358:3)
at process.emit (events.js:95:17)
at process._fatalException (node.js:295:26)
Joes-MacBook-Pro:~ jlb$

[smikes]

I can reproduce this with npm@2.6.0 on OSX, minimal repro steps are:

$ npm config set proxy http://%A3@a.uk
$ npm config set proxy null

Error appears to be that nopt uses url.parse, which throws.

@othiym23 , any suggestions on how to handle this? Ultimately the end-user needs a particular string passed to request to make their proxy work so I would lean towards changing the specification of proxy from url to String, and leave validation up to request.

Maybe check in npm doctor (upcoming) whether the proxy string looks like a URL.

[othiym23]

If it's an invalid URL, it's an invalid URL. This sounds like it's going to require more careful examination to resolve properly, and since the npm core team is pretty fully committed right now (and also doesn't have the time to set up a good test environment for this), it may be a while until we're able to take a look at this. If somebody wants to take a stab at putting together a patch, it would be much appreciated.

[smikes]

Does a proxy specification have to be a standard RFC 2396 URL? Or is it a potentially-opaque cookie that is used to communicate settings between the user and the network layer, that npm should avoid inspecting? Questions for the ages...

But in fact, I have gotten the thrust of this bug report wrong. Really the problem is that, once config.http_proxy is set to an invalid value (e.g., by setting with a Windows-1250 encoding where £ is encoded as %A3), npm fails on startup.

So if I do this, the only fix is to recover by manually editing npmrc:

> npm config set http_proxy http://%A3.com/
> npm config ls
[fails, everything is fubar]

[bailejl]

Correct, once a bad URL makes it in to the config file, it is broken, but you as the operator are unaware until you do another operation. This was confusing when I was setting both HTTP and HTTPS proxies. It delayed me from finding the real problem. To resolve this issue it would be great if the URLs were validated before they went in the file. This would reduce confusion.

[npm-robot]

We're closing this issue as it has gone thirty days without activity. In our experience if an issue has gone thirty days without any activity then it's unlikely to be addressed. In the case of bug reports, often the underlying issue will be addressed but finding related issues is quite difficult and often incomplete.

If this was a bug report and it is still relevant then we encourage you to open it again as a new issue. If this was a feature request then you should feel free to open it again, or even better open a PR.

For more information about our new issue aging policies and why we've instituted them please see our blog post.

[celamb4]

Is there a workaround for special chars in the URL?