Support NoProxy setting as CIDR internally #7885

Merged
merged 2 commits into from Sep 27, 2017

Conversation

Projects
None yet
5 participants
Member

wupeka commented Sep 27, 2017

Description of change

Internal HTTP client now supports no_proxy setting as CIDR, eg 10.0.0.0/24

QA steps

juju bootstrap --config
'proxy="some_working_proxy_that_does_not_allow_connections_back_to_controller" no_proxy="cidr_in_which_controller_resides"'

Documentation changes

None

Contributor

jujubot commented Sep 27, 2017

Can one of the admins verify this patch?

Member

wupeka commented Sep 27, 2017

$$merge$$

Contributor

jujubot commented Sep 27, 2017

Status: merge request accepted. Url: http://ci.jujucharms.com/job/github-merge-juju

@jujubot jujubot merged commit bea19b0 into juju:develop Sep 27, 2017

1 check passed

continuous-integration/jenkins/pr-merge This commit looks good
Details

I'm concerned that by doing this Juju will operate differently than every other client that uses NOPROXY settings, and people will be confused when they think something works with Juju, but doesn't end up working for LXD/curl/wget/other applications that they deploy.
I suppose the other alternative is that getting it supported starts somewhere?

Note that setting NOPROXY in this way will also get it set in things like /etc/apt/* stuff where at best it will just get ignored.

@@ -115,6 +114,9 @@ func (pc *ProxyConfig) useProxy(addr string) bool {
// no_proxy "foo.com" matches "bar.foo.com"
return false
}
+ if _, net, err := net.ParseCIDR(p); ip != nil && err == nil && net.Contains(ip) {
@jameinel

jameinel Oct 2, 2017

Owner

shouldn't you check ip != nil before you bother to ParseCIDR ? Given that if the original ip is nil, then the net.Contains doesn't make any sense. Maybe something like:
if ip != nil {
// this is an IP, check to see if we were given a CIDR instead of just an exact IP match
}

Contributor

dshcherb commented Oct 3, 2017

@jameinel

From my perspective we should reference other software and the fact that we are not really in control here more clearly.

https://curl.haxx.se/libcurl/c/CURLOPT_PROXY.html
https://curl.haxx.se/mail/archive-2007-11/0043.html
"> Is there an RFC which describes NO_PROXY usage?
Not that I'm aware of. When I implemented it (many years ago), I just checked
how other tools were documented to treat it and cloned that behavior. "

https://www.gnu.org/software/wget/manual/html_node/Proxies.html
"This variable should contain a comma-separated list of domain extensions proxy should not be used for. For instance, if the value of no_proxy is ‘.mit.edu’, proxy will not be used to retrieve documents from MIT."
golang/go#16704
golang/go#16704 (comment)

There is also a problem which has to be noted when a workaround such as the following is used:

printf -v no_proxy '%s,' 10.0.{1..255}.{1..255};
export no_proxy=127.0.0.1,localhost,${no_proxy%?}
export http_proxy=http://192.0.2.1:3128
export https_proxy=http://192.0.2.1:3128

echo $((printf '%s,' 10.1.{1..255}.{1..255} | wc -c/(4096*32)))
6

We simply hit a kernel limit on a command line argument size (or execve arg) with full /16 networks (or even /14) which is (PAGE_SIZE * 32) = 4096 * 32 = 131072 bytes

http://elixir.free-electrons.com/linux/v4.13.4/source/include/uapi/linux/binfmts.h#L14
#define MAX_ARG_STRLEN (PAGE_SIZE * 32)

I think it might be useful for Juju to get an option which would say: "for any address of any juju-allocated machine use noproxy unless stated otherwise".

From my perspective, this would certainly help as the main use-case is to prevent proxy usage with addresses known to Juju - doesn't have to be the whole /16 subnet or /8 subnet. It will largely help on the practical side given that no generic solution can be provided as we cannot rewrite everything to uniformly support noproxy.

Owner

jameinel commented Oct 3, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment