Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conditional proxy support #841

Open
phemmer opened this issue Aug 29, 2016 · 15 comments
Open

Conditional proxy support #841

phemmer opened this issue Aug 29, 2016 · 15 comments
Assignees
Milestone

Comments

@phemmer
Copy link

@phemmer phemmer commented Aug 29, 2016

Our environment requires the use of a proxy for outbound internet access. Thus we need proxy support for alert destinations like PagerDuty.
Go does support the http_proxy and https_proxy environment variables, however these appear to control ALL http calls, which is not what we want as we do not want to use the proxy server for access to influxdb.

So we need some way of controlling what uses the proxy.
2 solutions I can think of:

  1. Add a parameter to alert handlers which might need a proxy. E.G. alert().pagerDuty.proxy('http://foo:bar@proxyhost:3128').

or

[pagerduty]
    enabled = true
    proxy = "http://foo:bar@proxyhost:3128"
  1. Add a per-site proxy list to the config. E.G.
[proxy]
    events.pagerduty.com = "http://foo:bar@proxyhost:3128"

^if toml is even capable of such a config

@nathanielc nathanielc added this to the Unplanned milestone Aug 30, 2016
@nathanielc
Copy link
Contributor

@nathanielc nathanielc commented Aug 30, 2016

@phemmer Would it make sense to do something like?

[http_proxy]
    url = "http://foo:bar@proxyhost:3128"

[pagerduty]
    enabled = true
    use-proxy = true

That way you only have to configure the proxy once but then can use it for whichever services need it.

@phemmer
Copy link
Author

@phemmer phemmer commented Aug 30, 2016

seems acceptable to me.

@blaketmiller
Copy link

@blaketmiller blaketmiller commented Feb 17, 2017

What's the status of this? Trying to get my TICK stack to production and I'm behind HTTP proxies as well, so this is a blocker for me.

Conditional proxying based on raising flags for each plugin seems...unwieldy, but how about getting just a basic proxy ability going for the entire service? Looks like this should be pretty easy to add in Go: https://golang.org/pkg/net/http/#ProxyFromEnvironment

@blaketmiller
Copy link

@blaketmiller blaketmiller commented Feb 21, 2017

For anyone else waiting on this to get resolved, I made a quick workaround by tunneling through a relay: http://btmiller.com/2017/02/20/send-kapacitor-alerts-to-slack-through-a-proxy.html (only for Slack).

@phemmer
Copy link
Author

@phemmer phemmer commented Feb 22, 2017

We take a somewhat similar approach by using socat, and adding an /etc/hosts record for events.pagerduty.com to redirect to the local socat.

@desa desa self-assigned this Mar 2, 2017
desa added a commit that referenced this issue Mar 6, 2017
Most services and clients throughout Kapacitor use the Default HTTP
Client which uses the Default HTTP Transport, which uses HTTP proxies as
directed by the $HTTP_PROXY and $NO_PROXY (or $http_proxy and $no_proxy)
environment variables.

For the services that use a custom Transport, we use
http.ProxyFromEnvironment.

The downside to this is that all future implementers will need to copy
this pattern

Fixes #841 (though its not conditional proxy as the issue has requested)
desa added a commit that referenced this issue May 3, 2017
Most services and clients throughout Kapacitor use the Default HTTP
Client which uses the Default HTTP Transport, which uses HTTP proxies as
directed by the $HTTP_PROXY and $NO_PROXY (or $http_proxy and $no_proxy)
environment variables.

For the services that use a custom Transport, we use
http.ProxyFromEnvironment.

The downside to this is that all future implementers will need to copy
this pattern

Fixes #841 (though its not conditional proxy as the issue has requested)

Update Changelog
@desa desa closed this in #1238 May 3, 2017
@ghost ghost removed the in progress label May 3, 2017
@desa desa reopened this May 3, 2017
@desa
Copy link
Contributor

@desa desa commented May 3, 2017

Re-opened as per discussion #1238 (comment)

@desa
Copy link
Contributor

@desa desa commented May 3, 2017

@phemmer Just trying to gather a bit more information.

Our environment requires the use of a proxy for outbound internet access. Thus we need proxy support for alert destinations like PagerDuty.
Go does support the http_proxy and https_proxy environment variables, however these appear to control ALL http calls, which is not what we want as we do not want to use the proxy server for access to influxdb.

Would it be sufficient for us to allow proxying for everything except InfluxDB? Rather than having conditional proxy support for each external service?

@phemmer
Copy link
Author

@phemmer phemmer commented May 3, 2017

In my use case that would be sufficient. The only thing that jumps to mind with that sort of solution is if there are other services that people might use that are hosted within their own network, and which would also need to exempt from the proxy (E.G. kubernetes).

@desa
Copy link
Contributor

@desa desa commented May 4, 2017

Good point about Kubernetes.

As an aside, #1238 may be sufficient for your use case. Go's ProxyFromEnvironment supports an environment variable NO_PROXY which will skip the proxy for any requests to that host.

@phemmer
Copy link
Author

@phemmer phemmer commented May 4, 2017

Ah, ok, NO_PROXY may indeed work. Wasn't familiar with this variable, and the documentation on ProxyFromEnvironment doesn't offer much (any) insight on how it works. But it seems like I should be able to do NO_PROXY=localhost (since in my case the InfluxDB server lives on localhost).

@desa
Copy link
Contributor

@desa desa commented May 4, 2017

I wasn't familiar with it until just recently myself. I think your point about having a larger strategy for interacting with internal vs external services still makes sense though.

@jcmartins
Copy link

@jcmartins jcmartins commented Apr 24, 2018

cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)

rpm -qa |grep kapacitor
kapacitor-1.4.1-1.x86_64

[root@kapacitor]# env |grep proxy
http_proxy=http://proxy.intranet:80
https_proxy=http://proxy.intranet:80
no_proxy=localhost,127.0.0.1,localaddress,.localdomain.com
HTTPS_PROXY=http://proxy.intranet:80
HTTP_PROXY=http://proxy.intranet:80
NO_PROXY=localhost,127.0.0.1,localaddress,.localdomain.com

My kapacitor.log
ts=2018-04-24T14:46:27.800Z lvl=error msg="failed to send event" service=slack task=chronograf-v1-5c6eb4ed-678c-46d9-9407-6171b39892d6 err="Post https://hooks.slack.com/services/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: dial tcp 54.239.152.5:443: getsockopt: connection timed out"
ts=2018-04-24T14:48:03.326Z lvl=error msg="error while sending usage report on startup" service=reporting err="Post https://usage.influxdata.com/api/v1/usage/kapacitor: dial tcp 104.131.151.204:443: i/o timeout"

fdhex pushed a commit to fdhex/kapacitor that referenced this issue Jun 12, 2018
Most services and clients throughout Kapacitor use the Default HTTP
Client which uses the Default HTTP Transport, which uses HTTP proxies as
directed by the $HTTP_PROXY and $NO_PROXY (or $http_proxy and $no_proxy)
environment variables.

For the services that use a custom Transport, we use
http.ProxyFromEnvironment.

The downside to this is that all future implementers will need to copy
this pattern

Fixes influxdata#841 (though its not conditional proxy as the issue has requested)

Update Changelog
@Namita-S
Copy link

@Namita-S Namita-S commented Feb 6, 2020

Hi @nathanielc, What's the status of this? Trying to get my Chronograf to push slack alerts but it needs proxy to reach the Webhook url.

@bmgante
Copy link

@bmgante bmgante commented Nov 17, 2020

Hi, i am facing this same issue. HTTP post to microsoft teams using proxy gets timeout even with env variables properly defined. Any idea?

@bmgante
Copy link

@bmgante bmgante commented Nov 17, 2020

It seems to work defining the env vars directly on /etc/default/kapacitor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants