Override IP address in http_response

[nicolas17]

Feature Request

Proposal:

Add a config setting to the http_response input plugin to override the IP address, to use instead of the resolved hostname. Alternatively, add a way to override the hostname when giving an IP address to urls.

Current behavior:

If I set urls=["http://somehostname/"], telegraf will resolve that hostname in DNS and connect to that IP address. If there are multiple IP addresses, I guess it connects to an arbitrary one. I can't work around it with urls=["http://192.0.2.42/"] and overriding inputs.http_response.headers.Host, because then it would fail to verify SSL. Even with insecure_skip_verify it would fail because it wouldn't send the correct hostname in SNI.

Desired behavior:

A new config setting lets me change the IP address the plugin connects to, so I can use the hostname in the URL but connect to another IP.

Alternatively, a new config setting lets me change the hostname used in the Host: header, the SSL SNI, and the SSL certificate verification, so I can use the IP address in the URL but everything else still behaves as if the URL had the hostname.

In either case, both the hostname and the IP address should be somewhere in the resulting tags, so that requests to same hostname but different IP, or viceversa, can be distinguished later.

Use case:

A hostname may point to multiple IP addresses corresponding to different servers. Or it may have both IPv4 and IPv6. Or it may use geodns to return a single IP out of multiple depending on the source's location, and a telegraf instance would always get the same one.

I want to send a request to all IP addresses individually, and collect the response_time for each, so I can know if there is a failure or increased latency for only one server, or for only IPv6.

[danielnelson]

It looks like this is possible today by setting the Host header:

[[inputs.http_response]]
  urls = ["http://127.0.0.1:9999/debug"]
  [inputs.http_response.headers]
    Host = "github.com"

I haven't tested with a TLS certificate, is this where the issue appears?

For the tags, you can add them manually:

  [inputs.http_response.tags]
    http_host = "github.com"

[antifuchs]

I just tried this (correcting the .tags to .headers) and unfortunately, this doesn't work for SNI: The Host header gets sent way after the client has indicated to the server's TLS stack which hostname it wishes to speak to (and it looks like telegraf takes the SNI hostname from the URL).

I think what would be really neat here is a list of IP addresses to send the configured requests to: That way, you could monitor a set of redundant load balancers directly for whether they're able to respond to the configured SNI hostnames.

[antifuchs]

I submitted a PR for this, #7897; I hope you can weigh in and check if that fits your use case as well, @nicolas17!