Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should use the user-provided Host header for SNI #414

Open
mgedmin opened this issue Nov 23, 2015 · 11 comments · May be fixed by #1531
Open

Should use the user-provided Host header for SNI #414

mgedmin opened this issue Nov 23, 2015 · 11 comments · May be fixed by #1531

Comments

@mgedmin
Copy link

mgedmin commented Nov 23, 2015

I'm trying to test Apache configuration in a Vagrant VM that forwards port 8443 to VM's port 443. The command I'm running is:

$ http get https://localhost:443/ Host:www.example.com --verify=no

and I get a 400 Bad Request from Apache, because

[Mon Nov 23 09:43:17 2015] [error] Hostname localhost provided via SNI and hostname www.example.com provided via HTTP are different

I think httpie should use the user-provided Host header for the SSL negotiation, or perhaps even provide a command-line option to explicitly specify a hostname to use in SNI.

@Lukasa
Copy link

Lukasa commented Nov 23, 2015

The easiest way to get this to work in the short term is to update your hosts file to point to the host in question. This will enable you to avoid needing to set the Host header as well.

In the longer term, I could see some value in HTTPie doing something similar to what curl does, or possibly even something a bit simpler that just works for SNI if we're happy to let people keep setting custom Host headers.

Note, however, that a lot of other things will get done wrong if you set a custom host header. For example, httpie probably won't handle cookies the way you expect. So using the hosts file is the better solution in the general case. Works for all your other tools too!

@mgedmin
Copy link
Author

mgedmin commented Nov 24, 2015

Good point about cookies! What about

http get https://foo.example.com/ --connect-to 127.0.0.1:8443

then?

I don't like curl's --resolve solution (aka #362) because it requires me to repeat the host name, which is unnecessary in 99% of use-cases. (It's probably necessary if you want to follow redirects to a different domain name, that you're also testing in Vagrant or something like that.)

I don't like the /etc/hosts solution because:

  • it requires root
  • it affects the entire system, not just my little experimental shell session
  • it's persistent and I have to remember to undo any testing changes I make

@Lukasa
Copy link

Lukasa commented Nov 24, 2015

Yeah, curl's --resolve solution may be overkill in this case.

I don't like the /etc/hosts solution because:

You can use the HOSTALIASES environment variable to point at a file in your own part of the namespace that is used as a hosts file. That allows you to do something like env HOSTALIASES=~/hosts http get https://foo.example.com/.

@mgedmin
Copy link
Author

mgedmin commented Nov 24, 2015

Thank you, I didn't know about HOSTALIASES!

(It still feels like extra work, compared to a command-line argument. Would you look favorably upon a pull request that implemented a --connect-to option for httpie?)

@Lukasa
Copy link

Lukasa commented Nov 24, 2015

I am not a maintainer of httpie, so I can't speak for @jkbrzt. However, the idea certainly doesn't bother me. =)

@mgedmin
Copy link
Author

mgedmin commented Feb 15, 2016

NB: the suggested HOSTALIASES solution doesn't work for me -- I see httpie connect to the real IP address, and not the fake one provided via my /tmp/hosts, despite HOSTALIASES=/tmp/hosts http get ....

@normanr
Copy link

normanr commented Apr 14, 2019

HOSTALIASES only works for "name[s] consist[ing] of a single component, that is, contains no dot", ref: http://man7.org/linux/man-pages/man7/hostname.7.html

@fdcds
Copy link

fdcds commented Jul 25, 2019

gnutls-cli has the --sni-hostname=... option. It would be great to have that for http too, and it would prevent unexpected effects when re-using the value of a header for a different purpose (TLS and HTTP live on different OSI layers, so should be treated independently).

dkasak added a commit to dkasak/mitmproxy that referenced this issue Nov 23, 2020
Unless this is done, the SNI server name will not be sent, often making
the curl/httpie command have different behaviour than the original
request (most often in the form of failing to establish a TLS
connection).

With this change, we always use the original host, fixing this failure.
However, if the original host is a domain, it may sometimes resolve to
a different IP address later on. In curl, we solve this problem by
forcing it to connect to the original IP using `--resolve`. For httpie
there is currently no easy solution (see:
httpie/cli#414).
dkasak added a commit to dkasak/mitmproxy that referenced this issue Nov 30, 2020
Unless this is done, the SNI server name will not be sent, often making
the curl/httpie command have different behaviour than the original
request (most often in the form of failing to establish a TLS
connection).

With this change, we always use the original host, fixing this failure.
However, if the original host is a domain, it may sometimes resolve to
a different IP address later on. In curl, we solve this problem by
forcing it to connect to the original IP using `--resolve`. For httpie
there is currently no easy solution (see:
httpie/cli#414).
dkasak added a commit to dkasak/mitmproxy that referenced this issue Feb 5, 2021
Unless this is done, the SNI server name will not be sent, often making
the curl/httpie command have different behaviour than the original
request (most often in the form of failing to establish a TLS
connection).

With this change, we always use the original host, fixing this failure.
However, if the original host is a domain, it may sometimes resolve to
a different IP address later on. In curl, we solve this problem by
forcing it to connect to the original IP using `--resolve`. For httpie
there is currently no easy solution (see:
httpie/cli#414).
mhils pushed a commit to mitmproxy/mitmproxy that referenced this issue Feb 9, 2021
…4307)

Use original flow host instead of IP when exporting to curl/httpie.

Unless this is done, the SNI server name will not be sent, often making
the curl/httpie command have different behaviour than the original
request (most often in the form of failing to establish a TLS
connection).

With this change, we always use the original host, fixing this failure.
However, if the original host is a domain, it may sometimes resolve to
a different IP address later on. In curl, we solve this problem by
forcing it to connect to the original IP using `--resolve`. For httpie
there is currently no easy solution (see:
httpie/cli#414).
@dwt
Copy link

dwt commented Nov 3, 2021

Sad this looks stale, something like --connect host:port seems to me to be the right approach.

@AndrewKvalheim
Copy link

Related: #99 (comment)

@voroninman
Copy link

I'm looking for a way to specify different hosts for connection, TLS and HTTP. Something like,

http https://foo.com --sni-host bar.com Host:baz.com

where I expect the client connect to the IP of the foo.com, set the SNI to bar.com and set HTTP Host header to baz.com.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants