rustls requires hostname for server

[ayende]

I was trying to write a small test server with rustls and I run into this issue: briansmith/webpki#54

I was trying to use: openssl s_client -connect 127.0.0.1:4888 -tls1_2 to test my server and it went up and died on me.
The problem was that it gave this error: Error: Io { source: Custom { kind: InvalidData, error: CorruptMessagePayload(Handshake) } }

This is a really bad experience. I understand that the dependency needs to be fixed, but in the mean time ( and in general ), reporting a better error in this case can make a lot of difference for figuring out what is wrong.

Changing ServerNamePayload::read_hostname to return a Result with a clear indication of the failure, would be great.

[briansmith]

One option would be, when failing to parse the DNS name using webpki's DNS name parser, to try to parse it as a IP address. If the IP address parsing succeeds after DNS name parsing fails, then report a more specific error.

[ayende]

I think that it is far more common to have an IP there than an actual invalid hostname.
Just having an error that talks about IPs are not allowed would be a huge help.

"Failed DNS name parsing, are you using an IP address, that isn't currently supported."

[hannesdejager]

Would be nice if this one can be adressed somehow.

[briansmith]

I suggest we close this as a dupe of #184, as it seems to be the same issue.

[djc]

I've been interpreting this as an issue to provide a better error for this, so I do think this is meaningfully distinct from #184. Whether it's worth keeping this open probably depends on how close #184 is. It's still a common pitfall for users.

[jbr]

It seems like one hurdle to implementing this is the fact that Codec is implemented in terms of Option instead of Result. Has there been thought put into making Codec return Result in order to propagate failure types?

[djc]

Yeah, I think it would make sense to change Codec to yield Result types.

[cpu]

The issues with the Codec return were addressed, and Rustls 0.21.0 has support for IP address subjects in certificates. I'm going to close this ticket since I believe the associated work is completed. Please feel free to comment or file a new issue if there are remaining gaps to consider.

Thanks!