crypto/tls: Using GODEBUG=tlsrsakex=1 behaves inconsistently between Go 1.22 and master

[cuiweixie]

          I think this is controlled by the GODEBUG setting `tlsrsakex` where the default changed from enabled in go1.21 to disabled in go1.22.

I don't think there's anything to do here as the new behavior is more correct and we aren't going to change the default GODEBUG for a previous version.

main » GODEBUG=tlsrsakex=1 go1.22.5 run .
2024/07/19 12:18:46 dial failed: websocket: bad handshake
exit status 1


main » GODEBUG=tlsrsakex=0 go1.22.5 run . 
2024/07/19 12:18:56 connection ok

Originally posted by @seankhliao in #68522 (comment)

[cuiweixie]

main » GODEBUG=tlsrsakex=1 gomaster run . 
result in  connection ok

I found is caused by cl585437
before this cl.

 GODEBUG=tlsrsakex=1 go run .
dial failed: websocket: bad handshake
exit status 1

please help to explain why this happen.

[seankhliao]

Sounds like an issue with the server being sensitive to extension ordering.

Unlike many projects, the Go project does not use GitHub Issues for general discussion or asking questions. GitHub Issues are used for tracking bugs and proposals only.

For questions please refer to https://github.com/golang/go/wiki/Questions