SSL-enabled TorchServe handles http: protocol incorrectly

[fbbradheintz]

If you're running a TorchServe installation with the following config:

inference_address=https://0.0.0.0:8443
management_address=https://0.0.0.0:8444
keystore=keystore.p12
keystore_pass=asdfasdf
keystore_type=PKCS12

And you attempt an http call - not https - such as:

curl -X POST "http://localhost:8444/models?url=r18ts.mar&initial_workers=1"

There is no response, and an exception shows in the log:

2020-04-15 17:04:46,302 [ERROR] epollEventLoopGroup-3-2 org.pytorch.serve.http.HttpRequestHandler -
io.netty.handler.codec.DecoderException: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 504f5354202f6d6f64656c733f75726c3d72313874732e6d617226696e697469616c5f776f726b6572733d3120485454502f312e310d0a486f73743a206c6f63616c686f73743a383434340d0a557365722d4167656e743a206375726c2f372e36302e300d0a4163636570743a202a2f2a0d0a0d0a
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:459)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
        at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:808)
        at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:404)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:304)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 504f5354202f6d6f64656c733f75726c3d72313874732e6d617226696e697469616c5f776f726b6572733d3120485454502f312e310d0a486f73743a206c6f63616c686f73743a383434340d0a557365722d4167656e743a206375726c2f372e36302e300d0a4163636570743a202a2f2a0d0a0d0a
        at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1156)
        at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1221)
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428)
        ... 14 more

Suggested Remediation:

• Handle this case without throwing an exception, as it is something that is likely to happen during normal operation.
• Log an informative error message, perhaps in access_log.log.
• Give a useful error message in an HTTP response to the user, e.g., "This TorchServe instance only accepts HTTPS requests.', with a response code of 403.

[fbbradheintz]

Also fails for Inference API calls (e.g., curl https://localhost:8443/ping).

[fbbradheintz]

Also fails for localhost-only config:

inference_address=https://127.0.0.1:8443
management_address=https://127.0.0.1:8444
keystore=keystore.p12
keystore_pass=aoeuaoeu
keystore_type=PKCS12

[fbbradheintz]

Also fails for "Option 2" (the openssl option), with config file:

inference_address=https://127.0.0.1:8443
management_address=https://127.0.0.1:8444
private_key_file=tstest.key
certificate_file=tstest.pem

Same result with the 0.0.0.0 configuration.

[mycpuorg]

Sorry I was meaning to change tags on #203
Reverting the tags to original here.

[fbbradheintz]

This is still an issue.

A better solution would have the server not throwing an exception, but spotting the issue and sending a 301 redirect to the client.