New Platform broke Canvas

[w33ble]

Kibana version: master, 1418310 or newer

Elasticsearch version: master/snapshot

Describe the bug:

Canvas doesn't work anymore.

Steps to reproduce:

1. Run Canvas with a dev setup, using commit 1418310 or newer of Kibana
2. Load Canvas

Expected behavior:

It should work

Screenshots (if relevant):

Errors in browser console (if relevant):

GET http://localhost:5601/pnb//socket.io?EIO=3&transport=polling&t=MILpf75 404 (Not Found)

Note the extra / in the basepath... maybe that's related?

[azasypkin]

I'll take a look on Monday, but I've never run Kibana with Canvas. Would you mind pointing me to any docs describing how one can run it @w33ble ?

[azasypkin]

Okay, I see what happened here and how we can fix that, but I'm concerned about the way Canvas uses socket.io + Kibana HTTP server that'd likely won't work as is with the new platform in the future.

What has happened

Canvas initializes socket.io connection using listener property of Hapi server provided by Kibana. After #18951 listener is no longer http.Server instance, but our own wrapper around http.Server created in the new platform. To construct correct "socket" path Canvas relies on server.rewriteBasePath that's only processed by the core and in fact its internal detail. I think it was required previously since socket.io intercepts all incoming requests and injects its own request handler in the front of the queue and hence base path rewrite logic hasn't removed base path yet and socket.io should take care of it. Direct manipulations on the underlying server will be likely forbidden in new platform, so plugins would either use API we provide for maintaining websocket/http connections or create their own server.

Short-term fix

With the listener wrapper socket.io request handler can't become the first one (it will be "injected" right after request handlers attached by Hapi in new platform and before request handlers attached by the "legacy" platform), so it doesn't have to deal with base path at all since new platform will rewrite it (if enabled in config).

Additionally listener wrapper may need to re-transmit close and upgrade events that are not required by Hapi 14 in the "legacy" platform, but used by socket.io here and here. And obviously this may complicate socket.io upgrades as it may start doing different things with the listener....

But before we do anything here I'd like to hear more details about socket.io use case in Canvas? @elastic/kibana-canvas is it supposed to be upgraded to real websocket connection or the plan is to use HTTP polling fallback for now?

/cc @elastic/kibana-platform @epixa

[azasypkin]

Hmm, Canvas's use of server.inject in non-test code will fail at some point too. We definitely need to discuss this sort of things and find a better way before 6.5.

[rashidkpc]

But before we do anything here I'd like to hear more details about socket.io use case in Canvas? @elastic/kibana-canvas is it supposed to be upgraded to real websocket connection or the plan is to use HTTP polling fallback for now?

We use socket.io for bi-directional communication between the server and client. We use websockets for the lower overhead and reduced latency. However we need to continue supporting long polling for now as the cloud proxy doesn't support websockets yet.

[uboness]

Maybe it's an opportunity to think about introducing websockets support in the new platform... Having a high level abstraction like socket.io has in the platform can be helpful (especially as we need the long polling fallback)... Initially it can be implemented with socket.io, later on we can consider replacing it with our own websockets/long polling implementation (that is if we find socket.io to be unstable or we get to a point where we have the proper support in cloud to remove the long polling fallback requirement)

I can see many places where websockets can be useful in kibana as a whole... I know one specific use case in the Notebook app that it can really help with collab editing feature.

[epixa]

#18301 is the issue for websocket support in the new platform

[epixa]

I'm going to close this because I'm pretty sure canvas is working again in master. Please reopen if I'm misunderstanding something.