[ML] Explain log rate spikes: Move API stream demos to Kibana examples.

[walterra]

Summary

Part of #136265.
Follow up to #132121.

To run Kibana with the described examples, use yarn start --run-examples.

This creates a response_stream plugin in the Kibana /examples section. The plugin demonstrates API endpoints that can stream data chunks with a single request with gzip/compression support. gzip-streams get decompressed natively by browsers. The plugin demonstrates two use cases to get started: Streaming a raw string as well as a more complex example that streams Redux-like actions to the client which update React state via useReducer().

Code in @kbn/aiops-utils contains helpers to set up a stream on the server side (streamFactory()) and consume it on the client side via a custom hook (useFetchStream()). The utilities make use of TS generics in a way that allows to have type safety for both request related options as well as the returned data.

No additional third party libraries are used in the helpers to make it work. On the server, they integrate with Hapi and use node's own gzip. On the client, the custom hook abstracts away the necessary logic to consume the stream, internally it makes use of a generator function and useReducer() to update React state.

On the server, the simpler stream to send a string is set up like this:

const { end, push, responseWithHeaders } = streamFactory(request.headers);

The request's headers get passed on to automatically identify if compression is supported by the client.

On the client, the custom hook is used like this:

const { error, start, cancel, data, isRunning } = useFetchStream<
    ApiSimpleStringStream, typeof basePath
>(`${basePath}/internal/response_stream/simple_string_stream`);

Checklist

• Unit or functional tests were updated or added to match the most common scenarios
• This was checked for cross-browser compatibility

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[elasticmachine]

Pinging @elastic/ml-ui (:ml)

[mistic]

The changes related to a new package creation LGTM

[walterra]

@qn895 addressed your comments, ready for another look!

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: e80dffc
• Storybooks Preview
• Webpack Bundle Analyzer

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
aiops	78	96	+18
ml	1612	1610	-2
total			+16

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
@kbn/aiops-utils	-	18	+18

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
aiops	219.0KB	275.6KB	+56.6KB
ml	3.3MB	3.3MB	-1.6KB
total			+55.0KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
aiops	3.9KB	3.6KB	-308.0B
ml	40.4KB	40.3KB	-122.0B
total			-430.0B

Unknown metric groups

API count

id	before	after	diff
@kbn/aiops-utils	-	35	+35
aiops	12	11	-1
total			+34

async chunk count

id	before	after	diff
aiops	3	1	-2

ESLint disabled line counts

id	before	after	diff
aiops	6	5	-1

Total ESLint disabled count

id	before	after	diff
aiops	6	5	-1

History

• 💚 Build #46813 succeeded eebf7fb
• 💔 Build #46792 failed d5b8c5c
• 💔 Build #46758 failed 29b34ce
• 💔 Build #46500 failed 51667c2
• 💔 Build #46488 failed cf5447d
• 💔 Build #46479 failed acf825f

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @walterra

[peteharverson]

Tested locally and LGTM. Just one suggestion for the title.

[peteharverson]

I'd go with Response stream as the title for the example.

[qn895]

Latest changes LGTM 🎉 Amazing work!!