ElasticsearchSource: Clear Scroll On Exhaustion or Completion

[michaeljmarshall]

References #2314

In the referenced issue, I had asked about the proper way to keep the stage alive for clean up when downstream has already finished. I noticed that some other stages were doing this by calling setKeepGoing(true), so I used that here. The documentation seems to support the use case, but please let me know if it looks incorrect.

I plan to add tests and documentation in the coming days. I wanted to get this PR up first to see if there was any feedback on the initial implementation.

[ennru]

Thank you for trying it out.
Your use of setKeepGoing is correct.
But I wonder if there is a good reason to do await the response from the delete request, we can't do anything but log it anyway. So alternatively this would be just a fire and forget and the stage can complete immediately.

[michaeljmarshall]

My primary motivation for awaiting the response from the delete request is a clean shutdown. When I stop the stream early with a KillSwitch, I want to make sure that the source stage properly makes the delete call before letting the application shutdown. Given that there is a real cost to keeping a scroll open on the Elasticsearch side (and that the Elasticsearch documentation explicitly recommends clearing scrolls when they're no longer needed), it seems worth the slight delay to ensure that the call is made.

Regarding the logging, I do think it could be valuable for troubleshooting.

If you think awaiting the result of the delete request is too opinionated, I could add a setting to the ElasticsearchSourceSettings to allow end users to choose the right behavior for their use case.

[ennru]

LGTM.

[ennru]

Thank you @michaeljmarshall, great to see Alpakka Elasticsearch being improved!