Consider specialized API for forwarding events in the pipeline

[jszwedko]

@lukesteensen and I briefly talked about this, but I wanted to make sure I got it written down somewhere. He can probably add more detail as I think he had a better vision for what this would be.

We had a recent issue with the splunk_hec source where, if there was an error receiving data, the last good event would not be sent due to futures not flushing the sink (#5058). I covered one more case for the stdin source in #5082.

This issue is a very easy one to crop back up (neglecting to flush the sinks when forwarding streams). Luke mentioned creating a specialized API to use on Pipeline rather than using the futures stream and sink APIs that could handle things like ensuring flushing as well as things like determining what's blocking (the stream or the sink) when forwarding an event.

[jszwedko]

cc/ @binarylogic

[tobz]

@lukesteensen I think this is mostly covered by all the work we've previously done to SourceSender/Fanout/etc. Anything else we still could do at a high-level here?

[lukesteensen]

Yeah I think this is done. The initial conversation with @jszwedko had something to do with confusion around what happens when a stream of Result is sent, but I don't think that's an issue anymore since we stopped using Sink.