Consider specialized API for forwarding events in the pipeline #5107
Labels
domain: reliability
Anything related to Vector's reliability
domain: topology
Anything related to Vector's topology code
have: must
We must have this feature, it is critical to the project's success. It is high priority.
type: task
Generic non-code related tasks
@lukesteensen and I briefly talked about this, but I wanted to make sure I got it written down somewhere. He can probably add more detail as I think he had a better vision for what this would be.
We had a recent issue with the
splunk_hec
source where, if there was an error receiving data, the last good event would not be sent due tofutures
not flushing the sink (#5058). I covered one more case for thestdin
source in #5082.This issue is a very easy one to crop back up (neglecting to flush the sinks when forwarding streams). Luke mentioned creating a specialized API to use on
Pipeline
rather than using thefutures
stream and sink APIs that could handle things like ensuring flushing as well as things like determining what's blocking (the stream or the sink) when forwarding an event.The text was updated successfully, but these errors were encountered: