Skip to content
This repository has been archived by the owner on Sep 8, 2018. It is now read-only.

ingest: implement bulk mode #4

Open
peterbourgon opened this issue Jan 16, 2017 · 0 comments
Open

ingest: implement bulk mode #4

peterbourgon opened this issue Jan 16, 2017 · 0 comments

Comments

@peterbourgon
Copy link
Member

The ingester can take an entire segment in a single write or series of writes, somehow. It can block until that segment has been successfully replicated to store nodes, and only then accept the next segment from the forwarder. This would provide the "most durable" mode of ingestion.

There are some tricky things here. One, plumbing through the necessary infrastructure to communicate to the forwarder that the segment has been successfully replicated. Two, deciding when and how to assign ULIDs. Three, thinking through the failure modes carefully.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant