Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
River: A pluggable river (indexer like) support #377
A river is a pluggable entity running within elasticsearch cluster pulling data (or being pushed with data) that is then indexed into the cluster.
A river is composed of a unique name and a type. The type is the type of the river (out of the box, there is the
Rivers are singletons within the cluster. They get allocated automatically to one of the nodes and run. If that node fails, an river will be automatically allocated to another node.
River allocation on nodes can be controlled on each node. The
Rivers require both meta data (what type they are, and additional information) that forms the "settings" of the river, and possibly need to store runtime state (indexed up to data X, continue from it in case of failover). Everything is driven by working an internal index called
In that index (
With the fact that the river(s) information is stored as an index, it is fully persistent, and allows for very frequent state storage (under one or more documents).
Sounds confusing, but its really simple, here is an example for creating the
And deleting the river is: