Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
State: Use go-datastore to implement the state interface #655
Since the beginning, we have used a Go map to store the shared state (pinset)
hashicorp.Raft assumes an in-memory state which is snapshotted from time to
This commit adds a
This will be the last migration with the current system. Once this has been
Depends on libp2p/go-libp2p-raft#50
I don't fully know. But since checking state version will mean opening the datastore and reading one key, we know that will be different. The migration will be looping the keys and overwriting them with their updated form.
The key idea is that Marshal/Unmarshal will either not be used, or can happen normally regardless of the state version (unlike now).
I only see jenkins, which I retriggered now. It make take a while before I merge this anyway.
Does this approval include changes in go-libp2p-raft ?
Since the beginning, we have used a Go map to store the shared state (pinset) in memory. The mapstate knew how to serialize itself so that libp2p-raft would know how to write to disk when it: * Saved snapshots of the state on shutdown * Sent the state to a newcomer peer hashicorp.Raft assumes an in-memory state which is snapshotted from time to time and read from disk on boot. This commit adds a `dsstate` implementation of the state interface using `go-datastore`. This allows to effortlessly switch to a disk-backed state in the future (as we will need), and also have at our disposal the different implementations and utilities of Datastore for fine-tuning (caching, batching etc.). `mapstate` has been reworked to use dsstate. Ideally, we would not even need `mapstate`, as it would suffice to initialize `dsstate` with a `MapDatastore`. BUT, we still need it separate to be able to auto-migrate to the new format. This will be the last migration with the current system. Once this has been released and users have been able to upgrade we will just remove `mapstate` as it is now. License: MIT Signed-off-by: Hector Sanjuan <firstname.lastname@example.org>
Additionally, remove persisting the state version to the go-datastore. In the future versions of the state, there is not a global format anymore (with a global version). Instead, every pin element can potentially be stored in a different version. License: MIT Signed-off-by: Hector Sanjuan <email@example.com>