An embeddable, leaderless distributed cache for Go applications. Drop it into any Go service to share in-memory state across instances — no external infrastructure required.
node, _ := hive.NewNode(hive.Config{
Mode: hive.ModeCluster,
Seeds: []string{"node1:7946"},
})
defer node.Shutdown()
cache := node.Cache()
sessions := hive.NewValueStore[Session](cache, "sessions")
sessions.Set("user:123", Session{UserID: 123, Token: "abc"})
s, err := sessions.Get("user:123")Each instance of your application runs a Hive node. Nodes discover each other through a seed list and form a self-organizing cluster using a gossip protocol. Keys are distributed across nodes using consistent hashing, replicated according to your replication factor, and automatically redistributed when nodes join or leave.
- Leaderless — every node is equal, no election needed
- Self-healing — nodes that go silent are detected and their keys redistributed
- Embeddable — no sidecar, no separate process, just a Go import
- Minimal dependencies — uses
msgpackfor serialization, nothing else added to yourgo.mod
go get github.com/EmilioRosiles/hiveGood for development or single-node deployments. Data stays local, no networking.
node, err := hive.NewNode(hive.Config{})
if err != nil {
log.Fatal(err)
}
defer node.Shutdown()
cache := node.Cache()
counters := hive.NewValueStore[int](cache, "counters")
counters.Set("visits", 42)
v, err := counters.Get("visits")Each application instance joins the same cluster by pointing at one or more seed addresses. Seeds only need to be reachable at startup — once the node has joined, membership is maintained through gossip.
node, err := hive.NewNode(hive.Config{
Mode: hive.ModeCluster,
BindPort: 7946,
Seeds: []string{"10.0.0.1:7946", "10.0.0.2:7946"},
})In a containerized environment, seeds are typically set via an environment variable:
seeds := strings.Split(os.Getenv("HIVE_SEEDS"), ",")
node, err := hive.NewNode(hive.Config{
Mode: hive.ModeCluster,
Seeds: seeds,
})status := node.Status()
fmt.Printf("node %s, cluster size %d\n", status.NodeID, status.Size)
for _, p := range status.Peers {
fmt.Printf(" peer %s addr=%s alive=%v\n", p.NodeID, p.Addr, p.Alive)
}Multiple stores can share the same node — they are namespaced views over the same underlying cluster. Obtain a Cache handle from the node and pass it to each store constructor.
cache := node.Cache()
sessions := hive.NewValueStore[Session](cache, "sessions")
online := hive.NewSetStore(cache, "online_users")
streams := hive.NewHashStore[Stream](cache, "streams")
queue := hive.NewListStore[Task](cache, "work_queue")
scores := hive.NewZSetStore(cache, "leaderboard")Each store type maps to a Redis-style API.
A typed key/value store. Values are msgpack-encoded structs or scalars.
type Session struct {
UserID int
Token string
}
sessions := hive.NewValueStore[Session](cache, "sessions")
// Set stores a value.
err := sessions.Set("user:123", Session{UserID: 123, Token: "abc"})
// Get retrieves and decodes a value. Errors if missing or expired.
s, err := sessions.Get("user:123")
// Del removes a key.
sessions.Del("user:123")
// Expire sets a TTL. The key is deleted automatically after the duration elapses.
sessions.Expire("user:123", 30*time.Minute)A distributed string set. Members can carry independent per-member TTLs, making it useful for tracking presence or short-lived memberships.
online := hive.NewSetStore(cache, "online_users")
// SAdd adds a member to the set at key.
online.SAdd("room:1", "user:123")
// SExpireMember sets a per-member TTL. Other members and the key are unaffected.
online.SExpireMember("room:1", "user:123", 30*time.Second)
// SMembers returns all live members.
members, err := online.SMembers("room:1")
// SIsMember checks membership.
ok, err := online.SIsMember("room:1", "user:123")
// SCard returns the number of live members.
n, err := online.SCard("room:1")
// SRem removes a single member.
online.SRem("room:1", "user:123")
// Del removes the entire set. Expire sets a key-level TTL.
online.Del("room:1")
online.Expire("room:1", 5*time.Minute)A typed key/field/value store. Fields within a key carry independent TTLs, making it well-suited for tracking per-entity state with automatic eviction.
type Stream struct {
StartedAt time.Time
BitRate int
}
streams := hive.NewHashStore[Stream](cache, "streams")
// HSet stores a value under key/field.
streams.HSet("user:123", "stream:abc", Stream{StartedAt: time.Now()})
// HExpireField sets a TTL on a single field. Other fields are unaffected.
streams.HExpireField("user:123", "stream:abc", 30*time.Minute)
// HGet retrieves and decodes a single field.
s, err := streams.HGet("user:123", "stream:abc")
// HGetAll retrieves all live fields under a key.
all, err := streams.HGetAll("user:123")
// HKeys returns the names of all live fields.
fields, err := streams.HKeys("user:123")
// HDel removes a single field.
streams.HDel("user:123", "stream:abc")
// Del removes the entire hash. Expire sets a key-level TTL.
streams.Del("user:123")
streams.Expire("user:123", 1*time.Hour)A typed distributed ordered list. Elements are msgpack-encoded. Supports efficient push/pop from both ends, making it suitable for queues, stacks, and activity feeds.
type Task struct {
ID string
Payload []byte
}
queue := hive.NewListStore[Task](cache, "work_queue")
// RPush appends to the tail. LPush prepends to the head.
queue.RPush("jobs", Task{ID: "t1", Payload: data})
queue.LPush("jobs", Task{ID: "t0", Payload: data})
// LPop removes and returns the head. RPop removes and returns the tail.
task, err := queue.LPop("jobs")
// LLen returns the number of elements.
n, err := queue.LLen("jobs")
// LIndex returns the element at index. Negative indices count from the tail.
last, err := queue.LIndex("jobs", -1)
// LRange returns a slice from start to stop inclusive. Negative indices supported.
page, err := queue.LRange("jobs", 0, 9)
// LSet overwrites the element at index.
queue.LSet("jobs", 0, Task{ID: "t0-updated"})
// Del removes the entire list. Expire sets a key-level TTL.
queue.Del("jobs")
queue.Expire("jobs", 1*time.Hour)A distributed sorted set. Each member is a unique string associated with a float64 score. Members are always kept in ascending score order, with ties broken lexicographically.
scores := hive.NewZSetStore(cache, "leaderboard")
// ZAdd inserts or updates member with score.
scores.ZAdd("game:1", 9500.0, "alice")
scores.ZAdd("game:1", 8200.0, "bob")
// ZScore returns the score for a member. Errors if member does not exist.
s, err := scores.ZScore("game:1", "alice")
// ZRank returns the 0-based rank in ascending order (lowest score = 0).
// ZRevRank returns the rank in descending order (highest score = 0).
rank, err := scores.ZRank("game:1", "bob")
rank, err = scores.ZRevRank("game:1", "alice")
// ZCard returns the number of members.
n, err := scores.ZCard("game:1")
// ZRange returns members from rank start to stop inclusive.
// Negative indices count from the top (highest rank).
top3, err := scores.ZRange("game:1", -3, -1)
// ZRangeByScore returns all members with min <= score <= max in ascending order.
mid, err := scores.ZRangeByScore("game:1", 8000.0, 9000.0)
// ZRem removes a member.
scores.ZRem("game:1", "bob")
// Del removes the entire sorted set. Expire sets a key-level TTL.
scores.Del("game:1")
scores.Expire("game:1", 24*time.Hour)ZRange and ZRangeByScore return []ZSetEntry, where each entry has Member string and Score float64.
All stores support two levels of TTL:
- Key-level TTL (
Expire) — deletes the entire key when it elapses - Field-level TTL (
SExpireMember,HExpireField) — evicts a single member or field independently, without affecting other members or the key itself. If all members/fields expire, the key is cleaned up automatically.
hive.Config{
// Unique identifier for this node.
// Auto-generated if empty.
NodeID string
// ModeStandalone (default) or ModeCluster.
Mode Mode
// Address to bind the peer communication port to.
// Default: "0.0.0.0"
BindAddr string
// Port for peer communication.
// Default: 7946
BindPort int
// Seed peer addresses (host:port) used to bootstrap cluster membership.
// Required when Mode is ModeCluster.
Seeds []string
// Number of nodes that store a copy of each key.
// Higher values improve fault tolerance but increase write overhead.
// Must be <= cluster size. Default: 1
ReplicationFactor int
// Maximum memory this node intends to use, in bytes.
// Controls two things: capacity enforcement (writes are rejected once the
// limit is reached) and keyspace allocation (nodes with more memory receive
// proportionally more vnodes on the hash ring, and therefore more keys).
// Default: total system memory
MemLimit uint64
// How often this node sends heartbeats to peers.
// Default: 3s
GossipInterval time.Duration
// Number of peers contacted per gossip round.
// Default: 3
GossipFanout int
// How long to wait after a topology change before rebalancing.
// Prevents cascading migrations when multiple nodes join or leave at once.
// Default: 500ms
RebalanceDebounce time.Duration
// How often the cluster janitor runs to evict expired store entries
// and remove dead peer tombstones from the membership table.
// Default: 30s
CleanupInterval time.Duration
// Verbosity of internal log output written to stderr.
// nil defaults to slog.LevelError (quiet).
// Set to &slog.LevelInfo or &slog.LevelDebug for more detail.
LogLevel *slog.Level
}Hive is an ephemeral, eventually consistent cache.
- Reads and writes go to the key's primary owner as determined by consistent hashing
- Replication is asynchronous — replicas may be briefly behind the primary
- When a network partition heals and keys are redistributed, Hive uses last-write-wins (LWW) conflict resolution: every stored entry carries a nanosecond-precision write timestamp (
writeAt), and rebalance only overwrites a local copy if the incoming entry is strictly newer. This prevents split-brain partitions from silently clobbering fresher data. - There is no durability — a node restart loses its local data. Surviving replicas retain their copies
This makes Hive well-suited for session caches, rate-limit counters, presence tracking, leaderboards, job queues, and other short-lived shared state where occasional staleness is acceptable.
Values must be serializable by msgpack:
- All fields you want preserved must be exported
- Pointers, slices, maps, and structs are all supported
Membership state is propagated using a gossip protocol. Every node periodically sends its view of the cluster to a random subset of peers (GossipFanout). Each outgoing heartbeat carries an incarnation number — a monotonically increasing counter seeded with the current Unix timestamp when the node starts. Seeding from wall time means a restarted node's first heartbeat carries a higher incarnation than any stale dead rumor about it, allowing it to rejoin without manual intervention.
A peer's state is updated only when the incoming incarnation is strictly higher than what is locally known. This prevents stale gossip from overwriting fresh state and avoids the clock-skew problems that arise from comparing wall-clock timestamps directly across machines.
Nodes that fail to respond to a heartbeat are marked dead immediately. Their keys are redistributed after RebalanceDebounce to allow the cluster to stabilize before migrating data.
The hash ring uses virtual nodes (vnodes) to distribute keyspace. Each node's vnode count is derived from its MemLimit relative to the rest of the cluster: a node with twice the memory of its peers owns roughly twice as much keyspace. This means data naturally flows toward nodes with more capacity without any manual weighting.
A background janitor runs every CleanupInterval and performs two tasks:
- Expired entry eviction — scans the local store and removes entries whose TTL has elapsed
- Tombstone cleanup — removes dead peer records from the membership table once they are no longer needed for gossip convergence
Each stored entry carries a writeAt timestamp (Unix nanoseconds, set at the time of the write). When rebalancing after a partition heals, incoming entries are written only if their writeAt is strictly newer than the local copy. This last-write-wins strategy ensures the most recently written value survives without requiring coordination between nodes.
Ports — each node needs its BindPort reachable by all other nodes. In Docker/Kubernetes, expose and map the port explicitly.
Seeds — at least one seed must be reachable when a node starts. Seeds do not need to be stable or permanent — any alive cluster member works.
Replication factor — keep it ≤ the minimum expected cluster size. A factor of 2 with a 2-node cluster means every node holds every key.
Graceful shutdown — calling node.Shutdown() announces the departure to peers so they can redistribute keys immediately.
Memory limits — MemLimit affects both write rejection and ring weight. Nodes that exceed their limit return an error on write; they do not evict existing entries to make room. Use TTLs on keys that should not accumulate indefinitely.
MIT