Corrosion

Gossip-based service discovery (and more) for large distributed systems.

Why we built Corrosion

We built Corrosion specifically for service discovery across a large global network, replacing Consul’s central state database with eventually consistent state distributed across our hosts.

Our new tool needed to deliver the following:

Fast reads and writes

Getting state (data) from a central remote source can be incredibly expensive (at least 300ms for a round-trip to something on the other side of the world), but usually takes less than 1ms from a local source.

Fast, eventual, consistency

Many use cases can cope with eventual consistency, especially if a consistent state is attained sooner than later. Raft fell short for some use cases at Fly.io where round-trips to a centralized location are too expensive.

Flexibility

Global state for a distributed system isn't one-size-fits-all. Flexible schemas and queries are essential.

How Corrosion works

In a nutshell, Corrosion:

Maintains a SQLite database on each node
Gossips local changes throughout the cluster
Uses CR-SQLite for conflict resolution with CRDTs
Uses Foca to manage cluster membership using a SWIM protocol
Periodically synchronizes with a subset of other cluster nodes, to ensure consistency

Features

A flexible API to read from and write to Corrosion's store using SQL statements
File-based schemas with on-the-fly updates
HTTP streaming subscriptions based on SQL queries
Live population of configuration files from Corrosion state with user-defined Rhai templates
Storage and propagation of state from locally registered Consul services, replacing the central database with Corrosion's distributed state
Secure peer-to-peer communication with the QUIC transport protocol (using Quinn)

Usage overview

Run the Corrosion agent on every node/host in the cluster. Other programs running on the node use Corrosion's HTTP API to query the local Corrosion SQLite database, add and update data, and subscribe to change notifications.

The Corrosion CLI provides commands for administration and access to database and features.

Quick start

See the WIP Corrosion documentation for more details.

Building Corrosion

Clone https://github.com/superfly/corrosion.git.

From within the repo directory:

cargo build --release && mv target/release/corrosion ./

License

mmfallacy/corrosion

ErrorLooks like something went wrong!

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Error
Looks like something went wrong!

Packages