Nexus is a Go project providing high-performance, lock-free concurrency primitives designed for systems requiring extremely fast, low-level message passing.
It currently includes:
spsc: A lock-free, Single-Producer-Single-Consumer (SPSC) queue.sharded: A Sharded Mailbox for highly scalable, lock-free, Multiple-Producer-Multiple-Consumer (MPMC) message exchange.
The primitives in this project are built for speed and scalability by adhering to core principles of mechanical sympathy:
- Lock-Free: All structures rely on atomic operations instead of slower, mutex-based locks, minimizing kernel context switches.
- Decentralized Design: The
shardedmailbox uses a fully decentralized architecture. There are no central counters or shared "hint" variables, which eliminates bottlenecks and ensures performance scales with the number of cores. - Cache-Friendly: Data structures are padded to prevent false sharing between CPU cores.
The ShardedMailbox is not a traditional queue but a collection of single-element "mailboxes" (shards). It is designed to facilitate a high-throughput exchange of messages between many producers and many consumers.
The core principle is "producer locks and sends, consumer receives and unlocks."
Note: The
ShardedMailboxdesign favors throughput and resilience over strict FIFO (First-In, First-Out) ordering across different shards. While the order of messages sent by a single producer to a single consumer is generally preserved, no such guarantee is made for the global order of messages across all producers and consumers.
- The
ShardedMailboxis initialized withNindependent mailboxes (shards). - Each producer and consumer is assigned a unique ID. This ID determines their "home" shard.
- Producers (
Enqueue): A producer attempts to "lock" and place a message in a mailbox. It uses a "casino" strategy, starting with a shard next to its home shard, to immediately distribute load and avoid creating "hot spots." If all mailboxes are locked, it waits (blocks). - Consumers (
Dequeue): A consumer attempts to "unlock" and retrieve a message. It first checks its home shard for high cache affinity. If empty, it attempts to "steal" work from other shards. If all mailboxes are empty, it waits (blocks).
This decentralized approach minimizes contention and allows for excellent scalability on multi-core systems.
A classic MPMC queue with a single ring buffer suffers from severe contention on highly parallel systems (32+ cores). All producers and consumers must atomically update the same head and tail counters, causing a "traffic jam" on the memory bus.
The ShardedMailbox solves this by leveraging the Go runtime's scheduler for self-balancing.
- Internal Shard Locks: Each shard is an independent mailbox with its own lock state ("empty" or "full"). This is the only point of contention.
- Go Scheduler: The Go runtime independently distributes goroutines (producers and consumers) across available CPU cores. More powerful cores naturally execute more operations.
- Emergent Behavior: Because our
EnqueueandDequeueoperations are blocking, a goroutine on a fast core will quickly find a free/full shard, complete its operation, and move on to the next. A goroutine on a slower core will simply wait longer on its shard's lock. This creates a natural, emergent load-balancing effect: more messages are processed on more performant cores without any central coordinator. - Guaranteed Delivery: Once a producer or consumer "locks" a shard for its session, it does not release it until the operation is complete, guaranteeing message delivery without interference from other links in the chain.
This approach trusts the Go runtime to handle thread scheduling while providing a fine-grained, decentralized locking mechanism that scales. Instead of trying to outsmart the platform, we use its inherent characteristics (like scheduler behavior and non-uniform core performance) to enhance competitiveness and ensure stable system scaling. We don't just calculate where it's best to run; we allow the system to self-tune and adapt. As a result, overhead is minimized, and performance gains a direct correlation with scaling.
The ShardedMailbox demonstrates excellent performance and scalability, significantly outperforming a standard unbuffered Go channel (which is also a blocking operation and the closest ideological equivalent).
Benchmarks are run on a 64-core AMD EPYC 7763 server processor.
The ShardedMailbox demonstrates significant performance gains over a standard Go channel, especially with increased core counts.
At 4 cores, ShardedMailbox is ~2.4 times faster than a standard channel.
goos: linux
goarch: amd64
pkg: github.com/GenshIv/nexus/sharded
cpu: AMD Ryzen 9 7950X3D 16-Core Processor
BenchmarkShardedMailbox_Throughput-4 93373442 64.57 ns/op 0 B/op 0 allocs/op
BenchmarkStandardChannel_MPMC-4 38434584 154.1 ns/op 0 B/op 0 allocs/op
At 32 cores, the performance difference becomes even more pronounced, with ShardedMailbox being ~10.5 times faster than a standard channel, highlighting its excellent scalability.
goos: windows
goarch: amd64
pkg: github.com/GenshIv/nexus/sharded
cpu: AMD Ryzen 9 7950X3D 16-Core Processor
BenchmarkShardedMailbox_Throughput-32 354208934 16.57 ns/op 0 B/op 0 allocs/op
BenchmarkStandardChannel_MPMC-32 34359813 174.1 ns/op 0 B/op 0 allocs/op
The SPSC queue is ~10.6 times faster than a standard channel in a single-producer-single-consumer scenario.
goos: linux
goarch: amd64
pkg: github.com/GenshIv/nexus/spsc
cpu: AMD EPYC 7763 64-Core Processor
BenchmarkSPSCQueue_Separate-4 238401357 5.094 ns/op
BenchmarkStdChannel_Separate-4 23047090 54.08 ns/op
This is the "native" environment for such structures.
- Why: In trading systems, every microsecond of latency in transmitting market quotes or order execution signals is critical.
- Application: Transferring signals between a UDP market data parsing stream and a business logic (strategy) stream. The
spscqueue will work tens of times faster here than channels, removing latency "jitter."
- Application: You have thousands of players performing actions (clicks, purchases, leveling up). You need to update the "world state" in real-time.
- Why:
ShardedMailboxallows distributing the load across processing threads without deadlocks, ensuring player state is not corrupted by data access conflicts.
- Application: Implementing Write-Ahead Log (WAL) or LSM-tree structures.
- Why: When data needs to be quickly flushed from memory to disk, the writing thread must not block threads that are reading data. Nexus acts as an ideally fast buffer between application code and disk I/O.
- Application: Transferring packets between a network reading thread and a processing thread (e.g., for filtering HTTP headers or TLS termination).
- Why: API gateways (like Envoy or custom Go solutions) handle tens of thousands of requests in a second. At these speeds, standard channels become a bottleneck. This module allows building lock-free pipelines that can handle immense load on a single node.
- Application: Transferring frames or tensors between a capture stream (e.g., from a camera or video stream) and a model inference stream (TensorRT/ONNX).
- Why: If the model runs fast but the data transport (queue) is slow, the GPU sits idle. The
spscqueue is an ideal solution for feeding data to the model without CPU-side bottlenecks.
Use Nexus wherever:
- Latency is critical: When 50-100 ns of channel latency is "too much."
- Concurrency is high: When you have 32, 64, or 128 cores, and standard mutexes start fighting for cache lines.
- Predictability is needed: You want to avoid GC "freezes" caused by extra allocations and runtime object creation.
- Low-load applications: For a web service with 10 requests per second, standard channels are a perfect choice because they are simpler to maintain.
- Complex
selectlogic: Go channels are unique in their ability to wait for events from multiple sources. Nexus is a "fast pipe"; it has no selection logic, only direct data exchange.
package main
import (
"fmt"
"sync"
"github.com/GenshIv/nexus/sharded"
)
func main() {
mailbox := sharded.NewShardedMailbox[int]()
var wg sync.WaitGroup
numMessages := 100
wg.Add(numMessages * 2)
fmt.Printf("Launching %d producer-consumer pairs on %d shards...\n", numMessages, mailbox.ShardCount())
for i := 0; i < numMessages; i++ {
go func(consumerID int) {
defer wg.Done()
item, err := mailbox.Dequeue(uint64(consumerID))
if err != nil {
fmt.Printf("Consumer %d failed: %v\n", consumerID, err)
return
}
fmt.Printf("Consumer %d received: %d\n", consumerID, item)
}(i)
}
for i := 0; i < numMessages; i++ {
go func(producerID int) {
defer wg.Done()
item := 1000 + producerID
err := mailbox.Enqueue(uint64(producerID), item)
if err != nil {
fmt.Printf("Producer %d failed: %v\n", producerID, err)
return
}
fmt.Printf("Producer %d sent: %d\n", producerID, item)
}(i)
}
wg.Wait()
fmt.Println("\nAll messages have been successfully exchanged.")
mailbox.Close()
}go get github.com/GenshIv/nexus- Continuous Benchmarking: Setting up regular benchmark runs in GitHub Actions to ensure performance does not degrade with new Go versions or code changes.
- Zero-Copy Potential: For extreme performance scenarios, exploring the possibility of passing objects via
unsafepointers or usingsync.Poolto reuse mailbox objects, completely eliminating GC overhead over long runs.
Igor Ivanuto
This project is licensed under the MIT License.