Skip to content
/ viator Public

Viator is a high-performance, memory-safe Redis-compatible toolkit

License

MIT license
2 stars 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

panggi/viator

BranchesTags

Repository files navigation

Viator

A high-performance, memory-safe Redis-compatible toolkit

Crates.io Rust License Redis Compatible

Note: Viator is Redis® 8.4.0 protocol-compatible. Redis is a registered trademark of Redis Ltd. Viator is not affiliated with or endorsed by Redis Ltd.

Status

Published on crates.io - Verified with redis-cli 8.4.0. Suitable for testing and non-production workloads.

Category Status
Core Data Types ✅ String, List, Hash, Set, Sorted Set, Stream
Redis Stack ✅ JSON, Bloom, Cuckoo, CMS, TopK, TimeSeries
Persistence (VDB/AOF) ✅ CRC64 checksums, LZF compression, atomic writes, non-blocking
Security ✅ ACL, TLS, AUTH rate limiting, constant-time password comparison
Memory Management ✅ LRU eviction with Redis-style 5-key sampling
Cluster/Sentinel ✅ Full implementation
Lua Scripting ✅ Lua 5.4, real SHA1, EVALSHA
VectorSet ✅ HNSW-based similarity search
Operability ✅ Accurate INFO metrics, SHUTDOWN SAVE/NOSAVE/ABORT

Overview

Viator is a high-performance, memory-safe Redis-compatible toolkit featuring:

  • Full RESP3 Protocol - Wire-compatible with Redis clients
  • All Core Data Types - String, List, Hash, Set, Sorted Set, Stream, VectorSet
  • Vector Similarity Search - HNSW algorithm for semantic search and RAG applications
  • Redis Stack Modules - JSON, TimeSeries, Bloom, Cuckoo, TopK, T-Digest, CMS
  • High Availability - Cluster mode with automatic failover, Sentinel monitoring
  • Persistence - AOF and VDB with CRC64 checksums and LZF compression
  • Lua Scripting - Full Lua 5.4 scripting with real SHA1 hashing
  • Enterprise Security - ACL, TLS, audit logging, AUTH rate limiting

Installation

From crates.io

cargo install viator

This installs all binaries to ~/.cargo/bin/:

  • viator-server - Main server
  • viator-cli - Command-line client
  • viator-benchmark - Performance benchmarking
  • viator-check-vdb - VDB file integrity checker
  • viator-check-aof - AOF file checker
  • viator-sentinel - High availability monitor
  • viator-dump - Data export tool
  • viator-load - Data import tool
  • viator-shake - Live data synchronization

Build from Source

# Clone the repository
git clone https://github.com/panggi/viator.git
cd viator

# Build in release mode
cargo build --release

# Run the server
./target/release/viator-server

Quick Start

# Start the server
viator-server --port 6379 --bind 127.0.0.1

Using viator-cli

# Connect with viator-cli
viator-cli -p 6379

# Test basic commands
127.0.0.1:6379> SET hello "world"
OK
127.0.0.1:6379> GET hello
"world"

Configuration Options

viator-server --port 6380           # Custom port
viator-server --bind 0.0.0.0        # Listen on all interfaces
viator-server --help                # Show all options

Features

Data Types

Type Commands
Strings GET, SET, MGET, MSET, INCR, APPEND, GETRANGE, SETEX, SETNX
Lists LPUSH, RPUSH, LPOP, RPOP, LRANGE, LINDEX, LINSERT, LLEN
Hashes HSET, HGET, HMGET, HGETALL, HDEL, HINCRBY, HGETDEL*, HSETEX*
Sets SADD, SREM, SMEMBERS, SINTER, SUNION, SDIFF, SCARD
Sorted Sets ZADD, ZREM, ZRANGE, ZRANK, ZSCORE, ZINCRBY, ZRANGEBYSCORE
Streams XADD, XREAD, XRANGE, XLEN, XGROUP, XACK, XCLAIM
HyperLogLog PFADD, PFCOUNT, PFMERGE
Bitmaps SETBIT, GETBIT, BITCOUNT, BITOP, BITPOS
Geo GEOADD, GEODIST, GEOHASH, GEOPOS, GEORADIUS, GEOSEARCH
VectorSet VADD, VSIM, VGET, VDEL, VCARD, VINFO

*Redis 8.0+ commands

Redis Stack Modules

# JSON
JSON.SET user $ '{"name":"John","age":30}'
JSON.GET user $.name

# Bloom Filter
BF.ADD myfilter "item1"
BF.EXISTS myfilter "item1"

# Time Series
TS.CREATE temperature RETENTION 86400
TS.ADD temperature * 23.5

# Top-K
TOPK.RESERVE trending 10
TOPK.ADD trending "item1" "item2"

Vector Similarity Search

HNSW-based vector similarity search for semantic search, recommendations, and RAG:

# Add vectors (128-dimensional embeddings)
VADD embeddings FP32 VALUES 128 0.1 0.2 ... 0.9 "doc1"
VADD embeddings FP32 VALUES 128 0.2 0.3 ... 0.8 "doc2" SETATTR category "tech"

# Find similar vectors
VSIM embeddings FP32 VALUES 128 0.1 0.2 ... 0.9 TOPK 10

# With filtering
VSIM embeddings FP32 VALUES 128 0.1 0.2 ... 0.9 TOPK 10 FILTER "category == 'tech'"

# Get vector info
VINFO embeddings
VCARD embeddings

See VECTORSET.md for detailed documentation.

Cluster Mode (Redis 8.4 Compatible)

# Cluster information
CLUSTER INFO
CLUSTER NODES
CLUSTER SLOTS

# Redis 8.2+ Slot Statistics
CLUSTER SLOT-STATS SLOTSRANGE 0 100

# Redis 8.4 Atomic Migration
CLUSTER MIGRATION IMPORT 0-1000 NODES source-node-id
CLUSTER MIGRATION STATUS

Sentinel High Availability

# Monitor a master
SENTINEL MONITOR mymaster 127.0.0.1 6379 2

# Get master address
SENTINEL GET-MASTER-ADDR-BY-NAME mymaster

# Check quorum
SENTINEL CKQUORUM mymaster

Transactions & Scripting

# Transactions
MULTI
SET key1 "value1"
INCR counter
EXEC

# Lua Scripting
EVAL "return redis.call('GET', KEYS[1])" 1 mykey

Pub/Sub

# Subscribe to channels
SUBSCRIBE news sports

# Publish messages
PUBLISH news "Breaking news!"

Architecture

Viator is built on a modular architecture:

┌─────────────────────────────────────────────────────────────┐
│                        Client Layer                          │
│  ┌─────────┐  ┌─────────┐  ┌─────────┐  ┌─────────────────┐ │
│  │  RESP3  │  │   TLS   │  │  Rate   │  │  Connection     │ │
│  │ Parser  │  │ Support │  │ Limiter │  │  Manager        │ │
│  └─────────┘  └─────────┘  └─────────┘  └─────────────────┘ │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│                       Command Layer                          │
│  ┌─────────────────────────────────────────────────────────┐│
│  │              Command Registry (200+ commands)           ││
│  └─────────────────────────────────────────────────────────┘│
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────────────────┐ │
│  │  Strings │ │  Lists   │ │  Hashes  │ │  Sorted Sets   │ │
│  └──────────┘ └──────────┘ └──────────┘ └────────────────┘ │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────────────────┐ │
│  │  Streams │ │   Geo    │ │   JSON   │ │  TimeSeries    │ │
│  └──────────┘ └──────────┘ └──────────┘ └────────────────┘ │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│                       Storage Layer                          │
│  ┌─────────────────────────────────────────────────────────┐│
│  │          DashMap (Concurrent Hash Tables)               ││
│  └─────────────────────────────────────────────────────────┘│
│  ┌──────────────┐  ┌──────────────┐  ┌───────────────────┐ │
│  │    Expiry    │  │   Pub/Sub    │  │    Persistence    │ │
│  │   Manager    │  │     Hub      │  │   (AOF + VDB)     │ │
│  └──────────────┘  └──────────────┘  └───────────────────┘ │
└─────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────┐
│                    Distributed Layer                         │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────┐ │
│  │  Cluster Bus    │  │    Sentinel     │  │ Replication │ │
│  │  (Gossip)       │  │    Monitor      │  │   Manager   │ │
│  └─────────────────┘  └─────────────────┘  └─────────────┘ │
└─────────────────────────────────────────────────────────────┘

For detailed documentation:

Performance

Viator leverages Rust's zero-cost abstractions for high performance:

  • Memory Allocator: mimalloc for fast short-lived allocations
  • Concurrency: Lock-free data structures (DashMap, crossbeam)
  • I/O: Tokio async runtime
  • Zero-Copy: Bytes crate for efficient buffer management
  • SIMD: memchr for fast byte searching
  • Perfect Hashing: O(1) command dispatch with PHF
  • Sharded Storage: 64-shard architecture for reduced lock contention
  • Inline Responses: Pre-allocated RESP responses for common replies
  • Response Coalescing: Batched writes to reduce syscall overhead

Security

Memory Safety

  • Written primarily in safe Rust with minimal, documented unsafe blocks
  • Eliminates buffer overflows, use-after-free, data races
  • Bounds checking on all array/slice access
  • NaN-safe float comparisons (no panic on edge cases)
  • All unsafe blocks have SAFETY documentation comments

Authentication & Authorization

  • ACL system with user permissions and key patterns
  • Argon2id password hashing (memory-hard)
  • Constant-time password comparison (prevents timing attacks)
  • AUTH rate limiting: 10 failures → 60-second lockout
  • Command-level access control with categories

Memory Management

  • Maxmemory enforcement with configurable limits
  • LRU eviction with Redis-style 5-key sampling algorithm
  • Volatile-TTL, volatile-random, allkeys-random policies supported
  • Real-time memory metrics via INFO command

Network Security

  • TLS 1.3 via rustls (memory-safe)
  • Rate limiting per IP
  • Connection limits and buffer size controls

Lua Sandboxing

  • Restricted function set (no os, io, debug)
  • Memory and instruction limits
  • Execution timeouts
  • Real SHA1 script identification (Redis-compatible)

See SECURITY.md for detailed security documentation.

Building

Requirements

  • Rust 1.85 or later
  • Cargo

Feature Flags

# Default features
cargo build --release

# With TLS support
cargo build --release --features tls

# With OpenTelemetry
cargo build --release --features telemetry

# Minimal build
cargo build --release --no-default-features

Running Tests

# Run all tests
cargo test

# Run benchmarks
cargo bench

# Test with viator-cli (start server first)
./target/release/viator-server &
viator-cli PING
viator-cli SET foo bar
viator-cli GET foo

CLI Tools

Viator includes a complete suite of command-line utilities:

viator-server

The main server binary.

viator-server --port 6379 --bind 127.0.0.1
viator-server --help

viator-cli

Interactive command-line client (redis-cli compatible).

# Connect to server
viator-cli -h 127.0.0.1 -p 6379

# Execute single command
viator-cli -h 127.0.0.1 -p 6379 SET foo bar

# With authentication
viator-cli -h 127.0.0.1 -p 6379 -a mypassword

viator-benchmark

Performance benchmarking tool (redis-benchmark compatible).

# Run default benchmarks
viator-benchmark -h 127.0.0.1 -p 6379

# Custom benchmark
viator-benchmark -h 127.0.0.1 -p 6379 -c 50 -n 100000 -t set,get

# With pipelining
viator-benchmark -h 127.0.0.1 -p 6379 -P 16

📊 Latest benchmark results: GitHub Actions Benchmark

viator-check-vdb

VDB (snapshot) file integrity checker.

# Check VDB file integrity
viator-check-vdb dump.vdb

# Verbose output
viator-check-vdb -v dump.vdb

viator-check-aof

AOF (append-only file) integrity checker and repair tool.

# Check AOF file
viator-check-aof appendonly.aof

# Attempt to fix corrupted AOF
viator-check-aof --fix appendonly.aof

viator-sentinel

High availability monitoring daemon.

# Monitor a master
viator-sentinel --master mymaster --host 127.0.0.1 --port 6379 --quorum 2

# With down-after threshold
viator-sentinel --master mymaster --host 127.0.0.1 --port 6379 --quorum 2 --down-after 5000

viator-dump

Data export tool for backup and migration.

# Export to RDB format
viator-dump -h 127.0.0.1 -p 6379 -o backup.rdb

# Export to JSON format
viator-dump -h 127.0.0.1 -p 6379 --format json -o backup.json

viator-load

Data import tool for restoration and migration.

# Import from RDB file
viator-load -h 127.0.0.1 -p 6379 -i backup.rdb

# Import from JSON file
viator-load -h 127.0.0.1 -p 6379 --format json -i backup.json

viator-shake

Live data synchronization between instances.

# Sync from source to target
viator-shake --source 127.0.0.1:6379 --target 127.0.0.1:6380

# With filtering
viator-shake --source 127.0.0.1:6379 --target 127.0.0.1:6380 --filter "user:*"

Compatibility

Viator targets Redis 8.4.0 protocol compatibility:

Feature Status Details
RESP3 Protocol ✅ Full All frame types, streaming
Core Commands ✅ 200+ String, List, Hash, Set, ZSet, Stream
VectorSet ✅ Full HNSW similarity search, persistence
Cluster Mode ✅ Full Gossip, failover, Redis 8.4 migration
Sentinel ✅ Full Monitoring, automatic failover
Lua Scripting ✅ Full Lua 5.4, real SHA1, sandboxed
Pub/Sub ✅ Full Channels, patterns, sharded
Transactions ✅ Full MULTI/EXEC, WATCH/UNWATCH
VDB Persistence ✅ Full CRC64 checksums, LZF decompression
AOF Persistence ✅ Full Atomic rewrite, fsync policies
Redis Stack ✅ Full JSON, TimeSeries, Bloom, Cuckoo, TopK, T-Digest, CMS
Security ✅ Full ACL, TLS 1.3, AUTH rate limiting

Project Structure

├── src/
│   ├── bin/            # CLI tool binaries
│   │   ├── viator-cli.rs        # Command-line client
│   │   ├── viator-benchmark.rs  # Performance benchmarking
│   │   ├── viator-check-vdb.rs  # VDB file checker
│   │   ├── viator-check-aof.rs  # AOF file checker
│   │   ├── viator-sentinel.rs   # HA monitoring daemon
│   │   ├── viator-dump.rs       # Data export tool
│   │   ├── viator-load.rs       # Data import tool
│   │   └── viator-shake.rs      # Live data sync
│   ├── commands/       # Command implementations (200+ commands)
│   ├── protocol/       # RESP3 parser and frame handling
│   ├── server/         # TCP server, connections, cluster, sentinel
│   ├── storage/        # Database, expiry management
│   ├── persistence/    # AOF and VDB persistence (CRC64, LZF)
│   ├── security/       # ACL, TLS, audit, rate limiting
│   ├── types/          # Data type implementations (VectorSet, etc.)
│   ├── pool.rs         # Buffer pooling
│   ├── error.rs        # Error types
│   ├── main.rs         # Server entry point (viator-server binary)
│   └── lib.rs          # Library root
├── docs/
│   ├── API.md          # RESP3 protocol and client guide
│   ├── ARCHITECTURE.md # System design overview
│   ├── BUILDING.md     # Build and development guide
│   ├── CLUSTER.md      # Cluster and Sentinel guide
│   ├── COMPATIBILITY.md # Redis command compatibility
│   ├── CONFIG.md       # Configuration reference
│   ├── DOCKER.md       # Docker deployment guide
│   ├── PERSISTENCE.md  # VDB/AOF documentation
│   ├── SECURITY.md     # Security features
│   └── VECTORSET.md    # Vector similarity search
├── fuzz/               # Fuzz testing targets
├── benches/            # Performance benchmarks
└── tests/              # Integration tests

License

MIT License - see LICENSE for details.

Acknowledgments

  • Redis Ltd. for creating Redis and the RESP protocol specification. Redis is a registered trademark of Redis Ltd.
  • The Rust community for excellent async and systems libraries
  • tokio, dashmap, bytes, and other crate authors

About

Viator is a high-performance, memory-safe Redis-compatible toolkit

Topics

rust redis database cache memory-safety

Resources

Readme

License

MIT license

Security policy

Security policy
Activity

Stars

2 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •  

Languages