RWDB (Read-Write Database) In-Memory Node Store

[shortthefomo]

xls: XXXX
title: RWDB (Read-Write Database) In-Memory Node Store
description:
  An in-memory node store backend with null-backend mode that eliminates disk
  I/O for the node store while retaining ledger state through shared_ptr chains
  and a sliding window of recent ledgers.
implementation: https://github.com/XRPLF/rippled/pull/6549
author: shortthefomo (@shortthefomo)
category: System
status: Draft
proposal-from: https://github.com/XRPLF/rippled/pull/6549
created: 2026-06-07
updated: 2026-06-07

RWDB (Read-Write Database) In-Memory Node Store

1. Abstract

RWDB is an in-memory node store backend for xrpld that eliminates all persistent node-store disk I/O by operating in "null-backend" mode. In this mode, fetch() operations against the node store always return notFound and store() operations are no-ops. Ledger state is retained entirely through shared_ptr chains linking Ledgers to SHAMaps, with a configurable sliding window of recent ledgers kept resident in memory.

The feature provides a lighter, faster node configuration suitable for validators, submission nodes, pathfinding nodes, and any workload that requires fast access to recent tran sactions without historical data retention on disk. A synced RWDB node typically consumes 11-13GB of memory.

Index

1. Abstract
2. Motivation
3. Introduction
   • 3.1. Terminology
   • 3.2. Scope
   • 3.3. Non-Goals
4. Specification
   • 4.1. Overview
   • 4.2. Configuration
   • 4.3. RWDB Node Store Backend
   • 4.4. RWDB Relational Database
   • 4.5. In-Memory Peer Finder Store
   • 4.6. SHAMap Store Integration
   • 4.7. Reader-Preference Shared Mutex
   • 4.8. Ledger Fully-Wired Tracking
   • 4.9. Ledger Retention and Delta Walk Wiring
   • 4.10. SHAMapSync FullBelowCache
   • 4.11. Peer Fallback
   • 4.12. Cache Eviction Prevention
5. Rationale
   • 5.1. Why Null-Backend Mode?
   • 5.2. Why Delta Walk Instead of Full Tree Walk?
   • 5.3. Why Disable Shared FullBelowCache?
   • 5.4. Why Reader-Preference Mutex?
   • 5.5. Alternate Designs Rejected
6. Backwards Compatibility
7. Test Plan
   • 7.1. Backend Correctness
   • 7.2. Relational Database Correctness
   • 7.3. SHAMap Store Integration
   • 7.4. Ledger Wiring and Retention
   • 7.5. Concurrency and Infrastructure
8. Operational Considerations
   • 8.1. Deployment Recommendations
   • 8.2. Configuration Guidance
   • 8.3. Monitoring and Alerting
   • 8.4. Recovery Procedures
9. Performance Analysis
   • 9.1. Memory Footprint
   • 9.2. Sync Timing
   • 9.3. Throughput and Latency
10. Security Considerations
    • 10.1. Data Loss on Restart
    • 10.2. Memory Exhaustion
    • 10.3. No Attack Surface Expansion
    • 10.4. Cache Integrity
    • 10.5. Environment Variable Isolation
11. Appendix A: FAQ

2. Motivation

Traditional xrpld nodes require fast, expensive SSD storage for the node database (RocksDB, SQLite, or Nudb). The node store is responsible for persisting SHAMap nodes (ledger state and transaction data), which generates significant disk I/O during validation and ledger building.

Current limitations of traditional node store backends:

• High hardware costs: Fast SSD storage is required for acceptable performance, increasing operational costs.
• Disk I/O bottleneck: Frequent reads and writes to the node store can become a performance bottleneck under high load.
• Unnecessary persistence for many workloads: Many node types (validators, submission nodes) do not benefit from historical node data retention.
• Complex rotation management: Traditional backends require careful management of database rotation and cleanup to prevent disk exhaustion.

RWDB addresses these gaps by providing:

• Zero disk I/O for node store: All node data is held in memory, eliminating the storage bottleneck entirely.
• Reduced hardware requirements: No SSD required for node storage; standard RAM suffices.
• Simplified operations: No database rotation, no cleanup cycles, no disk space monitoring for node data.
• Optimized for recent data: Sliding window retention keeps only the most useful data (recent ledgers) in memory.

3. Introduction

RWDB introduces a fundamentally different approach to node storage: instead of persisting SHAMap nodes to disk, it relies entirely on in-memory shared_ptr reference chains. When a ledger is validated or received, its SHAMap structure is pinned in memory via the ledger's internal pointers. A sliding window of recent ledgers maintains these pointers, preventing garbage collection of active state.

This approach trades historical data persistence for operational simplicity and performance. On restart, an RWDB node must resync from peers, but this is acceptable for workloads that prioritize current-state validation over historical queries.

3.1. Terminology

Core Concepts:

• Null-Backend Mode: An operating mode where the node store backend short-circuits all fetch() and store() operations. fetch() always returns NotFound and store() is a no-op. This is the default and only supported mode for RWDB.
• Fully-Wired Ledger: A ledger whose SHAMap has been completely walked (all state and transaction nodes pinned in memory). Tracked via an atomic fullyWired_ flag on the Ledger object.
• Delta Walk: A process of walking only the differences between two ledgers (using visitDifferences), rather than walking the entire SHAMap tree. Used to efficiently wire new ledgers when a fully-wired base ledger exists.
• Sliding Window: The set of N most recent ledgers retained in memory, configurable via ledger_history. Older ledgers are released, allowing their SHAMap nodes to be garbage collected.
• TreeNodeCache: An in-memory cache of SHAMap nodes. In null-backend mode, this cache IS the effective node store, as entries cannot be recovered from disk.

Components:

• RWDBBackend: The in-memory implementation of NodeStore::Backend, responsible for node object storage (or short-circuiting in null mode).
• RWDBDatabase: The in-memory implementation of RelationalDatabase, responsible for ledger header and transaction data storage.
• InMemoryStore: Replacement for the SQLite-based peer finder store, using simple std::vector storage.
• ReaderPreferringSharedMutex: Cross-platform mutex providing reader-preferring semantics to prevent reader starvation under write contention.

3.2. Scope

This XLS specifies the following components and behaviors:

• RWDB Node Store Backend: In-memory implementation of NodeStore::Backend with null-backend mode short-circuiting.
• RWDB Relational Database: In-memory implementation of RelationalDatabase for ledger headers, transactions, and account-tx indexes.
• In-Memory Peer Finder Store: Replacement for SQLite-based peer finder persistence.
• SHAMap Store Integration: Detection of RWDB type, environment variable propagation, rotation skip logic, and non-rotating database creation.
• Ledger Fully-Wired Tracking: Atomic flag tracking on Ledger objects for delta walk optimization.
• Delta Walk Wiring: Efficient wiring of inbound ledgers against closest fully-wired base ledger.
• Reader-Preferring Shared Mutex: Cross-platform mutex utility for reader-preferring concurrency semantics.
• FullBelowCache Disable: Disabling shared FullBelowCache in null-backend mode.
• Peer Fallback: TreeNodeCache and in-memory ledger fallback when node store returns empty results.
• Cache Eviction Prevention: Early return in clearCaches() to prevent irrecoverable data loss.

3.3. Non-Goals

• Provide a persistent in-memory database: RWDB is explicitly non-persistent. Data is lost on restart. For persistent storage, use RocksDB or SQLite.
• Replace archive nodes: RWDB cannot serve historical data beyond the ledger_history window. Archive nodes require disk-backed storage.
• Improve historical query performance: RWDB is optimized for recent data access, not historical queries.
• Add new RPC methods or protocol features: RWDB is a backend-only change with no user-facing API modifications.
• Support clustering or replication: RWDB does not include mechanisms for replicating in-memory state across nodes.

4. Specification

4.1. Overview

RWDB introduces two coordinated in-memory backends:

1. RWDB Node Store Backend (RWDBFactory / RWDBBackend): An in-memory implementation of the NodeStore::Backend interface using std::map<uint256, shared_ptr<NodeObject>> for storage. In null-backend mode (the default and only supported mode), fetch() always returns Status::NotFound and store() is a no-op.

2. RWDB Relational Database Backend (RWDBDatabase): An in-memory implementation of the RelationalDatabase interface using std::map containers for ledger headers, transactions, and account-transaction indexes.

4.2. Configuration

RWDB is activated by setting type=rwdb in the [node_db] configuration section. This automatically enables null-backend mode — no additional configuration keys or environment variables are required.

[node_db]
type=rwdb
advisory_delete=0

[relational_db]
backend=rwdb

Required constraints:

• ledger_history must be greater than 0.
• online_delete is automatically defaulted to ledger_history if not explicitly set, clamped to the minimum deletion interval.

Environment Variable:
Setting type=rwdb automatically sets the XRPL_RWDB_NULL environment variable to "1", which is used by libxrpl helpers (which cannot access Config) to detect null mode. The parsing logic is:

• Unset or empty string: null mode disabled
• "0": null mode explicitly disabled
• Any other non-empty value (e.g., "1", "true"): null mode enabled

4.3. RWDB Node Store Backend

The RWDBBackend class implements the NodeStore::Backend interface:

Storage:

• Uses std::map<uint256 const, std::shared_ptr<NodeObject>> for in-memory storage.
• Protected by a reader_preferring_shared_mutex (see Section 4.7).

Null Mode Behavior:

• fetch(): Returns Status::NotFound immediately when XRPL_RWDB_NULL is set.
• store(): Returns immediately without storing when XRPL_RWDB_NULL is set.
• fetchBatch(): Iterates hashes and calls fetch() for each, respecting null mode.
• storeBatch(): Calls store() for each object in the batch, respecting null mode.

Lifecycle:

• open(): Sets internal isOpen_ flag; throws if already open.
• close(): Swaps the internal map to a local variable (O(1)) then destroys outside the lock, preventing fetch() calls from being blocked by map destruction.

4.4. RWDB Relational Database

The RWDBDatabase class implements the RelationalDatabase interface with in-memory storage:

Data Structures:

• ledgers_: std::map<LedgerIndex, LedgerData> — stores ledger headers and transaction data per sequence.
• ledgerHashToSeq_: std::map<uint256, LedgerIndex> — maps ledger hash to sequence number.
• transactionMap_: std::map<uint256, AccountTx> — global transaction lookup by hash.
• accountTxMap_: std::map<AccountID, AccountTxData> — per-account transaction index, keyed by ledger sequence.

Operations:

• getMinLedgerSeq() / getMaxLedgerSeq(): Return first/last ledger sequences from ledgers_.
• getTransactionsMinLedgerSeq(): Returns minimum sequence with non-empty transactions.
• getAccountTransactionsMinLedgerSeq(): Returns minimum sequence across all account transaction maps.
• deleteBeforeLedgerSeq(): Removes ledgers and associated transaction data below a threshold.
• deleteTransactionByLedgerSeq(): Purges transactions for a specific ledger sequence.
• saveValidatedLedger(): Stores ledger header and transactions in memory.
• getLedgerInfoByIndex(): Retrieves ledger header by sequence number.

4.5. In-Memory Peer Finder Store

The InMemoryStore class replaces the SQLite-based peer finder store for RWDB mode:

Storage:

• entries_: std::vector<Entry> — holds peer endpoint and valence data.

Operations:

• load(): Iterates stored entries and invokes the callback for each.
• save(): Replaces the entire entry list with the provided vector.

4.6. SHAMap Store Integration

When type=rwdb is detected in SHAMapStoreImp:

1. Non-rotating Database: A plain (non-rotating) NodeStore::Database is created via makeNodeStore(). No DatabaseRotatingImp is instantiated, and dbRotating_ remains nullptr.

2. Rotation Thread Behavior: The rotation cycle skips clearCaches(), makeBackendRotating(), and rotate() entirely. Only SQLite cleanup (clearPrior) runs during the rotation cycle. This is critical because the TreeNodeCache IS the node store in null mode, and evicting it would cause irrecoverable SHAMapMissingNode errors.

3. online_delete Default: When not explicitly configured, online_delete defaults to max(ledger_history, minimum_deletion_interval).

4.7. Reader-Preference Shared Mutex

A new reader_preferring_shared_mutex utility provides cross-platform reader-preferring semantics:

• On Linux: Wraps pthread_rwlock_t initialized with PTHREAD_RWLOCK_PREFER_READER_NP, ensuring readers are not starved by frequent writer contention (which occurs with the default PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP).
• On macOS/Windows: Type alias for std::shared_mutex, which is already reader-preferring on those platforms.

The interface is identical to std::shared_mutex, supporting std::shared_lock and std::unique_lock.

A possible future extension not explored here is to prioritize writes for a validator configuration.

4.8. Ledger Fully-Wired Tracking

Ledgers now track a fullyWired_ atomic boolean flag:

• isFullyWired(): Returns whether all SHAMap nodes have been pinned in memory.
• setFullyWired(): Marks the ledger as fully wired (idempotent).
• fullWireForUse(): Forces a complete walk of state and tx maps to pin all nodes, then sets the wired flag. Returns false if a SHAMapMissingNode is encountered.

The genesis ledger is marked as fully wired by default. In null-backend mode, this flag is used to determine if a delta walk can be used for wiring (faster) versus a full tree walk (expensive on mainnet with 70M+ leaves).

4.9. Ledger Retention and Delta Walk Wiring

LedgerMaster Sliding Window:

• mRetainedLedgers maintains a sliding window of N recent ledgers (configurable via ledger_history).
• getClosestFullyWiredLedger() efficiently finds the closest fully-wired ledger on the same network as a target, used as a base for delta walks.

InboundLedger Delta Walk:
When an inbound ledger needs to be wired in null-backend mode:

1. If a fully-wired base ledger exists on the same network, perform a delta walk (visitDifferences) against the base, which only visits changed state nodes.
2. If no base exists (e.g., during initial sync), trust that SHAMapSync already pinned all children via canonicalizeChild, and only wire the small tx map.
3. Without null-backend mode, fullWireForUse() always returns true immediately (existing behavior unchanged).

InboundLedgers Cache:

• recentHistoryLedgers_ caches recently fetched inbound ledgers for retention.
• onLedgerFetched() passes the inbound ledger for retention in the sliding window.

4.10. SHAMapSync FullBelowCache

In null-backend mode, the shared FullBelowCache is disabled to prevent cross-SHAMap subtree-skipping (which could cause incorrect results when entries are irrecoverable). Local isFullBelow() checks are preserved for per-map efficiency.

The useFullBelowCache() helper checks XRPL_RWDB_NULL and returns false when null mode is active.

4.11. Peer Fallback

When the node store returns empty results, PeerImp falls back to:

1. TreeNodeCache lookup
2. In-memory ledger lookup

This ensures peers can still serve node requests even without persistent storage.

4.12. Cache Eviction Prevention

clearCaches() returns early in null-backend mode. The TreeNodeCache and FullBelowCache are not evicted because entries are irrecoverable without a real backend.

5. Rationale

5.1. Why Null-Backend Mode?

The original Xahau implementation designed RWDB as a memory-only backend from the outset. The null-backend mode provides the most significant performance improvement for the target use cases by eliminating all node-store disk I/O. A persistent in-memory database would add operational complexity and external dependencies while providing minimal benefit over existing disk-backed options like RocksDB. Therefore, RWDB type always implies null backend.

5.2. Why Delta Walk Instead of Full Tree Walk?

On mainnet, a full SHAMap tree walk touches 70M+ leaves, which takes longer than a consensus round. Delta walks against a recent fully-wired base touch only the changed nodes (typically thousands), making wiring fast enough to keep up with consensus.

5.3. Why Disable Shared FullBelowCache?

The shared FullBelowCache allows different SHAMap syncs to share "full below" information. In null-backend mode, where entries cannot be recovered from disk, incorrect cache state could cause permanent data loss. Disabling the shared cache while keeping per-map checks provides safety without significant performance impact.

5.4. Why Reader-Preference Mutex?

Linux's default pthread_rwlock_t prefers writers, causing reader starvation under write contention. The RWDB backend has frequent reads (fetch operations) and periodic writes (store operations), making reader preference the correct choice for throughput.

again we can alter this to be adjusted for validators... atm not implemented.

5.5. Alternate Designs Rejected

• Persistent in-memory database (e.g., Redis-backed): Rejected due to added operational complexity, external dependencies, and network latency.
• Write-through cache with disk backend: Rejected because it still requires SSD storage and does not eliminate the disk I/O bottleneck that RWDB aims to solve.
• Shared memory between processes: Rejected due to complexity in cross-process synchronization and lack of benefit for single-process xrpld deployments.
• Hybrid mode (hot data in memory, cold on disk): Rejected because the hot/cold boundary is difficult to define for SHAMap nodes, and the implementation complexity approaches that of RocksDB with less predictable performance.

6. Backwards Compatibility

This feature introduces no backwards incompatibilities:

• RWDB is opt-in via configuration (type=rwdb).
• Existing node types (RocksDB, SQLite, Nudb) are unaffected.
• No protocol changes, transaction format changes, or RPC API changes.
• The XRPL_RWDB_NULL environment variable is internal and does not affect network behavior.
• Peer protocol remains unchanged; RWDB nodes participate normally in the overlay network.

7. Test Plan

This feature is accompanied by comprehensive unit tests across the rippled codebase. The test strategy validates both correctness of the in-memory backends and proper integration with the SHAMap store, ledger management, and peer protocols.

7.1. Backend Correctness

• RWDBBackend_test: Validates XRPL_RWDB_NULL environment variable parsing (unset, "1", "true", "0", empty string). Verifies null-mode short-circuiting of fetch/store operations. Tests cleanup regression to ensure environment variable state is not polluted between test runs.
• Backend_test: Validates RWDB backend creation, batch store, and batch fetch operations. Verifies that RWDB data is ephemeral (not persisted across database reopen).

7.2. Relational Database Correctness

• RWDBDatabase_test: Compile-time verification that RWDBDatabase implements the RelationalDatabase interface correctly.
• Database_test: Validates RWDB database configuration and SQLite pragma settings. Verifies that safety level warnings are not incorrectly triggered for RWDB.
• AccountTx_test: Validates account_tx RPC with RWDB relational backend. Verifies transaction pagination and filtering work correctly with in-memory storage.
• Transaction_test: Validates tx RPC with RWDB relational backend. Verifies range requests and CTID (Canonical Transaction ID) resolution.

7.3. SHAMap Store Integration

• SHAMapStore_test: Validates RWDB rotation behavior with online_delete configured. Verifies that rotation skips cache clearing and backend rotation in null mode. Tests ledger retrieval via RPC after rotation cycle completes.
• LedgerReplay_test: Validates ledger replay functionality adapted for RWDB mode, ensuring historical ledger processing works correctly without persistent node store.

7.4. Ledger Wiring and Retention

• LedgerFullyWired_test: Validates default wired state of genesis ledger. Tests setFullyWired() idempotency (calling multiple times produces same result). Tests fullWireForUse() success path and thread-safe concurrent access to the wired flag.
• Config_test: Validates RWDB configuration parsing and validation. Verifies ledger_history > 0 requirement is enforced.

7.5. Concurrency and Infrastructure

• InMemoryStore_test: Validates basic save and load of peer finder entries. Tests empty store load (no callbacks invoked). Tests overwrite behavior on save and multiple sequential loads return consistent data.
• ReaderPreferringSharedMutex_test: Validates concurrent shared lock acquisition (multiple readers). Tests exclusive lock mutual exclusion (single writer). Tests try_lock_shared() and try_lock() behavior. Validates reader preference under contention (readers are not starved by frequent writer contention).
• SHAMapSyncFullBelowCache_test: Validates FullBelowCache disable behavior in null mode, ensuring shared cache is not used while per-map checks remain functional.

8. Operational Considerations

8.1. Deployment Recommendations

• Pair with Clio: RWDB is an excellent backend for xrpld instances paired with Clio. Clio handles historical data queries and persistence, while the RWDB-backed xrpld focuses on validation and recent ledger processing.
• Use systemd or container restart policies: Configure automatic restart on failure to minimize downtime. Ensure your orchestrator has appropriate backoff and health check configurations.
• Deploy behind a load balancer for high availability: Multiple independent RWDB nodes can be deployed behind a load balancer. Each maintains its own in-memory state; there is no built-in replication.
• Ensure stable runtime environments: RWDB nodes should run on infrastructure with reliable power, memory, and network connectivity to minimize unexpected restarts and resync cycles.

8.2. Configuration Guidance

• ledger_history tuning: Set ledger_history based on available memory and query requirements. Higher values provide a larger retention window but increase memory usage proportionally. A value of 512 (default) provides approximately 2-3 hours of history on mainnet.
• online_delete behavior: When not explicitly configured, online_delete defaults to max(ledger_history, minimum_deletion_interval). This ensures old ledger data is cleaned up from the relational database in sync with the sliding window.
• advisory_delete: Recommended to set to 0 for RWDB, as there is no persistent node store data to advisory-delete.
• Memory allocation: Deploy with at least 16GB RAM for mainnet (11-13GB steady-state usage plus headroom). Consider 24-32GB for validators or nodes under high load (again x2/x3 this value for safety).

8.3. Monitoring and Alerting

• RSS Memory Usage: Monitor Resident Set Size and configure alerts at 75% and 90% of available memory. Proactive alerts allow operators to adjust ledger_history before OOM kills occur.
• Ledger Height Lag: Track the difference between the node's current ledger and the network's validated ledger. Large or growing gaps may indicate sync issues, peer problems, or resource constraints.
• Process Restarts: Alert on unexpected xrpld process terminations. Frequent restarts indicate underlying infrastructure issues (OOM, crashes, network partitions).
• Peer Count: Monitor active peer connections. RWDB nodes rely on peers for resync after restart; insufficient peers will slow recovery.
• Sync Progress: During initial sync or post-restart resync, monitor the ledger catch-up rate. Typical rates vary based on peer quality and hardware.

8.4. Recovery Procedures

• After unexpected restart: The node will automatically begin resync from peers on startup. No manual intervention is required. Ensure adequate peer availability during recovery.
• After OOM kill: Check system logs for OOM events. Consider increasing available memory, reducing ledger_history, or adjusting cgroup limits before restarting.
• Switching from RWDB to RocksDB: Change type=rwdb to type=rocksdb (or desired backend) in [node_db], then restart. A full resync will be required as no persistent node store data exists to migrate.
• Switching from RocksDB to RWDB: Change type=rocksdb to type=rwdb in [node_db], then restart. The existing RocksDB data files will be ignored; the node will resync from peers.

9. Performance Analysis

9.1. Memory Footprint

RWDB memory consumption.

Size	Load	Notes
Total (Steady-State)	Synced mainnet node at rest	11-13 GB
Total (Peak Sync)	During initial sync or catch-up (transient allocations)	14-17 GB

Memory usage scales with:

• Network ledger size (mainnet > testnet > devnet).
• ledger_history value (higher = more retained ledgers = more memory).
• Transaction volume during peak periods (more SHAMap nodes pinned temporarily).

9.2. Sync Timing

RWDB nodes must resync from peers after every restart, as no persistent node store data survives process termination:

Scenario	Estimated Time	Notes
Initial sync (mainnet, cold start)	8-15 min	Depends on peer quality, network congestion, and hardware.

Factors affecting sync speed:

• Peer availability and quality: More peers with good connectivity = faster sync.
• Network congestion: High transaction volumes slow down node propagation.
• Hardware: CPU cores and memory bandwidth affect SHAMap processing speed.
• Delta walk vs full tree walk: Delta walks (milliseconds to low seconds per ledger) are orders of magnitude faster than full tree walks (minutes per ledger on mainnet with 70M+ leaves).

9.3. Throughput and Latency

Compared to disk-backed node stores (RocksDB, SQLite):

Metric	RWDB (Null-Backend)	RocksDB	Notes
Node store fetch latency	< 1 µs (in-memory map lookup, or immediate NotFound in null mode)	10-100 µs (disk cache hit) / ms (disk read)	RWDB eliminates disk I/O entirely.
Node store write latency	No-op in null mode	100 µs - ms (disk write + fsync)	RWDB store() is a no-op.
Recent transaction lookup	< 10 µs (RWDBDatabase in-memory map)	10-100 µs (RocksDB point query)	Comparable for recent data; RWDB wins on consistency.
Historical transaction lookup	N/A (data not retained beyond ledger_history)	10-100 µs	RocksDB advantage for archive workloads.
Disk I/O (node store)	Zero	High during validation and ledger close	Primary RWDB benefit.
CPU overhead	Lower (no RocksDB compression/decompression)	Higher (compression, WAL management)	RWDB reduces CPU contention.

These are representative estimates based on the architectural differences between in-memory and disk-backed storage. Actual measurements will vary by hardware, workload, and configuration.

10. Security Considerations

10.1. Data Loss on Restart

RWDB nodes store all node data in memory. On restart, the node must resync from peers. This is a design trade-off, not a vulnerability, but operators should understand the implications:

Impact:

• Complete loss of node store data (SHAMap nodes) on process termination.
• Historical ledger data beyond the ledger_history window is unavailable.
• Resync from peers is required after every restart, which takes time proportional to network size.

Mitigations:

• Deploy in environments with reliable power and stable runtime (containers with restart policies, systemd services).
• Ensure adequate memory allocation (11-13GB for a synced mainnet node minimum today) to prevent OOM kills. Operator is expected to validate their node has enough head room. For example x2,x3... whatever their requirements are, a validator would choose a higher multiple.
• Pair with Clio, Full History or other archive nodes for historical data queries.
• Monitor node health and configure alerts for unexpected restarts.

10.2. Memory Exhaustion

The in-memory storage grows with the number of retained ledgers and SHAMap nodes pinned in memory.

Attack Scenarios:

1. Memory exhaustion via ledger bloat: An adversary could attempt to fill the ledger with data, causing the RWDB node to consume excessive memory during the retention window.
2. OOM kill cascades: If the host system runs out of memory, the OS OOM killer may terminate the xrpld process, causing data loss and requiring a full resync.

Mitigations:

• Set ledger_history appropriately for available memory; lower values reduce memory footprint.
• Configure cgroup memory limits or container memory caps to bound maximum usage.
• Monitor RSS (Resident Set Size) and configure proactive alerts before OOM thresholds.
• Consider that peak memory during initial sync may exceed steady-state usage by 20-30%.
• Nodes looking to use this with a Validator configuration should target a x3 minimum RSS. To protect against possible state ballooning attack vectors.

10.3. No Attack Surface Expansion

RWDB does not introduce new network protocols, RPC methods, or transaction types. The feature is purely an internal storage backend change, visible only through configuration. Peer interactions remain identical to other node types.

10.4. Cache Integrity

Disabling the shared FullBelowCache in null mode prevents potential cache poisoning across SHAMap instances. The early return in clearCaches() prevents accidental eviction of irrecoverable data.

Why this matters:

• In traditional backends, evicted cache entries can be re-fetched from disk.
• In null-backend mode, evicted entries are permanently lost, causing SHAMapMissingNode errors that cannot be recovered.
• The rotation cycle explicitly skips clearCaches() to prevent this scenario.

10.5. Environment Variable Isolation

The XRPL_RWDB_NULL environment variable is set automatically by SHAMapStoreImp when type=rwdb is configured. Tests properly save and restore the environment variable state to prevent test pollution (verified by RWDBBackend_test::testNullModeCleanupRegression).

11. Appendix A: FAQ

A.1: Can I use RWDB for an archive node?

No. RWDB is designed for nodes that do not require historical data persistence beyond the ledger_history window. Archive nodes should use RocksDB or SQLite with appropriate retention settings.

A.2: What happens if the node runs out of memory?

The node will be terminated by the OS OOM killer (or container orchestrator). On restart, it will resync from peers. Operators should monitor RSS memory usage and configure ledger_history to stay within available memory bounds. Consider setting cgroup memory limits with appropriate swap behavior.

A.3: Can I switch between RWDB and RocksDB/NuDB?

Yes, by changing the type value in the [node_db] section and restarting xrpld. Note that switching to RWDB will require a full resync from peers, as RWDB does not read RocksDB/NuDB data files. Switching from RWDB to RocksDB will also require a resync, as no persistent data exists to migrate.

A.4: Does RWDB affect consensus or validation?

No. RWDB nodes participate in consensus and validation identically to other node types. The protocol behavior, ledger signing, and validation logic are unchanged. The difference is only in how node data is stored (memory vs. disk).

A.5: Can RWDB nodes serve historical ledger queries?

Only for ledgers within the ledger_history retention window. Queries for older ledgers will return errors (e.g., lgrNotFound), similar to a non-archive node configured with limited history.

A.6: Is RWDB suitable for public RPC servers?

RWDB is suitable for RPC servers that primarily serve recent data (e.g., transaction submission, account balances, recent ledgers, pathfinding). It is not suitable for servers that need to serve historical data queries (e.g., account_tx for old ledgers, historical ledger state lookups).

A.7: How long does resync take after a restart?

Resync time depends on network conditions and peer availability. NuDB/RocksDB nodes will typically sync back to the network faster if they hold valid state from a fast restart. This is because a memory node has no prior history. Mainnet typically suncs around ledger index ~125 (it starts counting at 0 as has no state).

A.8: Can I run multiple RWDB nodes in a cluster?

Yes, but each node maintains its own independent in-memory state. There is no built-in replication or state sharing between RWDB nodes. For high availability, deploy multiple independent RWDB nodes behind a load balancer.

A.9: Does RWDB work with Clio?

Yes. RWDB is an excellent backend for xrpld instances that are paired with Clio. Clio handles historical data queries and persistence, while the RWDB-backed xrpld focuses on validation and recent ledger processing.

A.10: What is the minimum recommended memory for an RWDB node?

For mainnet, a synced RWDB node typically consumes 11-13GB of RAM. We recommend deploying with at least 16GB to provide headroom for peak usage during sync and ledger processing. For testnet/devnet, significantly less memory is required.

A.11: Can I use RWDB on testnet or devnet?

Yes. RWDB works on any XRPL network. The memory footprint will be proportional to the network's ledger size. Testnet and devnet have much smaller ledgers, requiring less memory.

A.12: Can I use RWDB for local development?

Yes. RWDB works on any XRPL network. The none disk load can actually help with SSD life times as they have a limited read/write cycle.

A.13: Does RWDB affect peer-to-peer node propagation?

No. RWDB nodes participate in the peer overlay network identically to other node types. Node requests that cannot be satisfied from memory will fall back to the TreeNodeCache or in-memory ledgers, then propagate to peers if unavailable locally.

A.14: What happens during a network amendment with RWDB?

RWDB nodes process amendments identically to other node types. The amendment process is independent of the node store backend. If an amendment requires ledger state that is outside the retention window, the node will need to fetch it from peers (same as any non-archive node).

A.15: Does RWDB persist its state to disck on shutdown?

No. RWDB in the current implementation does not performa this on node shut down (as it could speed up a nodes restart). This is left to future adjustments to the protocol.

Acknowledgements

I would like to thank the Xahau project for their pioneering implementation of in-memory RWDB backends, which served as the foundation for this feature. Additionally, I thank the XRPL community members and node operators who provided feedback on the design and use cases during development.

Additional Links

• Implementation PR
• Original XAHAU Implementation
• XRPLD Documentation