feat: make rocksdb cache configurable per CF #2132

[bronzelle-cw]

User description

Description

This PR addresses issue #2132 by making RocksDB cache configurable per Column Family (CF), replacing the previous hardcoded cache values and cache_size_multiplier approach.

Changes Made

New Configuration System

• Created RocksCfCacheConfig: A new configuration struct that allows individual cache size settings for each Column Family
• Individual CF configurations: Each CF now has its own configurable cache size with appropriate defaults:
  • accounts: 10GB (optimized for frequent point lookups)
  • account_slots: 30GB (largest cache for high-volume slot storage)
  • accounts_history: 2GB
  • account_slots_history: 2GB
  • transactions: 2GB
  • blocks_by_number: 2GB
  • blocks_by_hash: 2GB
  • block_changes: 2GB

CLI and Environment Variable Support

• Each CF cache size can be configured via:
  • Command line arguments (e.g., --rocks-cf-cache-accounts)
  • Environment variables (e.g., ROCKS_CF_CACHE_ACCOUNTS)
  • Default values for production use

Integration Points

• Updated PermanentStorageConfig to include the new RocksCfCacheConfig
• Modified RocksStorageState to use the new configuration system
• Updated generate_cf_options_map() to apply individual cache settings

Cleanup

• Removed obsolete ROCKS_CACHE_SIZE_MULTIPLIER from configuration files
• Replaced hardcoded cache multiplier approach with explicit per-CF configuration

Configuration Examples

Command line:

stratus --rocks-cf-cache-accounts 15000000000 --rocks-cf-cache-account-slots 40000000000

Environment variables:

export ROCKS_CF_CACHE_ACCOUNTS=15000000000
export ROCKS_CF_CACHE_ACCOUNT_SLOTS=40000000000

Fixes #2132

---

PR Type

Enhancement, Configuration changes

---

Description

• Introduced RocksCfCacheConfig for per-CF cache configuration

• Replaced cache_size_multiplier with individual CF settings

• Updated storage initialization to use new config

• Removed obsolete ROCKS_CACHE_SIZE_MULTIPLIER configuration

---

Diagram Walkthrough

flowchart LR
  OldConfig["Old Cache Config"] -- "Replaced" --> NewConfig["RocksCfCacheConfig"]
  NewConfig -- "Configures" --> CFs["Individual Column Families"]
  CFs -- "Applied to" --> RocksDB["RocksDB Storage"]
  CLI["CLI Arguments"] -- "Sets" --> NewConfig
  ENV["Environment Variables"] -- "Sets" --> NewConfig

Loading

File Walkthrough

	Relevant files
Enhancement	5 files rocks_cf_cache_config.rs Introduce RocksCfCacheConfig struct for per-CF cache settings +102/-0  rocks_state.rs Update generate_cf_options_map to use RocksCfCacheConfig  +16/-22  rocks_permanent.rs Update RocksPermanentStorage to use RocksCfCacheConfig      +3/-2      data_migration.rs Update RocksStorageState initialization with RocksCfCacheConfig +2/-1      historic_events_processor.rs Update RocksStorageState initialization with RocksCfCacheConfig +2/-1
Configuration changes	2 files mod.rs Replace rocks_cache_size_multiplier with rocks_cf_cache in config +6/-5      importer-offline.env.local Remove obsolete ROCKS_CACHE_SIZE_MULTIPLIER configuration +0/-1
Tests	1 files stratus_storage.rs Update StratusStorage to use RocksCfCacheConfig in tests  +10/-3
Additional files	1 files mod.rs +4/-0

---

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Use configuration for cache sizes Consider using a configuration file or environment variables to set these cache sizes instead of hard-coding them. This would allow for easier tuning and adjustment of cache sizes without recompiling the code. src/eth/storage/permanent/rocks/rocks_cf_cache_config.rs [9-16] -const DEFAULT_ACCOUNTS_CACHE: usize = 10 * GIGABYTE; -const DEFAULT_ACCOUNTS_HISTORY_CACHE: usize = 2 * GIGABYTE; -const DEFAULT_ACCOUNT_SLOTS_CACHE: usize = 30 * GIGABYTE; -const DEFAULT_ACCOUNT_SLOTS_HISTORY_CACHE: usize = 2 * GIGABYTE; -const DEFAULT_TRANSACTIONS_CACHE: usize = 2 * GIGABYTE; -const DEFAULT_BLOCKS_BY_NUMBER_CACHE: usize = 2 * GIGABYTE; -const DEFAULT_BLOCKS_BY_HASH_CACHE: usize = 2 * GIGABYTE; -const DEFAULT_BLOCK_CHANGES_CACHE: usize = 2 * GIGABYTE; +use config::{Config, File}; +lazy_static! { + static ref CACHE_CONFIG: Config = { + Config::builder() + .add_source(File::with_name("cache_config.toml").required(false)) + .add_source(config::Environment::with_prefix("ROCKS_CF_CACHE")) + .build() + .unwrap() + }; +} + +const fn get_cache_size(key: &str, default: usize) -> usize { + CACHE_CONFIG.get_int(key).unwrap_or(default as i64) as usize +} + +const DEFAULT_ACCOUNTS_CACHE: usize = get_cache_size("accounts", 10 * GIGABYTE); +const DEFAULT_ACCOUNTS_HISTORY_CACHE: usize = get_cache_size("accounts_history", 2 * GIGABYTE); +const DEFAULT_ACCOUNT_SLOTS_CACHE: usize = get_cache_size("account_slots", 30 * GIGABYTE); +const DEFAULT_ACCOUNT_SLOTS_HISTORY_CACHE: usize = get_cache_size("account_slots_history", 2 * GIGABYTE); +const DEFAULT_TRANSACTIONS_CACHE: usize = get_cache_size("transactions", 2 * GIGABYTE); +const DEFAULT_BLOCKS_BY_NUMBER_CACHE: usize = get_cache_size("blocks_by_number", 2 * GIGABYTE); +const DEFAULT_BLOCKS_BY_HASH_CACHE: usize = get_cache_size("blocks_by_hash", 2 * GIGABYTE); +const DEFAULT_BLOCK_CHANGES_CACHE: usize = get_cache_size("block_changes", 2 * GIGABYTE); + Suggestion importance[1-10]: 7 __ Why: The suggestion to use a configuration file or environment variables for cache sizes is valuable. It enhances flexibility and ease of tuning without code changes. However, it's not addressing a critical issue, hence the score of 7.	Medium

[codecov]

Codecov Report

❌ Patch coverage is 68.62745% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.58%. Comparing base (57ec6ae) to head (47e786a).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...h/storage/permanent/rocks/rocks_cf_cache_config.rs	46.15%	14 Missing ⚠️
src/bin/historic_events_processor.rs	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2282      +/-   ##
==========================================
- Coverage   84.65%   84.58%   -0.08%     
==========================================
  Files         130      131       +1     
  Lines       10800    10826      +26     
==========================================
+ Hits         9143     9157      +14     
- Misses       1657     1669      +12     

Flag	Coverage Δ
contracts-rocks-	44.04% <34.88%> (-0.16%)	⬇️
e2e-admin-password	21.92% <34.88%> (-0.11%)	⬇️
e2e-clock-stratus	24.68% <34.88%> (-0.12%)	⬇️
e2e-genesis	26.17% <34.88%> (-0.12%)	⬇️
e2e-importer-offline	58.25% <34.88%> (-0.17%)	⬇️
e2e-rpc-downloader	53.41% <34.88%> (-0.09%)	⬇️
e2e-stratus	55.60% <34.88%> (-0.23%)	⬇️
leader-follower-	61.23% <34.88%> (-0.13%)	⬇️
rust-tests	31.66% <66.66%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.