Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
715ffbf
feat: add chainlock to inv
PastaPastaPasta Mar 27, 2025
3c69899
add chainlock / islock stuff; request chainlocks we see in inv
PastaPastaPasta May 11, 2025
7604f94
bloom no work
PastaPastaPasta May 24, 2025
592dcca
compact filters
PastaPastaPasta May 25, 2025
f609bf0
dash-spv crate
PastaPastaPasta May 25, 2025
baab12a
feat: implement BIP158 filter matching and comprehensive SPV monitoring
PastaPastaPasta May 26, 2025
1c5d02e
"Add Improved Network Message Handling and Block Processing
PastaPastaPasta May 26, 2025
e3acc3c
"fix: Update Regtest network constants and genesis block"
PastaPastaPasta May 26, 2025
24ef04e
feat: add batch header loading and reverse index to storage
PastaPastaPasta May 27, 2025
1afdc6d
perf: optimize masternode sync header feeding by 1000x
PastaPastaPasta May 27, 2025
e0e3297
feat: add modern terminal UI with real-time status display
PastaPastaPasta May 27, 2025
0d0b1ba
feat: integrate terminal UI with SPV client
PastaPastaPasta May 27, 2025
97d9683
feat: add terminal UI support to CLI and improve logging
PastaPastaPasta May 27, 2025
0f3de38
refactor: minor improvements to sync modules
PastaPastaPasta May 27, 2025
1733c03
remove redundant chainlock storage
PastaPastaPasta May 28, 2025
fa786a7
adjust how blocks are fed to masternode engine to avoid redundant blo…
PastaPastaPasta May 28, 2025
b423c28
reduce verbose logging
PastaPastaPasta May 28, 2025
c4b2d51
adds batch of tests, some that should've been commited earlier
PastaPastaPasta May 28, 2025
0af12d1
p2p: connect to multiple nodes, multiple threads
PastaPastaPasta May 28, 2025
953a29a
fixup network constants
PastaPastaPasta May 29, 2025
bfff00b
fix: correct genesis_block static values for mainnet
PastaPastaPasta Jun 2, 2025
167159c
feat: implement UTXO tracking and wallet functionality
PastaPastaPasta Jun 2, 2025
8c1ea6f
refactor: resolve cargo check warnings
PastaPastaPasta Jun 2, 2025
9facca2
feat: improve network architecture and multi-peer management
PastaPastaPasta Jun 3, 2025
8f33d4b
feat: enhance sync system with robust coordination and recovery
PastaPastaPasta Jun 3, 2025
7a44359
feat: implement centralized message routing and coordination
PastaPastaPasta Jun 3, 2025
d429ad0
test: add comprehensive sync testing and verification utilities
PastaPastaPasta Jun 3, 2025
50725a3
fix: correct sync state management to prevent premature completion
PastaPastaPasta Jun 3, 2025
c9c2b81
fix: add proper sync state completion handling in client
PastaPastaPasta Jun 3, 2025
022dc5f
debug: add comprehensive logging to header sync manager
PastaPastaPasta Jun 3, 2025
24fdba2
improve: enhance network error handling for checksum failures
PastaPastaPasta Jun 3, 2025
dcee516
docs: add comprehensive project documentation
PastaPastaPasta Jun 3, 2025
bb9bcd5
fix: resolve header sync state management issues
PastaPastaPasta Jun 4, 2025
ba077a2
fix: resolve critical race condition in header sync timing
PastaPastaPasta Jun 4, 2025
6b245b1
fix: resolve race condition by coordinating sync with monitoring loop
PastaPastaPasta Jun 4, 2025
c6a7b49
feat: implement interleaved header and filter header sync
PastaPastaPasta Jun 4, 2025
f0bd5c9
fix: ensure filter header requests are sent even when sync is active
PastaPastaPasta Jun 4, 2025
2d126f5
fix: remove hardcoded 10000 height limit in filter header sync
PastaPastaPasta Jun 4, 2025
ec59841
improve: enhance network monitoring resilience during peer disconnect…
PastaPastaPasta Jun 4, 2025
7f098fd
fix: improve storage concurrency safety in header storage
PastaPastaPasta Jun 4, 2025
2d9c5a9
improve: add adaptive timeout handling for header sync
PastaPastaPasta Jun 4, 2025
85fa18f
refactor: simplify filter header sync coordination logic
PastaPastaPasta Jun 4, 2025
b12b437
fix: auto-trigger masternode sync after header sync completion
PastaPastaPasta Jun 4, 2025
2c5bfdb
improve: enhance dash-spv CLAUDE.md with debugging and implementation…
PastaPastaPasta Jun 4, 2025
6c4875d
fix: resolve CFilter message processing and add comprehensive debug l…
PastaPastaPasta Jun 4, 2025
dd8f419
fix: resolve storage layer race condition in segmented eviction
PastaPastaPasta Jun 4, 2025
4fabfee
fix: resolve UTXO serialization and balance calculation issues
PastaPastaPasta Jun 5, 2025
ea7fdae
fix: resolve filter header sync storage consistency issue
PastaPastaPasta Jun 5, 2025
adda3ba
fix: implement exclusive peer connection mode and adjust peer discove…
PastaPastaPasta Jun 5, 2025
f57ac13
fix: resolve WatchItem deserialization issue with earliest_height field
PastaPastaPasta Jun 7, 2025
c53819b
feat: add ISLock message support to network message parsing
PastaPastaPasta Jun 7, 2025
64ed637
fix: enhance ProTx parsing logic for BasicBLS version and platform fi…
PastaPastaPasta Jun 7, 2025
16f1bc5
refactor: complete overhaul of filter processing architecture for bet…
PastaPastaPasta Jun 7, 2025
591e29b
fix: address over-reading issue in coinbase payload decoding for vers…
PastaPastaPasta Jun 8, 2025
789e70b
debug: log block hash for blocks the fail deser
PastaPastaPasta Jun 8, 2025
9dc9fd7
feat: implement request timeout handling and tracking for network mes…
PastaPastaPasta Jun 8, 2025
78bbb28
feat: add blocks_processed counter to SpvStats with logging
PastaPastaPasta Jun 8, 2025
5efdbf9
feat: add MNHF Signal transaction support in special transaction hand…
PastaPastaPasta Jun 9, 2025
46a48dd
feat: implement in-memory UTXO cache with disk persistence and addres…
PastaPastaPasta Jun 9, 2025
cba4029
feat: implement asynchronous block processing with dedicated worker a…
PastaPastaPasta Jun 9, 2025
e46fee7
feat: integrate storage access for current blockchain tip height retr…
PastaPastaPasta Jun 9, 2025
ba8d380
feat: clean up whitespace and improve code readability in mod.rs
PastaPastaPasta Jun 10, 2025
17d7d81
refactor: modularize SPV client and add wallet management functionality
PastaPastaPasta Jun 10, 2025
7e2c6f8
feat: implement filter sync tracking and progress reporting
PastaPastaPasta Jun 11, 2025
e9f76b2
feat: enhance filter synchronization with flow control and request tr…
PastaPastaPasta Jun 14, 2025
a029a98
refactor: de-duplicate filter header chain verification logic
PastaPastaPasta Jun 14, 2025
2051bda
feat: add CFHeader gap detection and auto-restart functionality
PastaPastaPasta Jun 16, 2025
050efa7
refactor: remove redundant CLSig and ISLock structs
PastaPastaPasta Jun 17, 2025
95fe46a
fix: build failure re CLSig and ISLock messages
PastaPastaPasta Jun 17, 2025
99aaec3
fix: add NetworkExt import to multiple files
PastaPastaPasta Jun 17, 2025
bdb33b0
refactor: simplify WatchManager usage by removing instance creation
PastaPastaPasta Jun 17, 2025
ac91ae9
fix: change logging initialization to use try_init with error handling
PastaPastaPasta Jun 17, 2025
dac0a7c
feat: implement handshake timeout mechanism with message polling
PastaPastaPasta Jun 17, 2025
e7b1bb0
fix: add terminal size check before drawing status bar
PastaPastaPasta Jun 17, 2025
d0fbb6c
fix: update filter segment paths to use the correct directory
PastaPastaPasta Jun 17, 2025
05de85f
fix: improve error handling for directory creation in disk module
PastaPastaPasta Jun 17, 2025
b1fe1c7
fix: ensure proper sync state handling during masternode synchronization
PastaPastaPasta Jun 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[workspace]
members = ["dash", "dash-network", "dash-network-ffi", "hashes", "internals", "fuzz", "rpc-client", "rpc-json", "rpc-integration-test", "key-wallet", "key-wallet-ffi"]
members = ["dash", "dash-network", "dash-network-ffi", "hashes", "internals", "fuzz", "rpc-client", "rpc-json", "rpc-integration-test", "key-wallet", "key-wallet-ffi", "dash-spv"]
resolver = "2"

[workspace.package]
Expand Down
1 change: 1 addition & 0 deletions block_with_pro_reg_tx.data

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion dash-network/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ impl Network {
Network::Dash => 0xBD6B0CBF,
Network::Testnet => 0xFFCAE2CE,
Network::Devnet => 0xCEFFCAE2,
Network::Regtest => 0xDAB5BFFA,
Network::Regtest => 0xDCB7C1FC,
}
Comment on lines +61 to 62
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Magic-byte update is only half-done – networking will mis-detect Regtest

You updated Network::magic() to 0xDCB7C1FC, but:

  1. Network::from_magic() (line 39) still returns the old value (0xDAB5BFFA).
  2. Unit tests below expect the old constant and now fail.
-            0xDAB5BFFA => Some(Network::Regtest),
+            0xDCB7C1FC => Some(Network::Regtest),

Don’t forget to adjust the corresponding expectations in tests::test_network_magic and tests::test_network_from_magic; otherwise the crate will not compile.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Network::Regtest => 0xDCB7C1FC,
}
// dash-network/src/lib.rs
impl Network {
pub fn from_magic(magic: u32) -> Option<Network> {
match magic {
0xF9BEB4D9 => Some(Network::Mainnet),
0x0F1C2E3A => Some(Network::Testnet),
- 0xDAB5BFFA => Some(Network::Regtest),
+ 0xDCB7C1FC => Some(Network::Regtest),
_ => None,
}
}
pub fn magic(&self) -> u32 {
match self {
Network::Mainnet => 0xF9BEB4D9,
Network::Testnet => 0x0F1C2E3A,
Network::Regtest => 0xDCB7C1FC,
}
}
}
🤖 Prompt for AI Agents
In dash-network/src/lib.rs around lines 39 and 61-62, the magic byte for
Network::Regtest was updated to 0xDCB7C1FC in Network::magic() but not in
Network::from_magic(), which still returns the old value 0xDAB5BFFA. Update
Network::from_magic() to return the new magic byte 0xDCB7C1FC for Regtest as
well. Also, update the expected values in the unit tests test_network_magic and
test_network_from_magic to reflect the new magic byte so the tests pass and the
crate compiles.

}

Expand Down
225 changes: 225 additions & 0 deletions dash-spv/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,225 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

**dash-spv** is a Rust implementation of a Dash SPV (Simplified Payment Verification) client library built on top of the `dashcore` library. It provides a modular, async/await-based architecture for connecting to the Dash network, synchronizing blockchain data, and monitoring transactions.

## Architecture

The project follows a layered, trait-based architecture with clear separation of concerns:

### Core Modules
- **`client/`**: High-level client API (`DashSpvClient`) and configuration (`ClientConfig`)
- **`network/`**: TCP connections, handshake management, message routing, and peer management
- **`storage/`**: Storage abstraction with memory and disk backends via `StorageManager` trait
- **`sync/`**: Synchronization coordinators for headers, filters, and masternode data
- **`validation/`**: Header validation, ChainLock, and InstantLock verification
- **`wallet/`**: UTXO tracking, balance calculation, and transaction processing
- **`types.rs`**: Common data structures (`SyncProgress`, `ValidationMode`, `WatchItem`, etc.)
- **`error.rs`**: Unified error handling with domain-specific error types

### Key Design Patterns
- **Trait-based abstractions**: `NetworkManager`, `StorageManager` for swappable implementations
- **Async/await throughout**: Built on tokio runtime
- **State management**: Centralized sync coordination with `SyncState` and `SyncManager`
- **Modular validation**: Configurable validation modes (None/Basic/Full)

## Development Commands

### Building and Running
```bash
# Build the library
cargo build

# Run the SPV client binary
cargo run --bin dash-spv -- --network mainnet --data-dir ./spv-data

# Run with custom peer
cargo run --bin dash-spv -- --peer 192.168.1.100:9999

# Run examples
cargo run --example simple_sync
cargo run --example filter_sync
```

### Testing

**Unit and Integration Tests:**
```bash
# Run all tests
cargo test

# Run specific test files
cargo test --test handshake_test
cargo test --test header_sync_test
cargo test --test storage_test
cargo test --test integration_real_node_test

# Run individual test functions
cargo test --test handshake_test test_handshake_with_mainnet_peer

# Run tests with output
cargo test -- --nocapture

# Run single test with debug output
cargo test --test handshake_test test_handshake_with_mainnet_peer -- --nocapture
```

**Integration Tests with Real Node:**
The integration tests in `tests/integration_real_node_test.rs` connect to a live Dash Core node at `127.0.0.1:9999`. These tests gracefully skip if no node is available.

```bash
# Run real node integration tests
cargo test --test integration_real_node_test -- --nocapture

# Test specific real node functionality
cargo test --test integration_real_node_test test_real_header_sync_genesis_to_1000 -- --nocapture
```

See `run_integration_tests.md` for detailed setup instructions.

### Code Quality
```bash
# Check formatting
cargo fmt --check

# Run linter
cargo clippy --all-targets --all-features -- -D warnings

# Check all features compile
cargo check --all-features
```

## Key Concepts

### Sync Coordination
The `SyncManager` coordinates all synchronization through a state-based approach:
- Header sync via `HeaderSyncManager`
- Filter header sync via `FilterSyncManager`
- Masternode list sync via `MasternodeSyncManager`
- Centralized timeout handling and recovery

### Storage Backends
Two storage implementations via the `StorageManager` trait:
- `MemoryStorageManager`: In-memory storage for testing
- `DiskStorageManager`: Persistent disk storage for production

### Network Layer
TCP-based networking with proper Dash protocol implementation:
- Connection management via `TcpConnection`
- Handshake handling via `HandshakeManager`
- Message routing via `MessageHandler`
- Multi-peer support via `PeerManager`

### Validation Modes
- `ValidationMode::None`: No validation (fast)
- `ValidationMode::Basic`: Basic structure and timestamp validation
- `ValidationMode::Full`: Complete PoW and chain validation

### Wallet Integration
Basic wallet functionality for address monitoring:
- UTXO tracking via `Utxo` struct
- Balance calculation with confirmation states
- Transaction processing via `TransactionProcessor`

## Testing Strategy

### Test Organization
- **Unit tests**: In-module tests for individual components
- **Integration tests**: `tests/` directory with comprehensive test suites
- **Real network tests**: Integration with live Dash Core nodes
- **Performance tests**: Sync rate and memory usage benchmarks

### Test Categories (from `tests/test_plan.md`)
1. **Network layer**: Handshake, connection management (3/4 passing)
2. **Storage layer**: Memory/disk operations (9/9 passing)
3. **Header sync**: Genesis to tip synchronization (11/11 passing)
4. **Integration**: Real node connectivity and performance (6/6 passing)

### Test Data Requirements
- Dash Core node at `127.0.0.1:9999` for integration tests
- Tests gracefully handle node unavailability
- Performance benchmarks expect 50-200+ headers/second sync rates

## Development Workflow

### Working with Sync
The sync system uses a monitoring loop pattern:
1. Call `sync_*()` methods to start sync processes
2. The monitoring loop calls `handle_*_message()` for incoming data
3. Use `check_sync_timeouts()` for timeout recovery
4. Sync completion is tracked via `SyncState`

### Adding New Features
1. Define traits for abstractions (e.g., new storage backend)
2. Implement concrete types following existing patterns
3. Add comprehensive unit tests
4. Add integration tests if network interaction is involved
5. Update error types in `error.rs` for new failure modes

### Error Handling
Use domain-specific error types:
- `NetworkError`: Connection and protocol issues
- `StorageError`: Data persistence problems
- `SyncError`: Synchronization failures
- `ValidationError`: Header and transaction validation issues
- `SpvError`: Top-level errors wrapping specific domains

## MSRV and Dependencies

- **Minimum Rust Version**: 1.80
- **Core dependencies**: `dashcore`, `tokio`, `async-trait`, `thiserror`
- **Built on**: `dashcore` library with Dash-specific features enabled
- **Async runtime**: Tokio with full feature set

## Key Implementation Details

### Storage Architecture
- **Segmented storage**: Headers stored in 10,000-header segments with index files
- **Filter storage**: Separate storage for filter headers and compact block filters
- **State persistence**: Chain state, masternode data, and sync progress persisted between runs
- **Storage paths**: Headers in `headers/`, filters in `filters/`, state in `state/`

### Async Architecture Patterns
- **Trait objects**: `Arc<dyn StorageManager>`, `Arc<dyn NetworkManager>` for runtime polymorphism
- **Message passing**: Tokio channels for inter-component communication
- **Timeout handling**: Configurable timeouts with recovery mechanisms
- **State machines**: `SyncState` enum drives synchronization flow

### Debugging and Troubleshooting

**Common Debug Commands:**
```bash
# Run with tracing output
RUST_LOG=debug cargo test --test integration_real_node_test -- --nocapture

# Run specific test with verbose output
cargo test --test handshake_test test_handshake_with_mainnet_peer -- --nocapture --test-threads=1

# Check storage state
ls -la data*/headers/
ls -la data*/state/
```

**Debug Data Locations:**
- `test-debug/`: Debug data from test runs
- `data*/`: Runtime data directories (numbered by run)
- Storage index files show header counts and segment info

**Network Debugging:**
- Connection issues: Check if Dash Core node is running at `127.0.0.1:9999`
- Handshake failures: Verify network (mainnet/testnet/devnet) matches node
- Timeout issues: Node may be syncing or under load

## Current Status

This is a refactored SPV client extracted from a monolithic example:
- ✅ Core architecture implemented and modular
- ✅ Compilation successful with comprehensive trait abstractions
- ✅ Extensive test coverage (29/29 implemented tests passing)
- ⚠️ Some wallet functionality still in development (see `PLAN.md`)
- ⚠️ ChainLock/InstantLock signature validation has TODO items

The project transforms a 1,143-line monolithic example into a production-ready, testable library suitable for integration into wallets and other Dash applications.
60 changes: 60 additions & 0 deletions dash-spv/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
[package]
name = "dash-spv"
version = "0.1.0"
edition = "2021"
authors = ["Dash Core Team"]
description = "Dash SPV (Simplified Payment Verification) client library"
license = "MIT"
repository = "https://github.com/dashpay/rust-dashcore"
rust-version = "1.80"

[dependencies]
# Core Dash libraries
dashcore = { path = "../dash", features = ["std", "serde", "core-block-hash-use-x11", "message_verification"] }
dashcore_hashes = { path = "../hashes" }

Comment on lines +13 to +15
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

path dependencies block publishing

Crates with path = "../dash" / "../hashes" cannot be published to crates.io.
If public release is a goal, switch to versioned dependencies or add:

[package]
publish = false
🤖 Prompt for AI Agents
In dash-spv/Cargo.toml around lines 13 to 15, the dependencies dashcore and
dashcore_hashes use local path references which prevent publishing to crates.io.
To fix this, either replace these path dependencies with versioned dependencies
from crates.io if available, or add 'publish = false' under the [package]
section to explicitly mark the crate as non-publishable.

# CLI
clap = { version = "4.0", features = ["derive"] }

# Async runtime
tokio = { version = "1.0", features = ["full"] }
async-trait = "0.1"

# Error handling
thiserror = "1.0"
anyhow = "1.0"

# Serialization
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
bincode = "1.3"

# Logging
tracing = "0.1"
tracing-subscriber = "0.3"

# Utilities
rand = "0.8"

# Terminal UI
crossterm = "0.27"

# DNS
trust-dns-resolver = "0.23"

# Also add log to main dependencies for consistency
log = "0.4"

[dev-dependencies]
tempfile = "3.0"
tokio-test = "0.4"
env_logger = "0.10"
hex = "0.4"

[[bin]]
name = "dash-spv"
path = "src/main.rs"

[lib]
name = "dash_spv"
path = "src/lib.rs"
Loading
Loading