- 
                Notifications
    
You must be signed in to change notification settings  - Fork 3
 
feat: initial implementation of SPV client in rust-dashcode #75
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
          
     Merged
      
      
    
  
     Merged
                    Changes from all commits
      Commits
    
    
            Show all changes
          
          
            77 commits
          
        
        Select commit
          Hold shift + click to select a range
      
      715ffbf
              
                feat: add chainlock to inv
              
              
                PastaPastaPasta 3c69899
              
                add chainlock / islock stuff; request chainlocks we see in inv
              
              
                PastaPastaPasta 7604f94
              
                bloom no work
              
              
                PastaPastaPasta 592dcca
              
                compact filters
              
              
                PastaPastaPasta f609bf0
              
                dash-spv crate
              
              
                PastaPastaPasta baab12a
              
                feat: implement BIP158 filter matching and comprehensive SPV monitoring
              
              
                PastaPastaPasta 1c5d02e
              
                "Add Improved Network Message Handling and Block Processing
              
              
                PastaPastaPasta e3acc3c
              
                "fix: Update Regtest network constants and genesis block"
              
              
                PastaPastaPasta 24ef04e
              
                feat: add batch header loading and reverse index to storage
              
              
                PastaPastaPasta 1afdc6d
              
                perf: optimize masternode sync header feeding by 1000x
              
              
                PastaPastaPasta e0e3297
              
                feat: add modern terminal UI with real-time status display
              
              
                PastaPastaPasta 0d0b1ba
              
                feat: integrate terminal UI with SPV client
              
              
                PastaPastaPasta 97d9683
              
                feat: add terminal UI support to CLI and improve logging
              
              
                PastaPastaPasta 0f3de38
              
                refactor: minor improvements to sync modules
              
              
                PastaPastaPasta 1733c03
              
                remove redundant chainlock storage
              
              
                PastaPastaPasta fa786a7
              
                adjust how blocks are fed to masternode engine to avoid redundant blo…
              
              
                PastaPastaPasta b423c28
              
                reduce verbose logging
              
              
                PastaPastaPasta c4b2d51
              
                adds batch of tests, some that should've been commited earlier
              
              
                PastaPastaPasta 0af12d1
              
                p2p: connect to multiple nodes, multiple threads
              
              
                PastaPastaPasta 953a29a
              
                fixup network constants
              
              
                PastaPastaPasta bfff00b
              
                fix: correct genesis_block static values for mainnet
              
              
                PastaPastaPasta 167159c
              
                feat: implement UTXO tracking and wallet functionality
              
              
                PastaPastaPasta 8c1ea6f
              
                refactor: resolve cargo check warnings
              
              
                PastaPastaPasta 9facca2
              
                feat: improve network architecture and multi-peer management
              
              
                PastaPastaPasta 8f33d4b
              
                feat: enhance sync system with robust coordination and recovery
              
              
                PastaPastaPasta 7a44359
              
                feat: implement centralized message routing and coordination
              
              
                PastaPastaPasta d429ad0
              
                test: add comprehensive sync testing and verification utilities
              
              
                PastaPastaPasta 50725a3
              
                fix: correct sync state management to prevent premature completion
              
              
                PastaPastaPasta c9c2b81
              
                fix: add proper sync state completion handling in client
              
              
                PastaPastaPasta 022dc5f
              
                debug: add comprehensive logging to header sync manager
              
              
                PastaPastaPasta 24fdba2
              
                improve: enhance network error handling for checksum failures
              
              
                PastaPastaPasta dcee516
              
                docs: add comprehensive project documentation
              
              
                PastaPastaPasta bb9bcd5
              
                fix: resolve header sync state management issues
              
              
                PastaPastaPasta ba077a2
              
                fix: resolve critical race condition in header sync timing
              
              
                PastaPastaPasta 6b245b1
              
                fix: resolve race condition by coordinating sync with monitoring loop
              
              
                PastaPastaPasta c6a7b49
              
                feat: implement interleaved header and filter header sync
              
              
                PastaPastaPasta f0bd5c9
              
                fix: ensure filter header requests are sent even when sync is active
              
              
                PastaPastaPasta 2d126f5
              
                fix: remove hardcoded 10000 height limit in filter header sync
              
              
                PastaPastaPasta ec59841
              
                improve: enhance network monitoring resilience during peer disconnect…
              
              
                PastaPastaPasta 7f098fd
              
                fix: improve storage concurrency safety in header storage
              
              
                PastaPastaPasta 2d9c5a9
              
                improve: add adaptive timeout handling for header sync
              
              
                PastaPastaPasta 85fa18f
              
                refactor: simplify filter header sync coordination logic
              
              
                PastaPastaPasta b12b437
              
                fix: auto-trigger masternode sync after header sync completion
              
              
                PastaPastaPasta 2c5bfdb
              
                improve: enhance dash-spv CLAUDE.md with debugging and implementation…
              
              
                PastaPastaPasta 6c4875d
              
                fix: resolve CFilter message processing and add comprehensive debug l…
              
              
                PastaPastaPasta dd8f419
              
                fix: resolve storage layer race condition in segmented eviction
              
              
                PastaPastaPasta 4fabfee
              
                fix: resolve UTXO serialization and balance calculation issues
              
              
                PastaPastaPasta ea7fdae
              
                fix: resolve filter header sync storage consistency issue
              
              
                PastaPastaPasta adda3ba
              
                fix: implement exclusive peer connection mode and adjust peer discove…
              
              
                PastaPastaPasta f57ac13
              
                fix: resolve WatchItem deserialization issue with earliest_height field
              
              
                PastaPastaPasta c53819b
              
                feat: add ISLock message support to network message parsing
              
              
                PastaPastaPasta 64ed637
              
                fix: enhance ProTx parsing logic for BasicBLS version and platform fi…
              
              
                PastaPastaPasta 16f1bc5
              
                refactor: complete overhaul of filter processing architecture for bet…
              
              
                PastaPastaPasta 591e29b
              
                fix: address over-reading issue in coinbase payload decoding for vers…
              
              
                PastaPastaPasta 789e70b
              
                debug: log block hash for blocks the fail deser
              
              
                PastaPastaPasta 9dc9fd7
              
                feat: implement request timeout handling and tracking for network mes…
              
              
                PastaPastaPasta 78bbb28
              
                feat: add blocks_processed counter to SpvStats with logging
              
              
                PastaPastaPasta 5efdbf9
              
                feat: add MNHF Signal transaction support in special transaction hand…
              
              
                PastaPastaPasta 46a48dd
              
                feat: implement in-memory UTXO cache with disk persistence and addres…
              
              
                PastaPastaPasta cba4029
              
                feat: implement asynchronous block processing with dedicated worker a…
              
              
                PastaPastaPasta e46fee7
              
                feat: integrate storage access for current blockchain tip height retr…
              
              
                PastaPastaPasta ba8d380
              
                feat: clean up whitespace and improve code readability in mod.rs
              
              
                PastaPastaPasta 17d7d81
              
                refactor: modularize SPV client and add wallet management functionality
              
              
                PastaPastaPasta 7e2c6f8
              
                feat: implement filter sync tracking and progress reporting
              
              
                PastaPastaPasta e9f76b2
              
                feat: enhance filter synchronization with flow control and request tr…
              
              
                PastaPastaPasta a029a98
              
                refactor: de-duplicate filter header chain verification logic
              
              
                PastaPastaPasta 2051bda
              
                feat: add CFHeader gap detection and auto-restart functionality
              
              
                PastaPastaPasta 050efa7
              
                refactor: remove redundant CLSig and ISLock structs
              
              
                PastaPastaPasta 95fe46a
              
                fix: build failure re CLSig and ISLock messages
              
              
                PastaPastaPasta 99aaec3
              
                fix: add NetworkExt import to multiple files
              
              
                PastaPastaPasta bdb33b0
              
                refactor: simplify WatchManager usage by removing instance creation
              
              
                PastaPastaPasta ac91ae9
              
                fix: change logging initialization to use try_init with error handling
              
              
                PastaPastaPasta dac0a7c
              
                feat: implement handshake timeout mechanism with message polling
              
              
                PastaPastaPasta e7b1bb0
              
                fix: add terminal size check before drawing status bar
              
              
                PastaPastaPasta d0fbb6c
              
                fix: update filter segment paths to use the correct directory
              
              
                PastaPastaPasta 05de85f
              
                fix: improve error handling for directory creation in disk module
              
              
                PastaPastaPasta b1fe1c7
              
                fix: ensure proper sync state handling during masternode synchronization
              
              
                PastaPastaPasta File filter
Filter by extension
Conversations
          Failed to load comments.   
        
        
          
      Loading
        
  Jump to
        
          Jump to file
        
      
      
          Failed to load files.   
        
        
          
      Loading
        
  Diff view
Diff view
There are no files selected for viewing
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              Large diffs are not rendered by default.
      
      Oops, something went wrong.
      
    
  
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,225 @@ | ||
| # CLAUDE.md | ||
| 
     | 
||
| This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. | ||
| 
     | 
||
| ## Project Overview | ||
| 
     | 
||
| **dash-spv** is a Rust implementation of a Dash SPV (Simplified Payment Verification) client library built on top of the `dashcore` library. It provides a modular, async/await-based architecture for connecting to the Dash network, synchronizing blockchain data, and monitoring transactions. | ||
| 
     | 
||
| ## Architecture | ||
| 
     | 
||
| The project follows a layered, trait-based architecture with clear separation of concerns: | ||
| 
     | 
||
| ### Core Modules | ||
| - **`client/`**: High-level client API (`DashSpvClient`) and configuration (`ClientConfig`) | ||
| - **`network/`**: TCP connections, handshake management, message routing, and peer management | ||
| - **`storage/`**: Storage abstraction with memory and disk backends via `StorageManager` trait | ||
| - **`sync/`**: Synchronization coordinators for headers, filters, and masternode data | ||
| - **`validation/`**: Header validation, ChainLock, and InstantLock verification | ||
| - **`wallet/`**: UTXO tracking, balance calculation, and transaction processing | ||
| - **`types.rs`**: Common data structures (`SyncProgress`, `ValidationMode`, `WatchItem`, etc.) | ||
| - **`error.rs`**: Unified error handling with domain-specific error types | ||
| 
     | 
||
| ### Key Design Patterns | ||
| - **Trait-based abstractions**: `NetworkManager`, `StorageManager` for swappable implementations | ||
| - **Async/await throughout**: Built on tokio runtime | ||
| - **State management**: Centralized sync coordination with `SyncState` and `SyncManager` | ||
| - **Modular validation**: Configurable validation modes (None/Basic/Full) | ||
| 
     | 
||
| ## Development Commands | ||
| 
     | 
||
| ### Building and Running | ||
| ```bash | ||
| # Build the library | ||
| cargo build | ||
| 
     | 
||
| # Run the SPV client binary | ||
| cargo run --bin dash-spv -- --network mainnet --data-dir ./spv-data | ||
| 
     | 
||
| # Run with custom peer | ||
| cargo run --bin dash-spv -- --peer 192.168.1.100:9999 | ||
| 
     | 
||
| # Run examples | ||
| cargo run --example simple_sync | ||
| cargo run --example filter_sync | ||
| ``` | ||
| 
     | 
||
| ### Testing | ||
| 
     | 
||
| **Unit and Integration Tests:** | ||
| ```bash | ||
| # Run all tests | ||
| cargo test | ||
| 
     | 
||
| # Run specific test files | ||
| cargo test --test handshake_test | ||
| cargo test --test header_sync_test | ||
| cargo test --test storage_test | ||
| cargo test --test integration_real_node_test | ||
| 
     | 
||
| # Run individual test functions | ||
| cargo test --test handshake_test test_handshake_with_mainnet_peer | ||
| 
     | 
||
| # Run tests with output | ||
| cargo test -- --nocapture | ||
| 
     | 
||
| # Run single test with debug output | ||
| cargo test --test handshake_test test_handshake_with_mainnet_peer -- --nocapture | ||
| ``` | ||
| 
     | 
||
| **Integration Tests with Real Node:** | ||
| The integration tests in `tests/integration_real_node_test.rs` connect to a live Dash Core node at `127.0.0.1:9999`. These tests gracefully skip if no node is available. | ||
| 
     | 
||
| ```bash | ||
| # Run real node integration tests | ||
| cargo test --test integration_real_node_test -- --nocapture | ||
| 
     | 
||
| # Test specific real node functionality | ||
| cargo test --test integration_real_node_test test_real_header_sync_genesis_to_1000 -- --nocapture | ||
| ``` | ||
| 
     | 
||
| See `run_integration_tests.md` for detailed setup instructions. | ||
| 
     | 
||
| ### Code Quality | ||
| ```bash | ||
| # Check formatting | ||
| cargo fmt --check | ||
| 
     | 
||
| # Run linter | ||
| cargo clippy --all-targets --all-features -- -D warnings | ||
| 
     | 
||
| # Check all features compile | ||
| cargo check --all-features | ||
| ``` | ||
| 
     | 
||
| ## Key Concepts | ||
| 
     | 
||
| ### Sync Coordination | ||
| The `SyncManager` coordinates all synchronization through a state-based approach: | ||
| - Header sync via `HeaderSyncManager` | ||
| - Filter header sync via `FilterSyncManager` | ||
| - Masternode list sync via `MasternodeSyncManager` | ||
| - Centralized timeout handling and recovery | ||
| 
     | 
||
| ### Storage Backends | ||
| Two storage implementations via the `StorageManager` trait: | ||
| - `MemoryStorageManager`: In-memory storage for testing | ||
| - `DiskStorageManager`: Persistent disk storage for production | ||
| 
     | 
||
| ### Network Layer | ||
| TCP-based networking with proper Dash protocol implementation: | ||
| - Connection management via `TcpConnection` | ||
| - Handshake handling via `HandshakeManager` | ||
| - Message routing via `MessageHandler` | ||
| - Multi-peer support via `PeerManager` | ||
| 
     | 
||
| ### Validation Modes | ||
| - `ValidationMode::None`: No validation (fast) | ||
| - `ValidationMode::Basic`: Basic structure and timestamp validation | ||
| - `ValidationMode::Full`: Complete PoW and chain validation | ||
| 
     | 
||
| ### Wallet Integration | ||
| Basic wallet functionality for address monitoring: | ||
| - UTXO tracking via `Utxo` struct | ||
| - Balance calculation with confirmation states | ||
| - Transaction processing via `TransactionProcessor` | ||
| 
     | 
||
| ## Testing Strategy | ||
| 
     | 
||
| ### Test Organization | ||
| - **Unit tests**: In-module tests for individual components | ||
| - **Integration tests**: `tests/` directory with comprehensive test suites | ||
| - **Real network tests**: Integration with live Dash Core nodes | ||
| - **Performance tests**: Sync rate and memory usage benchmarks | ||
| 
     | 
||
| ### Test Categories (from `tests/test_plan.md`) | ||
| 1. **Network layer**: Handshake, connection management (3/4 passing) | ||
| 2. **Storage layer**: Memory/disk operations (9/9 passing) | ||
| 3. **Header sync**: Genesis to tip synchronization (11/11 passing) | ||
| 4. **Integration**: Real node connectivity and performance (6/6 passing) | ||
| 
     | 
||
| ### Test Data Requirements | ||
| - Dash Core node at `127.0.0.1:9999` for integration tests | ||
| - Tests gracefully handle node unavailability | ||
| - Performance benchmarks expect 50-200+ headers/second sync rates | ||
| 
     | 
||
| ## Development Workflow | ||
| 
     | 
||
| ### Working with Sync | ||
| The sync system uses a monitoring loop pattern: | ||
| 1. Call `sync_*()` methods to start sync processes | ||
| 2. The monitoring loop calls `handle_*_message()` for incoming data | ||
| 3. Use `check_sync_timeouts()` for timeout recovery | ||
| 4. Sync completion is tracked via `SyncState` | ||
| 
     | 
||
| ### Adding New Features | ||
| 1. Define traits for abstractions (e.g., new storage backend) | ||
| 2. Implement concrete types following existing patterns | ||
| 3. Add comprehensive unit tests | ||
| 4. Add integration tests if network interaction is involved | ||
| 5. Update error types in `error.rs` for new failure modes | ||
| 
     | 
||
| ### Error Handling | ||
| Use domain-specific error types: | ||
| - `NetworkError`: Connection and protocol issues | ||
| - `StorageError`: Data persistence problems | ||
| - `SyncError`: Synchronization failures | ||
| - `ValidationError`: Header and transaction validation issues | ||
| - `SpvError`: Top-level errors wrapping specific domains | ||
| 
     | 
||
| ## MSRV and Dependencies | ||
| 
     | 
||
| - **Minimum Rust Version**: 1.80 | ||
| - **Core dependencies**: `dashcore`, `tokio`, `async-trait`, `thiserror` | ||
| - **Built on**: `dashcore` library with Dash-specific features enabled | ||
| - **Async runtime**: Tokio with full feature set | ||
| 
     | 
||
| ## Key Implementation Details | ||
| 
     | 
||
| ### Storage Architecture | ||
| - **Segmented storage**: Headers stored in 10,000-header segments with index files | ||
| - **Filter storage**: Separate storage for filter headers and compact block filters | ||
| - **State persistence**: Chain state, masternode data, and sync progress persisted between runs | ||
| - **Storage paths**: Headers in `headers/`, filters in `filters/`, state in `state/` | ||
| 
     | 
||
| ### Async Architecture Patterns | ||
| - **Trait objects**: `Arc<dyn StorageManager>`, `Arc<dyn NetworkManager>` for runtime polymorphism | ||
| - **Message passing**: Tokio channels for inter-component communication | ||
| - **Timeout handling**: Configurable timeouts with recovery mechanisms | ||
| - **State machines**: `SyncState` enum drives synchronization flow | ||
| 
     | 
||
| ### Debugging and Troubleshooting | ||
| 
     | 
||
| **Common Debug Commands:** | ||
| ```bash | ||
| # Run with tracing output | ||
| RUST_LOG=debug cargo test --test integration_real_node_test -- --nocapture | ||
| 
     | 
||
| # Run specific test with verbose output | ||
| cargo test --test handshake_test test_handshake_with_mainnet_peer -- --nocapture --test-threads=1 | ||
| 
     | 
||
| # Check storage state | ||
| ls -la data*/headers/ | ||
| ls -la data*/state/ | ||
| ``` | ||
| 
     | 
||
| **Debug Data Locations:** | ||
| - `test-debug/`: Debug data from test runs | ||
| - `data*/`: Runtime data directories (numbered by run) | ||
| - Storage index files show header counts and segment info | ||
| 
     | 
||
| **Network Debugging:** | ||
| - Connection issues: Check if Dash Core node is running at `127.0.0.1:9999` | ||
| - Handshake failures: Verify network (mainnet/testnet/devnet) matches node | ||
| - Timeout issues: Node may be syncing or under load | ||
| 
     | 
||
| ## Current Status | ||
| 
     | 
||
| This is a refactored SPV client extracted from a monolithic example: | ||
| - ✅ Core architecture implemented and modular | ||
| - ✅ Compilation successful with comprehensive trait abstractions | ||
| - ✅ Extensive test coverage (29/29 implemented tests passing) | ||
| - ⚠️ Some wallet functionality still in development (see `PLAN.md`) | ||
| - ⚠️ ChainLock/InstantLock signature validation has TODO items | ||
| 
     | 
||
| The project transforms a 1,143-line monolithic example into a production-ready, testable library suitable for integration into wallets and other Dash applications. | 
  
    
      This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
      Learn more about bidirectional Unicode characters
    
  
  
    
              
              | Original file line number | Diff line number | Diff line change | 
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| [package] | ||
| name = "dash-spv" | ||
| version = "0.1.0" | ||
| edition = "2021" | ||
| authors = ["Dash Core Team"] | ||
| description = "Dash SPV (Simplified Payment Verification) client library" | ||
| license = "MIT" | ||
| repository = "https://github.com/dashpay/rust-dashcore" | ||
| rust-version = "1.80" | ||
| 
     | 
||
| [dependencies] | ||
| # Core Dash libraries | ||
| dashcore = { path = "../dash", features = ["std", "serde", "core-block-hash-use-x11", "message_verification"] } | ||
| dashcore_hashes = { path = "../hashes" } | ||
| 
     | 
||
| 
         
      Comment on lines
    
      +13
     to 
      +15
    
   
  There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🛠️ Refactor suggestion 
 Crates with  [package]
publish = false🤖 Prompt for AI Agents | 
||
| # CLI | ||
| clap = { version = "4.0", features = ["derive"] } | ||
| 
     | 
||
| # Async runtime | ||
| tokio = { version = "1.0", features = ["full"] } | ||
| async-trait = "0.1" | ||
| 
     | 
||
| # Error handling | ||
| thiserror = "1.0" | ||
| anyhow = "1.0" | ||
| 
     | 
||
| # Serialization | ||
| serde = { version = "1.0", features = ["derive"] } | ||
| serde_json = "1.0" | ||
| bincode = "1.3" | ||
| 
     | 
||
| # Logging | ||
| tracing = "0.1" | ||
| tracing-subscriber = "0.3" | ||
| 
     | 
||
| # Utilities | ||
| rand = "0.8" | ||
| 
     | 
||
| # Terminal UI | ||
| crossterm = "0.27" | ||
| 
     | 
||
| # DNS | ||
| trust-dns-resolver = "0.23" | ||
| 
     | 
||
| # Also add log to main dependencies for consistency | ||
| log = "0.4" | ||
| 
     | 
||
| [dev-dependencies] | ||
| tempfile = "3.0" | ||
| tokio-test = "0.4" | ||
| env_logger = "0.10" | ||
| hex = "0.4" | ||
| 
     | 
||
| [[bin]] | ||
| name = "dash-spv" | ||
| path = "src/main.rs" | ||
| 
     | 
||
| [lib] | ||
| name = "dash_spv" | ||
| path = "src/lib.rs" | ||
      
      Oops, something went wrong.
        
    
  
      
      Oops, something went wrong.
        
    
  
  Add this suggestion to a batch that can be applied as a single commit.
  This suggestion is invalid because no changes were made to the code.
  Suggestions cannot be applied while the pull request is closed.
  Suggestions cannot be applied while viewing a subset of changes.
  Only one suggestion per line can be applied in a batch.
  Add this suggestion to a batch that can be applied as a single commit.
  Applying suggestions on deleted lines is not supported.
  You must change the existing code in this line in order to create a valid suggestion.
  Outdated suggestions cannot be applied.
  This suggestion has been applied or marked resolved.
  Suggestions cannot be applied from pending reviews.
  Suggestions cannot be applied on multi-line comments.
  Suggestions cannot be applied while the pull request is queued to merge.
  Suggestion cannot be applied right now. Please check back later.
  
    
  
    
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Magic-byte update is only half-done – networking will mis-detect Regtest
You updated
Network::magic()to0xDCB7C1FC, but:Network::from_magic()(line 39) still returns the old value (0xDAB5BFFA).Don’t forget to adjust the corresponding expectations in
tests::test_network_magicandtests::test_network_from_magic; otherwise the crate will not compile.📝 Committable suggestion
🤖 Prompt for AI Agents