fix: stop Satoshi historical scan log OOM - #604
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses BTC mainnet historical catch-up OOMs in the Satoshi mediator by removing per-transaction logging from the block scan hot path and improving pruned-node behavior by short-circuiting to a configured fallback RPC for blocks known to be pruned on the primary node.
Changes:
- Remove per-transaction
console.log(height, index, txid)during historical scans to prevent stdout buffering from causing V8 heap growth. - Capture the primary node prune height at startup.
- When a fallback RPC is configured and a block is at/below the primary prune height, fetch the raw block hex directly from fallback (avoiding repeated expected primary failures).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the remaining BTC mainnet historical catch-up OOM after PR #595 by removing per-transaction stdout logging from the scan hot path and avoiding repeated primary RPC failures for blocks known to be below the primary node's prune height.
Root Cause
The raw-block parser itself stays small, but the mediator still logged every transaction while scanning historical blocks. During catch-up, stdout could not flush Docker log lines as quickly as the mediator produced them, so V8 retained a huge backlog of log strings until the process hit the heap limit and exited with code 139.
The JSON DB showed the scan had advanced beyond the last flushed tx log line, confirming the process was still scanning while stdout buffering lagged behind.
Changes
console.log(height, index, txid)fromfetchBlock().Validation
npm run buildinservices/mediators/satoshinpx eslint services/mediators/satoshi/src/satoshi-mediator.ts services/mediators/satoshi/src/config.tsgit diff --checkbtc-mainnet-mediatorlocally.940211to941163+OOMKilled=false