Skip to content

Conversation

@chris-4chain
Copy link
Collaborator

Description of Changes

Provide a brief description of the changes you've made.

Linked Issues / Tickets

Reference any related issues or tickets, e.g. "Closes #123".

Testing Procedure

Describe the tests you've added or any testing steps you've taken.

  • I have added new unit tests
  • All tests pass locally
  • I have tested manually in my local environment

Checklist:

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have run the linter

@chris-4chain chris-4chain requested a review from Copilot November 17, 2025 12:45
@github-actions
Copy link
Contributor

Manual Tests

ℹ️ Remember to ask team members to perform manual tests and to assign tested label after testing.

Copilot finished reviewing on behalf of chris-4chain November 17, 2025 12:48
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a new bulk ingestor for WhatsOnChain (WoC) and refactors the existing WoC polling ingestor to use a shared HTTP client abstraction. The changes enable fetching bulk block header files from WhatsOnChain's bulk endpoints while maintaining backward compatibility with the polling-based approach.

Key changes include:

  • Addition of a new BulkIngestorWOC implementation that fetches bulk header files from WhatsOnChain
  • Refactoring of WoC-related HTTP calls into a shared wocClient for better code reuse
  • Migration from config-based initialization to an options builder pattern for both poll and bulk ingestors
  • Interface signature update to BulkIngestor.Synchronize() to accept a single HeightRange instead of HeightRanges

Reviewed Changes

Copilot reviewed 17 out of 19 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
pkg/services/chaintracks/models/height_range.go Adds Overlaps method to check range overlap, supporting bulk file filtering
pkg/services/chaintracks/ingest/woc_dto.go Adds prevChainWorkForGenesis constant and WOCHeadersResourcesDTO for bulk endpoint responses
pkg/services/chaintracks/ingest/live_ingestor_woc_poll_test.go Updates tests to use new options builder pattern instead of config structs
pkg/services/chaintracks/ingest/live_ingestor_woc_poll_options.go Introduces options builder pattern with IngestorWocPollOpts for flexible configuration
pkg/services/chaintracks/ingest/live_ingestor_woc_poll.go Refactors to use shared wocClient instead of direct HTTP calls
pkg/services/chaintracks/ingest/chaintraks_woc_client.go New shared HTTP client for WhatsOnChain API interactions across live and bulk ingestors
pkg/services/chaintracks/ingest/cdn_reader_test.go Comprehensive tests for CDN reader functionality with error scenarios
pkg/services/chaintracks/ingest/cdn_reader.go Implements CDN reader for fetching bulk header files from Project Babbage CDN
pkg/services/chaintracks/ingest/cdn_data.go Adds ToHeightRange method to convert bulk file metadata to height ranges
pkg/services/chaintracks/ingest/bulk_ingestor_woc_test.go Adds placeholder test (currently skipped) for WoC bulk ingestor
pkg/services/chaintracks/ingest/bulk_ingestor_woc_options.go Options builder pattern for configuring bulk WoC ingestor
pkg/services/chaintracks/ingest/bulk_ingestor_woc.go Core implementation of bulk ingestor fetching header files from WhatsOnChain
pkg/services/chaintracks/ingest/bulk_ingestor_cdn.go Updates signature to accept single HeightRange parameter
pkg/services/chaintracks/chaintracks_storage.interface.go Updates BulkIngestor interface to use HeightRange instead of HeightRanges
pkg/services/chaintracks/chaintracks_service_test.go Updates service test to use new options pattern
pkg/services/chaintracks/chaintracks_initializers.go Updates factory to pass API key using options builder
pkg/services/chaintracks/bulk_manager.go Implements logic to track missing ranges and iteratively fetch from multiple ingestors
pkg/defs/woc_poll_ingestor.go Removes deprecated WOCPollIngestorConfig struct (replaced by options pattern)
pkg/defs/chaintracks.go Adds WocAPIKey field to main chaintracks config

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@sonarqubecloud
Copy link

@chris-4chain chris-4chain merged commit 24b104e into main Nov 17, 2025
10 checks passed
@chris-4chain chris-4chain deleted the feat/bulk_ingestor_woc branch November 17, 2025 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] internalizeAction MVP on sql storage

4 participants