Skip to content

perf: replace O(n²) concat with push in metaculus fetchMarkets#571

Merged
realfishsam merged 1 commit into
mainfrom
fix/551-metaculus-fetchmarkets-perf
May 24, 2026
Merged

perf: replace O(n²) concat with push in metaculus fetchMarkets#571
realfishsam merged 1 commit into
mainfrom
fix/551-metaculus-fetchmarkets-perf

Conversation

@realfishsam
Copy link
Copy Markdown
Contributor

Fixes #551

@realfishsam
Copy link
Copy Markdown
Contributor Author

PR Review: PASS (NOT VERIFIED)

What This Does

Replaces all = all.concat(results) with all.push(...results) in the metaculus fetchMarkets.ts pagination loop, avoiding O(n^2) array re-allocation on each page.

Blast Radius

  • Core: core/src/exchanges/metaculus/fetchMarkets.ts only
  • No OpenAPI, SDK, or type changes

Findings

  1. The all variable is declared with let (line 45) but is no longer reassigned after this change. It could be tightened to const for clarity, though this is non-blocking.
  2. With push(...results) and BATCH_SIZE = 100, each spread is well under the V8 call-stack limit (~65k args). No risk of "Maximum call stack size exceeded" at this batch size.
  3. Correct behavior preserved: all.length continues to reflect cumulative results for the early-exit check on line 64.

PMXT Pipeline Check

  • Field propagation: N/A
  • OpenAPI sync: N/A
  • Type safety: OK

Semver Impact

patch -- internal perf improvement, no API or behavioral change

Risk

Not verified against live Metaculus data. The change is mechanically safe but untested in this review.

@realfishsam realfishsam merged commit 3cd3f1f into main May 24, 2026
11 of 12 checks passed
@realfishsam realfishsam deleted the fix/551-metaculus-fetchmarkets-perf branch May 24, 2026 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unbounded: core/src/exchanges/metaculus/fetchMarkets.ts — O(n²) array.concat() with MAX_PAGES=200

1 participant