🗺️ Performance optimization for large portfolios (10,000+ trades)

[GeiserX]

Context

Active traders or users with many years of history may have 10,000+ trades in their broker exports. The current implementation processes everything synchronously in the main thread, which can cause the UI to freeze during:

1. File parsing (XML/CSV/XLSX)
2. FIFO engine lot matching
3. ECB rate fetching (multiple years)
4. Results rendering (large tables)

Current bottlenecks (estimated)

Operation	1,000 trades	10,000 trades	50,000 trades
IBKR XML parse	~100ms	~800ms	~4s
FIFO processing	~50ms	~500ms	~3s
ECB rate fetch	~200ms	~200ms	~200ms
DOM rendering	~100ms	~1s	~5s+

Total for 50K trades: potentially 12+ seconds of UI freeze.

Proposed optimizations

1. Web Worker for FIFO engine

Move the heavy computation off the main thread:

// fifo.worker.ts
self.onmessage = (e) => {
  const { trades, rates, year } = e.data;
  const engine = new FIFOEngine();
  const disposals = engine.processTrades(trades, rates);
  self.postMessage({ disposals, warnings: engine.warnings });
};

Main thread stays responsive, shows progress indicator.

2. Virtual scrolling for results tables

With 5,000+ disposals, rendering all DOM nodes is wasteful. Use virtual scrolling:

• Only render visible rows + buffer (e.g., 50 rows at a time)
• Lightweight implementation (~100 LOC) without external dependencies
• Maintain keyboard navigation and search functionality

3. Incremental parsing with progress

For large XML files, use streaming parser or chunked processing:

// Report progress during parse
for (let i = 0; i < trades.length; i += CHUNK_SIZE) {
  const chunk = trades.slice(i, i + CHUNK_SIZE);
  processChunk(chunk);
  await yieldToMainThread(); // requestAnimationFrame or setTimeout(0)
  reportProgress(i / trades.length);
}

4. ECB rate caching

Already partially implemented via localStorage, but:

• Pre-fetch common currency rates on app load
• Cache aggressively (rates don't change for past dates)
• Batch requests by year to minimize API calls

5. Lazy section rendering

720/D-6/721 sections only computed when navigated to, not on initial process.

6. IndexedDB for large datasets

For users with multi-year history:

• Store parsed trades in IndexedDB (not just localStorage)
• Avoids re-parsing on revisit
• localStorage has 5-10MB limit; IndexedDB is practically unlimited

Metrics to track

• Time from "Process" click to results visible
• First Contentful Paint (FCP) on initial load
• Lighthouse Performance score
• Memory usage with large files

Target

• 10,000 trades: results in < 2 seconds
• 50,000 trades: results in < 5 seconds
• Never freeze UI for more than 100ms

Complexity

Medium — Web Worker is the biggest win and most isolated change. Virtual scrolling requires table refactoring. Both can be done independently.

References

• Web Workers API: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API
• Virtual scrolling pattern: render only visible viewport rows
• requestIdleCallback / scheduler.yield() for cooperative scheduling