Skip to content

Fetch Backfill Data Properly

Choose a tag to compare

@shlomiku shlomiku released this 22 Jul 16:00
· 124 commits to master since this release
f0a3c9a

When running live we:

  • Get historic data (through the rest api)
  • Get live data (through the websocket)

This release fixes issues with fetching of the historic data:

  1. Date range is too large
    • Polygon api doesn't indicate a limit for 1 api call but you can't get six months of data in 1 api call (makes since).
      Data is returned corrupted (patchy data)
    • Alpaca has a limit of 1000 records per api call
  2. Compression - or not 1 minute/daily bars (e.g 5 minute, 30 minute, 60 minute bars)
    • Polygon supports it. alpaca doesn't
  3. Out of market data
    • Both apis return out of market hours data. that is of course not what we want when we do these calls