Skip to content

Fix missing state/country in articles after 2025-02-03#20

Open
AlexCatarino wants to merge 2 commits intoQuantConnect:masterfrom
AlexCatarino:fix-missing-state-country
Open

Fix missing state/country in articles after 2025-02-03#20
AlexCatarino wants to merge 2 commits intoQuantConnect:masterfrom
AlexCatarino:fix-missing-state-country

Conversation

@AlexCatarino
Copy link
Copy Markdown
Member

@AlexCatarino AlexCatarino commented Apr 17, 2026

Summary

  • Fixes Missing state of recent articles #18
  • As of 2025-02-03, the Regalytics agencies field renamed state -> states and country -> countries. process.py only read the old keys, so articles processed after that date had empty states.
  • Reads the new keys with fallback to the legacy ones so older articles still process correctly.

Test plan

  • Reprocess a date after 2025-02-03 and confirm article['states'] is populated.
  • Reprocess a date before 2025-02-03 and confirm behavior is unchanged.

As of 2025-02-03, the Regalytics 'agencies' field uses 'states' and 'countries' instead of 'state' and 'country'. Read the new keys with fallback to the legacy ones so older articles keep working.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@AlexCatarino AlexCatarino force-pushed the fix-missing-state-country branch from ea4f760 to 3d240bf Compare April 17, 2026 17:41
Some article 'created_at' values omit microseconds, e.g. '2025-06-17T00:00:00-04:00'. Pick the strptime format based on whether a '.' is present so both shapes parse.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Missing state of recent articles

2 participants