Skip to content

v2.0 - Parquet dataset ETL architecture

Choose a tag to compare

View all tags
@ghukill ghukill released this 24 Feb 14:07
· 86 commits to main since this release
v2.0
52cb699
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

This v2.0 release marks a change in TIMDEX ETL to use parquet datasets as the primary storage architecture.

What's Changed

  • Install timdex-dataset-api library from main by @ghukill in #223
  • TIMX 454 - parse BS4 in isolated thread to avoid dangling memory pointers by @ghukill in #231
  • TIMX 406 - add provenance data to transformed records by @ghukill in #233
  • TIMX 459 - update logging by @ghukill in #244

Full Changelog: v1.6.0...v2.0

Contributors

ghukill
Assets 2
Loading