Skip to content

v0.2.0 – Unified Landing/Serving Schema and Read APIs

Choose a tag to compare

View all tags
@hsballoon hsballoon released this 31 Jul 15:08
· 98 commits to master since this release
v0.2.0
0fa575d

Highlights

  • Unified the landing and serving schemas with tags, search_text, chosen_trace, and rejected_trace.
  • Added Arrow JSON support for meta_json while preserving JSON-string compatibility in the Python SDK.
  • Standardized both tables on 128-bucket HASH(job_id) partitioning with workload-specific indexes.
  • Replaced query_landing() with the table-aware query_data(), which always returns List[dict] and excludes null fields by default.
  • Replaced fetch_data() with iter_data_batches() and added table selection to the primary read APIs.
  • Added export_data_batches() for reliable offline exports using a fixed ID manifest and per-batch validation.
  • Preserved the existing landing defaults and DataFrame behavior of pull_data() for SAfactory compatibility.
  • Updated operational scripts, documentation, and unit/integration coverage for the new schemas and APIs.

Breaking Changes

  • chosen_response and rejected_response have been replaced by chosen_trace and rejected_trace.
  • query_landing() has been replaced by query_data().
  • fetch_data() has been replaced by iter_data_batches().
  • Existing landing and serving tables must use the new schema and HASH(job_id) partition layout.
Assets 2
Loading