v0.4.0 – SDK-Managed Timestamps and Serving Upsert

Highlights

• Added source_updated_at and serving_updated_at as Unix epoch millisecond fields in the unified landing/serving schema.
• Automatically initializes source timestamps and refreshes them on landing updates without changing existing caller code.
• Added native upsert_serving() and upsert_serving_batch() APIs using id as the business key.
• Serving writes now preserve source_updated_at, refresh serving_updated_at, and never mutate caller-owned models.
• Added BTREE index definitions for the new timestamp fields.
• Expanded unit and real S3 integration coverage for timestamps, JSON payloads, landing updates, serving ingestion, and upsert behavior.
• Synchronized package metadata and runtime version reporting to 0.4.0.

Compatibility

Existing ingest_landing(), ingest_landing_batch(), query_data(), pull_data(), and update_landing() calls remain source-compatible.

update_landing() now refreshes source_updated_at by default. Pure operational updates may opt out with:

touch_source_updated_at=False

Serving upserts require a non-empty and immutable job_id. Callers must guarantee globally unique IDs because dldb HASH tables do not enforce uniqueness across buckets.

Migration

This release changes the physical LanceDB schema. Existing landing and serving tables must be rebuilt or migrated before upgrading writers to v0.4.0.

Downstream applications should update their dependency pin to:

wt-data-platform-sdk @ git+https://github.com/AI45Lab/wt-data-platform-sdk.git@v0.4.0

Validation

• 67 hermetic unit tests passed.
• 6 real S3 integration tests passed.
• Active landing and serving tables were rebuilt and verified with the new schema.
• Integration test data was cleaned successfully.
• Legacy archive tables were left unchanged.