feat(pj_base): attach_source_record runtime-host slot for the M3 source cache (0.28.0) - #189
Merged
Merged
Conversation
One provider->host call, stated once at download start: the canonical descriptor JSON of the reproducible request a source answers. The host copies the bytes and derives the record identity itself (the plugin never supplies an identity, per the descriptor_import doctrine), enabling the host-driven transparent source cache. Byte-identical re-attach is idempotent; different bytes fail; failure is a contract failure, never a trust verdict, and never affects ingest. Old hosts read as no-caching via PJ_HAS_TAIL_SLOT; the C++ wrapper reports the absence explicitly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The record is stored verbatim and keyed on the host's own digest (an internal keyspace — never required to agree with a provider identity scheme); the unspoofable datum is the provider id, taken from the binding. Refusal is allowlist-semantics (unknown fields are an error), not a field denylist. Last attach before the first push_message wins and the host may stage the record until its ingest transaction commits, so a replacing reload or in-place refill never silently loses it. Thread tag matches push_message ([stream-thread]). DatasetIngestHostView gains the forward; guide/architecture docs list the slot; tests add the unbound-host and short-struct_size-with-stale-pointer gates. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the SDK interface for the host-managed source cache: a provider calls
attachSourceRecord(descriptor_json)at download start to identify the request being ingested. The host owns capture, caching, and restore; this PR adds the declaration slot and C++ wrappers.The optional
attach_source_recordtail slot is exposed through bothDataSourceRuntimeHostViewandDatasetIngestHostView.push_messagewins. Byte-identical repeats before ingestion are idempotent; any attachment after ingestion begins is an error.PJ_HAS_TAIL_SLOTgates older hosts, with an explicit wrapper error indicating that attachment is unavailable.SDK version: 0.27.1 → 0.28.0. The runtime-host vtable grows from 104 to 112 bytes, with the new slot at offset 104; the ABI baseline is unchanged. Hosts using designated initializers may need an explicit
.attach_source_record = nullptrwhen adopting the new headers.Also includes the provider-job watchdog startup synchronization fix. The changelog, C++ documentation, guides, and test error example now agree with the ABI contract.
Validation: 85/85 SDK tests passed in the reviewed RelWithDebInfo build and in Debug+ASAN after the documentation changes. Pre-commit checks passed.