fix(schemas): backfill gemini catalog identity evidence for schema drift

[Sinity]

Summary

Backfills real exact_structure_ids/profile_tokens identity evidence onto the committed gemini v1/v2 session_document schema packages, so live aistudio-drive ingest stops classifying every record as unseen_shape schema drift.

Problem

Live ops.db schema_drift_samples showed 100% of 302 aistudio-drive records ingested since 2026-07-01 classifying as unseen_shape (element_kind=session_document). Inspecting real cached payloads at ~/.local/share/polylogue/drive-cache/gemini/*.json confirmed the actual wire shape is a bare {chunkedPrompt, runSettings, systemInstruction} document (occasionally with top-level applets/citations), with no id/title/createTime — drive.py's parser already handles this correctly via fallback logic, so parsing itself was never affected.

The committed gemini schema catalog (polylogue/schemas/providers/gemini/{v1,v2}) already modeled this exact shape structurally (verified by decompressing and diffing the committed session_document.schema.json.gz files against the real cached payloads — same top-level properties, same runSettings/chunkedPrompt.chunks substructure). The actual defect was one level deeper: both packages' session_document element carried empty exact_structure_ids and profile_tokens lists in catalog.json/package.json/the schema's own x-polylogue-* annotations. SchemaRegistry.resolve_payload (polylogue/schemas/runtime_registry.py) only escapes its package_default fallback (which classify_schema_drift reports as unseen_shape) via an exact_structure, bundle_scope, or profile_family match — with both identity lists empty, no live payload could ever match, regardless of how well the schema's properties actually described reality.

Solution

Ran the real schema-generation pipeline (polylogue.schemas.generation.workflow._build_provider_bundle / _collect_cluster_accumulators) against a structurally faithful synthetic gemini corpus, built with the repo's own CorpusSpec.for_provider("gemini", ...) + SyntheticCorpus.write_spec_artifacts + parse_sources_archive test infrastructure (the same real ingestion path tests/infra/workload_artifacts.py's build_seeded_archive uses) — no real/private conversation content anywhere in this corpus. This produced genuinely-computed exact_structure_ids (structural fingerprint hashes) and profile_tokens (coarser shape tokens like field:chunkedPrompt, child:runSettings:model) for the shape.

Rather than running the full destructive commit_provider_schema/generate_all_schemas replace-all pipeline (which deletes and rewrites the entire versions/ tree from only the versions the fresh generation run produces — here just one version, which would have silently destroyed the existing, richer v2 package), I spliced just the two missing identity fields into the existing v1 and v2 schema documents via SchemaRegistry.write_schema_version (the real production single-version write path), leaving all existing schema structure, sample_count/artifact_count provenance, and other x-polylogue-* annotations untouched. Diffed the decompressed schema documents before/after to confirm properties is byte-identical; only the two new identity fields and a x-polylogue-registered-at timestamp bump changed. v1 additionally had a stale $id/x-polylogue-element-kind of conversation_document, inconsistent with its own catalog manifest's session_document — write_schema_version corrected this as a side effect of using the real write path.

profile_tokens is the durable part of the fix: profile_family matching (profile_similarity) tolerates new/missing optional fields, so it keeps resolving future real traffic even as message content and optional attachment fields (driveDocument, driveVideo, etc.) vary. exact_structure_ids is a bonus fast-path that will only hit incidentally, since it fingerprints the full document structure recursively and real conversations vary too much for exact hash reuse.

Added tests/unit/schemas/test_gemini_catalog_identity.py, which exercises the default (bundled, no storage_root override) SchemaRegistry — the same one polylogue.pipeline.services.ingest_worker._runtime_schema_registry() resolves payloads through in production — against payloads shaped like the real cached exports, with message content that never appeared in any fixture/corpus, proving the match generalizes via profile-family evidence rather than exact-content coincidence.

Verification

• devtools test tests/unit/schemas/test_gemini_catalog_identity.py — 4 passed. Confirmed red-before via git stash (all 4 fail with package_default/unseen_shape against the pre-fix committed schema), green after popping the stash.
• devtools test tests/unit/schemas/ tests/integration/test_schema_operator_workflow.py — 104 passed.
• devtools test tests/unit/sources/test_gemini_drive_normalization_laws.py — 9 passed (existing gemini/drive normalization coverage unaffected).
• python3 -m devtools lab schema audit --provider gemini — 4 pass, 1 warn (pre-existing schema_staleness warning based on x-polylogue-generated-at, untouched by this change), 0 fail.
• devtools verify --quick — 22/22 steps passed (exit_code: 0), no out of sync in render all output.

Not run: a live end-to-end check against production ops.db — no write access to the live archive from this lane. The schema-catalog classification function itself is exercised directly instead (see test file), which is the mechanism the bead's own scope asked to fix.

Ref polylogue-tu1f

[coderabbitai]

Warning

Review limit reached

@Sinity, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 8 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: f21ee8cc-bc91-423c-9564-192db28a0021

📥 Commits

Reviewing files that changed from the base of the PR and between 991b3bc and 617579b.

⛔ Files ignored due to path filters (2)

• polylogue/schemas/providers/gemini/versions/v1/elements/session_document.schema.json.gz is excluded by !**/*.gz, !**/*.json.gz
• polylogue/schemas/providers/gemini/versions/v2/elements/session_document.schema.json.gz is excluded by !**/*.gz, !**/*.json.gz

📒 Files selected for processing (4)

• polylogue/schemas/providers/gemini/catalog.json
• polylogue/schemas/providers/gemini/versions/v1/package.json
• polylogue/schemas/providers/gemini/versions/v2/package.json
• tests/unit/schemas/test_gemini_catalog_identity.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}