Skip to content

Sanitize provenance and support authenticated source URIs - #10

Merged
CraigMyles merged 1 commit into
mainfrom
agent/sanitize-provenance-and-authenticated-uris
Jul 13, 2026
Merged

Sanitize provenance and support authenticated source URIs#10
CraigMyles merged 1 commit into
mainfrom
agent/sanitize-provenance-and-authenticated-uris

Conversation

@CraigMyles

Copy link
Copy Markdown
Owner

Summary

  • Keep the full source URI only for reads; use a credential-free canonical URI in store metadata, receipts, output identity, info output, and captured command provenance.
  • Redact secret CLI option values and authenticated URLs from success and failure paths, including non-FileNotFoundError CLI errors.
  • Support query-authenticated HTTP(S) OME-Zarr stores by attaching the original raw query to every metadata and chunk request.
  • Keep remote manifest sharding stable when credentials rotate, while preserving the existing local-path and file:// behaviour.

Compatibility

Unauthenticated remote URIs remain byte-for-byte unchanged. For authenticated remote URIs, semantic selectors (for example version, generation, series, and fragments) remain part of identity, but credentials do not.

Query credentials must authorize the Zarr prefix and its child objects. A presigned URL for a single S3/GCS object will not generally authorize an entire Zarr hierarchy.

Validation

  • ruff check .
  • focused provenance/URI/OME-Zarr regressions: 69 passed
  • full non-slow, non-network Spark suite: 392 passed, 87 deselected
  • uv lock --check
  • loopback HTTP integration requiring the exact raw query on metadata and chunk requests

Regenerating uv.lock also reconciled pre-existing optional-extra metadata drift.

Refs #6

@CraigMyles
CraigMyles marked this pull request as ready for review July 13, 2026 15:11
@CraigMyles
CraigMyles merged commit 8d9f40d into main Jul 13, 2026
5 checks passed
@CraigMyles
CraigMyles deleted the agent/sanitize-provenance-and-authenticated-uris branch July 13, 2026 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant