Skip to content

evalhub 0.1.4

Choose a tag to compare

@github-actions github-actions released this 25 Sep 01:16
· 28 commits to main since this release
c262373

Relations: a reader or writer learns nothing of a version it may not see (0.1.4) (#26)

For #25. Releases 0.1.4.

evalhub_schema::eval promises that when a public Card points at a private Eval, a reader without access learns that the Eval exists, its version_id and its content_hash, and nothing else. The hub kept that promise only in GET …/relations. Three other paths gave the Eval away:

  1. Reads returned the Card body verbatim. GET /cards/… and POST /cards/query returned relations[].to, which is the private ns/name@seq as text.
  2. Query conditions matched every edge. A relations.to condition (exact or prefix) was evaluated against targets the caller could not see, so a caller could confirm a private name by whether a public Card was hit.
  3. Writes resolved targets regardless of visibility. Posting a Card or POST …/relations that named someone else's private version resolved it, so the response and the refs_resolved badge told the writer that the version existed.

Export was not a path: hf-model-index is rendered from results[] and carries no relation.

What changes

  • Reads withhold what the reader may not see. Every read that returns a body removes the relations[] elements whose target the reader may not see, and lists them in the envelope as the commitment:

    "withheld": { "relations": [ { "type": "core/uses_eval", "version_id": "…", "content_hash": "…" } ] }

    A body with elements withheld is not the stored body, so its content_hash does not match it. Verify the hash only on responses that have no withheld. A member of the Eval's namespace gets the body unchanged and no withheld.

  • Query conditions see only visible targets. A relations.to condition matches only edges whose target the caller may see. Negated, the hidden edge is just as absent.

  • A write learns no more than a read. At ingest, in POST …/relations and in the deferred resolver, a target the writer may not see is handled exactly like one that does not exist: it is stored unresolved, the response is the same and no refs_resolved badge is given. Once the target is visible to a reader, the edge resolves for them. The writer's own namespace and their organisations' namespaces are visible, so legitimate private references resolve as before.

  • One visibility predicate (evalhub_store::relations::visible_to) serves the relation DTO, body redaction and resolution.

Contract

  • withheld is new and optional on the version envelope and on query hits. The OpenAPI snapshot and the web/ client are regenerated.
  • The SDK crates' public API is unchanged: just semver-check reports no semver update required for evalhub-schema, -core and -query. The schema change is limited to the crate doc (eval § "Privacy across the relation").
  • .sqlx is regenerated for the changed queries.

Verification

  • crates/evalhub-server/tests/privacy.rs: five HTTP tests covering the body on GET, the body on query, the relations.to condition, the badge on POST, and POST …/relations. All five were first run against main and failed on the defect itself: alice/secret@1 appeared in the body, bob's condition hit, the badge differed from that of a nonexistent target, and the relation response revealed the target.
  • Store tests: relations_resolve_only_to_versions_the_writer_may_see, resolution_never_reveals_a_private_version and relation_filters_see_only_targets_the_caller_may_see were checked against this commit with the three visibility checks disabled, and each failed on the leak.
  • cargo fmt --check; cargo clippy --workspace --all-targets -- -D warnings; cargo test -p evalhub-schema / -core / -query / -store / -server (Docker); just sqlx-check; pnpm check; just e2e (18 passed); just package + package-gate; just semver-check.

Test image: MinIO from Chainguard

The first CI run of this PR failed in evalhub-store tests/objects.rs before any test code ran: quay.io now answers 401 to anonymous pulls of quay.io/minio/minio (and minio/mc), which passed on main the day before. The store and server test harnesses and compose.yml now use cgr.dev/chainguard/minio, pinned by digest (it publishes latest only). It ships mc and a shell, so the bucket setup is unchanged. The tests mount a tmpfs on /data and wait for the API: http banner, because on the overlay root MinIO first logs a rename error that also starts with API:.


SDK crates: evalhub-schema evalhub-core evalhub-query
Image: ghcr.io/ynishi/evalhub:0.1.4