-
Notifications
You must be signed in to change notification settings - Fork 94
Description
Problem
When pulling a model from Hugging Face, we build the OCI artifact locally (since Hugging Face is not an OCI registry). The creation date in the OCI config is set to time.Now(), which means the same model pulled at different points in time produces a different config JSON and therefore a different SHA digest.
This breaks content-addressability: the same logical model (same weights, same config files, same revision) should always produce the same OCI digest regardless of when it was pulled.
Affected code paths:
-
pkg/distribution/builder/builder.go—fromFormat()— used when building fromFromPaths(the Hugging Face pull path viabuildModelFromFiles)created := time.Now() // ← non-deterministic
-
pkg/distribution/builder/from_directory.go—FromDirectory()— same patterncreated := time.Now() // ← non-deterministic
Both set types.Descriptor.Created to the current wall-clock time, which is embedded in the OCI config JSON and influences the overall manifest digest.
Impact
- Pulling the same HF model twice yields different digests, making it appear as two distinct models in the local store.
- Digest-based deduplication and caching cannot work correctly.
- Users cannot reliably pin or verify a model by its digest.
Proposed solutions
Use the Hugging Face last commit date for the revision__ Query the HuggingFace API (e.g., GET /api/models/{repo} which returns lastModified, or the commit-specific endpoint GET /api/models/{repo}/revision/{revision}) and use the commit timestamp as the Created date. This is semantically correct—the model was "created" when it was last committed—and is deterministic for a given revision.
Additional context
The RepoFile struct in pkg/distribution/huggingface/repository.go does not currently carry any timestamp from Hugging Face. We would need to either:
- Fetch the repo/revision metadata (a single API call) to get the commit date, or
- Propagate a
createdAtparameter throughBuildModel→buildModelFromFiles→builder.FromPaths/fromFormat.