feat: add weights build and push commands

[markphelps]

Summary

This PR adds (hidden) CLI commands for building and pushing model weights as OCI blobs, building on the OCI bundle infrastructure in md/oci-bundle.

New Commands

• cog weights build - Generates a weights.lock file from weight sources defined in cog.yaml
• cog weights push - Pushes weight files as OCI blobs to a registry with progress tracking

Key Changes

Configuration (pkg/config/)

• Added WeightSource type with source and optional target fields
• Added Weights field to Config struct
• Updated JSON schema for cog.yaml validation

CLI (pkg/cli/weights.go)

• cog weights build: Reads weights section from cog.yaml, computes SHA256 digests, generates weights.lock
• cog weights push: Pushes weight blobs concurrently with real-time progress bars (TTY) or summary (non-TTY)
• Progress tracking with progressTracker for multi-file concurrent uploads

Model Layer (pkg/model/)

• WeightsLockGenerator: Processes weight sources (files or directories), computes digests by streaming
• WeightsPusher: Concurrent blob push with progress callbacks
• fileBackedLayer: Implements v1.Layer interface for memory-efficient streaming from disk
• Updated index_factory.go to use uncompressed layers (weights don't compress well)

Registry (pkg/registry/)

• Added WriteLayer and WriteLayerWithProgress methods to Client interface
• Implemented in RegistryClient using go-containerregistry

Design Decisions

1. No compression: Model weights (safetensors, GGUF, etc.) are already highly optimized binary data that doesn't compress well. Skipping gzip saves CPU time and simplifies the code.

2. Streaming: Files are never loaded entirely into memory. The fileBackedLayer streams directly from disk, and hashFile() computes digests via io.Copy.

3. Concurrent uploads: Weight files are pushed in parallel with progress tracking per file.

4. Lock file: weights.lock pins exact digests and sizes, similar to package manager lock files.

Example Usage

# cog.yaml
weights:
  - source: weights/model.safetensors
  - source: weights/tokenizer.json
    target: /custom/path/tokenizer.json

# Generate weights.lock
$ cog weights build
Processing 2 weight source(s)...
  model -> /cache/weights/model.safetensors (4.2GB)
  tokenizer -> /custom/path/tokenizer.json (1.2MB)

Generated weights.lock with 2 file(s) (4.2GB total)

# Push to registry
$ cog weights push registry.example.com/user/model
Pushing 2 weight file(s) to registry.example.com/user/model...
  model.safetensors  [====================] 100.0%  4.2GB / 4.2GB
  tokenizer.json     [====================] 100.0%  1.2MB / 1.2MB

Pushed 2 weight blobs to registry.example.com/user/model
Total: 4.2GB

Testing

• Unit tests for WeightsLockGenerator, WeightsPusher, IndexFactory
• Integration test: weights_build.txtar
• All existing tests pass

Example w/ Weights Gen Tool (renamed)

cog/test-weights >> weights-gen --count 5 --min-size 100mb --max-size 500mb --output-dir weights
Generating 5 random weight files (100.0MB - 500.0MB each)...
  Creating weights-001.bin (390.3MB)...
  Processed: weights-001 -> /cache/weights-001.bin
  Creating weights-002.bin (378.3MB)...
  Processed: weights-002 -> /cache/weights-002.bin
  Creating weights-003.bin (297.6MB)...
  Processed: weights-003 -> /cache/weights-003.bin
  Creating weights-004.bin (331.3MB)...
  Processed: weights-004 -> /cache/weights-004.bin
  Creating weights-005.bin (136.0MB)...
  Processed: weights-005 -> /cache/weights-005.bin

Generated weights.lock with 5 files
Weight files written to: weights

----

─────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
     │ File: weights.lock
─────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ {
   2 │   "version": "1.0",
   3 │   "created": "2026-02-04T20:04:17.840556Z",
   4 │   "files": [
   5 │     {
   6 │       "name": "weights-001",
   7 │       "dest": "/cache/weights/weights-001.bin",
   8 │       "digestOriginal": "sha256:5c771284a57297e899617478e103df34c2f280c42b66d32eb7e90838704459d3",
   9 │       "digest": "sha256:5c771284a57297e899617478e103df34c2f280c42b66d32eb7e90838704459d3",
  10 │       "size": 409238039,
  11 │       "sizeUncompressed": 409238039,
  12 │       "mediaType": "application/vnd.cog.weights.layer.v1"
  13 │     },
  14 │     {
  15 │       "name": "weights-002",
  16 │       "dest": "/cache/weights/weights-002.bin",
  17 │       "digestOriginal": "sha256:a1fdcc62da7517dc995fcb307e3140818935695cb11c3abdb8d35ed3680bbf1c",
  18 │       "digest": "sha256:a1fdcc62da7517dc995fcb307e3140818935695cb11c3abdb8d35ed3680bbf1c",
  19 │       "size": 396717228,
  20 │       "sizeUncompressed": 396717228,
  21 │       "mediaType": "application/vnd.cog.weights.layer.v1"
  22 │     },
  23 │     {
  24 │       "name": "weights-003",
  25 │       "dest": "/cache/weights/weights-003.bin",
  26 │       "digestOriginal": "sha256:705fb7e019b7ab1396d8eb62c026226446c75b873587b12d6ba890dc77012566",
  27 │       "digest": "sha256:705fb7e019b7ab1396d8eb62c026226446c75b873587b12d6ba890dc77012566",
  28 │       "size": 312063760,
  29 │       "sizeUncompressed": 312063760,
  30 │       "mediaType": "application/vnd.cog.weights.layer.v1"
  31 │     },
  32 │     {
  33 │       "name": "weights-004",
  34 │       "dest": "/cache/weights/weights-004.bin",
  35 │       "digestOriginal": "sha256:3e00492868066d45932f3ea4414f5a140315c56a575d977f990de37e9b30cce7",
  36 │       "digest": "sha256:3e00492868066d45932f3ea4414f5a140315c56a575d977f990de37e9b30cce7",
  37 │       "size": 347363237,
  38 │       "sizeUncompressed": 347363237,
  39 │       "mediaType": "application/vnd.cog.weights.layer.v1"
  40 │     },
  41 │     {
  42 │       "name": "weights-005",
  43 │       "dest": "/cache/weights/weights-005.bin",
  44 │       "digestOriginal": "sha256:a57d95c757b8ce41c649d4feb7284b9a0e59784e6dac3cf89951902947872e8b",
  45 │       "digest": "sha256:a57d95c757b8ce41c649d4feb7284b9a0e59784e6dac3cf89951902947872e8b",
  46 │       "size": 142604752,
  47 │       "sizeUncompressed": 142604752,
  48 │       "mediaType": "application/vnd.cog.weights.layer.v1"
  49 │     }
  50 │   ]
  51 │ }
─────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

----

cog/test-weights » cog weights push
Pushing 5 weight file(s) to {internal-registry}/cog-examples/hello-world...
  weights-002                     [=-------------------]   5.4%  20.3MB / 378.4MB
  weights-005                     [--------------------]   4.7%  6.3MB / 136.0MB

TODO

• Handle retries on layer push failures with backoff
• (Maybe) smarter concurrency on pushes?
• Add more ITs

[michaeldwan]

awesome