Skip to content

Releases: Alberto-Codes/inference-lab

v0.2.0: RTX 4090 receipt scaling

Pre-release

Choose a tag to compare

@github-actions github-actions released this 28 Oct 03:45
v0.2.0
3193fb2

v0.2.0: RTX 4090 receipt scaling

The experiment tag is immutable.

This experiment extends the receipt pilot with a manifest-driven open-loop runner,
bounded concurrency, durable checkpoints and offline scoring.
It preserves 9,228 measured outcomes and 955 separate warmup outcomes.
Coverage contains 56 CORD receipts; 20 evaluation receipts remain untouched.
The published v0.1.0 baseline is unchanged.

Five offered pages/s kept queueing low for five minutes.
Six offered pages/s grew a queue and rejected work.
Each long condition ran once. These results do not establish zero-failure capacity.
At five pages/s, 26 requests timed out on the same receipt.
Verified digit changes and unrelated generation remain quality failures.

Reproduce the matrix after preparing the pinned corpus:

uv run python scripts/scaling_suite.py \
  --input corpus/cord-scaling-56 --output runs/scaling-4090-matrix \
  --warmup-image corpus/page-01.png --release-llama

The service handover requires explicit operator authorization.
Follow the procedure for preparation, longer trials,
resume and offline rescoring of the retained raw evidence.
See the results and
failure report for limits and the next experiment.

Companion post: I sent my 4090 six receipts a second. The queue kept growing.

Experiment commit: 3193fb28d46e875d4f39324d4ba743f058fb872c.

v0.1.0: Nemotron Parse receipt pilot

Pre-release

Choose a tag to compare

@github-actions github-actions released this 27 Oct 05:16

v0.1.0: Nemotron Parse receipt pilot

Experiment snapshot: 1b412ff74cee4b60d07b17d24e41daef4df18335.

This experiment runs Nemotron-3.5-Parse through vLLM on a local RTX 4090.
It preserves 24 CORD receipt inputs' provenance and 48 measured responses.
All measured requests completed. One amount changed from 79,500 to 79,300.
Diagnostic ROUGE-L F1 was 0.801; exact numeric-string recall was 212/241.
The evaluation does not measure layout or table-structure accuracy.

Recalculate the saved experiment without a GPU:

uv sync --locked
uv run python scripts/score_receipts.py \
  --input evidence/cord-test-24 --output runs/tag-inspection/scores.json

Companion post: I ran Nemotron Parse on a 4090. Then I checked the numbers.

Measured results · Reproduction

Experiment commit: 1b412ff74cee4b60d07b17d24e41daef4df18335.