Repository navigation
Releases
Release 0.50.28
Install
Docker
docker pull ghcr.io/jedwards1230/earmark:0.50.28Helm
helm install earmark oci://ghcr.io/jedwards1230/charts/earmark --version 0.50.28earmark v0.50.28 Release Notes
This patch release adds an optional PriorityClass setting for the database pod in the Helm chart and bumps the chart and app version to 0.50.28. With the default empty value, deployments behave as before.
Features
Configurable PriorityClass for the CloudNativePG database pod
The Helm chart now accepts cnpg.priorityClassName, which sets priorityClassName on the CNPG Cluster's pods. Use it to keep the single-instance database from being evicted in favor of less important workloads when the cluster is under scheduling pressure.
To use it, set the value in your values file to the name of an existing PriorityClass:
cnpg:
priorityClassName: "your-priority-class"Notes:
- The default is
"", which leaves the field unset so the cluster default applies. - The PriorityClass must already exist. Admission rejects pods that reference a missing class, so create it before upgrading with this value set.
- Choose a class at or above your application's tier, per the guidance in
values.yaml.
Commits
- chore(earmark): bump chart version to 0.50.28 (ddae6b8)
- feat(chart): add cnpg.priorityClassName for the database pod (#234) (ae53d42)
Full Changelog: v0.50.27...v0.50.28
Release 0.50.27
Install
Docker
docker pull ghcr.io/jedwards1230/earmark:0.50.27Helm
helm install earmark oci://ghcr.io/jedwards1230/charts/earmark --version 0.50.27v0.50.27 Release Notes
This patch release updates the Go toolchain to 1.26.11 to address GO-2026-6859, and bumps the container builder image and Helm chart version to 0.50.27. There are no feature or behavior changes.
Bug Fixes
- Go toolchain update for GO-2026-6859:
go.modnow declaresgo 1.26.11, up from 1.26.10. The fix is referenced in the commit title (#235) as addressing GO-2026-6859.
Improvements
- Container builder image: The Dockerfile builder stage now uses
golang:1.27.4-alpine, up fromgolang:1.27.3-alpine. - Helm chart version: The chart
versionandappVersionare both set to0.50.27, up from0.50.26.
Commits
- chore(earmark): bump chart version to 0.50.27 (b13fb44)
- fix(deps): bump Go to 1.26.11 for GO-2026-6859 (#235) (ea5256e)
Full Changelog: v0.50.26...v0.50.27
Release 0.50.26
Install
Docker
docker pull ghcr.io/jedwards1230/earmark:0.50.26Helm
helm install earmark oci://ghcr.io/jedwards1230/charts/earmark --version 0.50.26Summary
This patch release adds a "one spelling per book" pass to the decide step, which keeps a book's findings from applying two different spellings of the same name. It also bumps the Helm chart and app version to 0.50.26.
Features
One spelling per book in earmark decide
The decide step now checks each book's findings for contradictory spellings. Previously, the judge could propose one name in two spellings (for example, ilmas and ilmis for the same name in one book), and each finding was decided on its own. The pass runs after the model is asked and before conflicts are ranked, in both dry runs and --yes runs.
For each applied finding or audio-liftable hold whose differing words sound alike, the pass does the following:
- Held
spelling_conflict: the book's findings propose two or more spellings for the same name, and none is established. - Held
spelling_off_established: the book has an established spelling, and the finding targets a different one. The established spelling comes from the book's applied findings or, if none exist, from a spelling that at least two pilot transcripts each contain at least three times, with no competing spelling in any pilot. - Applied
spelling_established: ano_evidencehold is promoted only when an applied finding already established the same original-to-spelling pair. Pilot agreement never promotes a finding.
Rejects, duplicates, rung-0 failures, and other holds are not changed. Each affected decision records its group in finding_events.detail (rung spelling), so you can audit it. No database migration is required.
Usage:
- Preview with
earmark decide --book "<dir>" --json(without--yes) and checkreasons.spelling_conflict,reasons.spelling_off_established, andreasons.spelling_established. A dry run still makes model calls, which are cached and write tofn_calls. - After a
--yesrun, count the pass's decisions with:
SELECT reason, count(*) FROM finding_events WHERE kind = 'decision' AND recipe_id = '<id>' AND detail->>'rung' = 'spelling' GROUP BY reason - Undo a run with
earmark decide revert --recipe <id>. The pass's holds and promotions are part of that recipe and revert with it.
Things to know:
- The decide step's
step_versionis now 4, so this creates a new decide recipe. Earlier decisions are re-checked under the new recipe rather than treated as final. - Books over a read cap (50,000 live findings, 5,000 pilot rows, or 200,000 chunks) are skipped and logged. The run continues, and the report counts the skipped books (
spelling_skipped_books, also printed in the text output). - In paged
--yesruns, a junk target rejected only on a later page cannot undo a hold written on an earlier page. A paged run can therefore hold more findings than a dry run of the same book, but it never applies more. - The audio-vote rung (
--audio-vote) does not run the spelling pass. Findings held by the pass are not lifted by audio votes. - Holds are final for a given recipe. A
spelling_*hold is reconsidered only under the next decide recipe.
Improvements
- Helm chart
versionandappVersionare now 0.50.26. - The decide recipe parameters now include
spelling_rule(one_spelling@v1) andspelling_min_pilot(3). CONTRACT.mdandDATABASE_SCHEMA.mddocument the spelling pass, its reasons, and thedetailformat.
Commits
- chore(earmark): bump chart version to 0.50.26 (625382b)
- feat(decide): one spelling per book pass (should_apply step 4) (#227) (b8fa6c4)
Full Changelog: v0.50.25...v0.50.26
Release 0.50.25
Install
Docker
docker pull ghcr.io/jedwards1230/earmark:0.50.25Helm
helm install earmark oci://ghcr.io/jedwards1230/charts/earmark --version 0.50.25v0.50.25 Patch Release
This patch changes how earmark decide --audio-vote handles findings that both ASR models heard as the original text. Sound-alike re-spellings and misheard proper-noun fixes are now held for review instead of rejected. The Helm chart version is bumped to match.
Bug Fixes
- Audio-vote rejects no longer hit sound-alike fixes. When every configured model heard the original text, the rung used to reject the finding. It now holds the finding as
asr_soundalikeif the original and replacement sound alike (for examplesheana → sheeanaorsyanite → syenite). The audio cannot tell these spellings apart, so the vote is not evidence against them. Sound-different fixes are still rejected as before. - Misheard proper nouns are held, not rejected. Any
misheard_proper_nounfinding that both models heard as the original is held asasr_reject_type_held, whatever its phonetic keys. A name spelled differently is something the audio cannot judge reliably. Applies are unchanged: when both models hear the replacement, the finding can still be applied.
Improvements
- Recipe version bumped to
asr_vote_decide@v2(step version 2). The recipe now includesreject_held_issue_typesandsoundalike_min_keyparameters. Version 1 rejected every finding both models heard as the original. - Dry run guidance. Before running with
--yes, check thehold/asr_soundalikeandhold/asr_reject_type_heldgroups, which stay held for review. Spot-check thereject/asr_heard_originalsample, where each fix should change the sound.
Commits
- chore(earmark): bump chart version to 0.50.25 (d2ebd81)
- fix(decide): hold sound-alike audio-vote rejects instead of rejecting (#226) (aed2820)
Full Changelog: v0.50.24...v0.50.25
Release 0.50.24
Install
Docker
docker pull ghcr.io/jedwards1230/earmark:0.50.24Helm
helm install earmark oci://ghcr.io/jedwards1230/charts/earmark --version 0.50.24Summary
v0.50.24 is a patch release that adds the audio-vote rung to earmark decide. The rung decides held findings from second-ASR pilot votes without calling a model. The release also adds a database migration (19) and Helm chart support for the new configuration.
Features
Audio-vote rung: earmark decide --audio-vote
The new rung uses votes from second-ASR pilot models (produced by earmark asr-pilot vote) to decide findings that are currently held. When every configured pilot model hears the proposed replacement, the finding can be accepted. When every model hears the original text, the finding is rejected. Anything else stays held.
Configuration. Set an audio_vote block in MODELS_FILE with a rule and at least two {model_id, revision} models. For a single run, pass --audio-rule with two or more --audio-model <model_id>@<revision> flags. The flags replace the file block and must be given together. If neither is set, the command exits before connecting to the database.
Usage:
earmark decide --audio-vote [--book X] # dry run: writes nothing
earmark decide --audio-vote --yes --limit 2000 --max-accepts 500
earmark decide revert --recipe <audio recipe id> # undo the rung's decisions- A dry run writes nothing, including no recipe row. It previews what
--yeswould write and accepts--sample,--seed,--json,--limit,--shard, and--max-accepts. --yeswrites only applies and rejects, recorded under a separate recipe. Holds are never written, so theshould_applyrecipe's work list is unchanged.- Each written decision stores the votes it relied on in
finding_events.detail. The audit tool does not serve this column, so read it with SQL. --audio-votecannot be combined with--calibrate,--rung0-only,--dump,--human,--endpoint,--model,--policy, or--concurrency.
Suggested rollout: run the dry run first, then a bounded --yes run, spot-check the results in finding_events, and use earmark decide revert if needed.
Improvements
- Database migration 19 adds a nullable
finding_events.detailJSONB column. It has no default and requires no backfill. Commands migrate on connect, so this runs automatically the first time a v0.50.24 binary connects. - Helm chart:
config.models.audio_voteis validated byvalues.schema.json. A block with no rule or fewer than two models is rejected. The chart does not enable the block by default.values.yamlincludes a commented example with the deployment's pilot pins.
Commits
- chore(earmark): bump chart version to 0.50.24 (02fd905)
- feat(decide): audio-vote rung (decide --audio-vote) (#225) (4ae7b3f)
Full Changelog: v0.50.23...v0.50.24
Release 0.50.23
Install
Docker
docker pull ghcr.io/jedwards1230/earmark:0.50.23Helm
helm install earmark oci://ghcr.io/jedwards1230/charts/earmark --version 0.50.23Summary
v0.50.23 is a patch release that fixes how the second-ASR pilot scores its votes. Shared words at the edges of a correction no longer inflate a match, so the pilot votes for a correction only when the differing words also match it.
Bug Fixes
ASR pilot voting (asrpilot)
- Differing-core check: When the original reading, the replacement, and the heard text share leading or trailing words, the vote is now also tested on the differing words alone. A heard vote must pass both the whole-span test and the core test, and both must point to the same reading. Otherwise the vote is
neither. For example, "person would voluntarily" heard for "person voluntarily" → "person could voluntarily" no longer counts as a match for the replacement. - Pure insertions and deletions: When one reading has no differing words, only an exact match (ignoring spaces) decides the vote. Anything else is
neitherwith reasonheard_other. - Vote rule bumped to
asr_vote@v2: Votes now recordasr_vote@v2indetail.rule. The new check only removes votes, so it never adds a heard vote that v1 would not have produced. Anything that readsdetail.ruleshould expect both values. Votes also record the core texts and their similarities underdetail.corewhen shared end words were dropped.
Improvements
- Help and docs: The
earmark asr-pilot votehelp text and CONTRACT §1.11 describe the new rule and thedetail.corefields. - Helm chart: The chart and app version are now 0.50.23.
Commits
- chore(earmark): bump chart version to 0.50.23 (019c36a)
- fix(asrpilot): score votes on the differing core, not the whole span (#224) (3cbfe57)
Full Changelog: v0.50.22...v0.50.23
Release 0.50.22
Install
Docker
docker pull ghcr.io/jedwards1230/earmark:0.50.22Helm
helm install earmark oci://ghcr.io/jedwards1230/charts/earmark --version 0.50.22v0.50.22 Release Notes
This patch release changes the ASR pilot runner (runner/pilot.py) to load models offline from local Hugging Face snapshots, and it lowers the default Granite window length. It also adds step-run tracking for pilot runs and a few new CLI options.
Breaking Changes
- Granite default window reduced from 240 s to 90 s, and the hard maximum from 540 s to 120 s. Per the runner docs, longer windows caused re-reading and dropped speech. Canary is unchanged (38 s default, 40 s max).
- Migration: if you set
PILOT_WINDOW_SECfor Granite to a value above 120, the run now fails with a config error. Set it to 120 or lower. --bench-max-secno longer defaults to 1800. It now defaults tomax(--bench-batches) × 2 × longest window. Pass it explicitly to keep a fixed cap.
- Migration: if you set
- Offline model loading requires prefetched snapshots. Runs set
HF_HUB_OFFLINE=1and read model directories from the local HF cache. They no longer download on demand. A missing snapshot or file fails the load with an error naming theasr_pilot_windows_<role>_allow_patternsvariable to fix.- Migration: prefetch the snapshots before running. Run
python pilot.py --backend <granite|canary> --print-prefetch-specto list the repos, revisions, and allow-patterns that pilot.py needs. - Canary also needs the
Qwen/Qwen3-1.7Bsnapshot (revision70d244cc86ccca08cf5af4e1e306ecf908b1ad5e, or override withPILOT_QWEN_REVISION).
- Migration: prefetch the snapshots before running. Run
--benchbehavior changed. It gathers windows across the--asintracks instead of using only the first track, and it ignores--limit-tracks.
Features
- Offline model loading. Models load from local snapshot directories in the Hugging Face cache, and
--print-prefetch-specprints the required snapshots as JSON. Canary's LLM config is pointed at the local Qwen snapshot, so no repo lookup happens at load time. - Pilot runs appear in run tracking. A real run writes one
step_runsrow (stepasr_pilot), so it shows in the dashboard's Running now panel and inGET /api/v1/runs. It reports progress, and it is closed asdone,failed, orcancelled. Dry runs,--bench, and--self-checkrecord nothing. --bench-windowssweeps several window lengths (for example60,90,120) in one bench run. Bench output now includes per-row degenerate rate and word completeness.--no-repeat-ngram-size N(Granite, default off) is available for experiments.
Improvements
- Granite stops each batch row on its own once it repeats an 8-token n-gram three times, so one re-reading row no longer holds the whole batch.
- Degenerate-window detection now also flags re-reads, meaning the same 8-word run three times or the same 16-word run twice. It also flags windows with more than 4 words per audio second as
overlong. - Special tokens that leak into the output (such as
<|fim_middle|>) are stripped from the text of both backends.
Commits
- chore(earmark): bump chart version to 0.50.22 (f79e1a6)
- fix(runner): load pilot models from local HF snapshots, offline (Canary self-check) (#223) (342f6ab)
Full Changelog: v0.50.21...v0.50.22
Release 0.50.21
Install
Docker
docker pull ghcr.io/jedwards1230/earmark:0.50.21Helm
helm install earmark oci://ghcr.io/jedwards1230/charts/earmark --version 0.50.21earmark v0.50.21 Release Notes
This patch release fixes the ASR pilot's --self-check so it verifies transcription on a real English clip instead of silence. It also adds an output-language check that flags non-English transcription windows.
Features
--self-check-audio PATH: a new option that selects the clip used by--self-check. The expected text is read from the same path with a.txtsuffix. The default is the bundledrunner/testdata/selfcheck_en.flac, a 5.9 s LibriSpeech dev-clean utterance (CC BY 4.0).
Bug Fixes
--self-checknow transcribes real English speech. It previously transcribed 5 seconds of silence. The code comments note that Granite answered silence with a Spanish sentence, so the old check did not verify English transcription. The check now decodes the bundled clip through the normal ffmpeg path, compares the transcript with the reference text using word error rate (threshold 0.35), and reports the results in its JSON output.--self-checkexit behavior changed. It exits6when the transcript is not English, is degenerate, or exceeds the WER threshold. It exits2when the clip or its.txtfile is missing. Scripts that expected exit0after a silent-input check should handle these codes.- Non-English windows are flagged during runs. A transcription window whose text does not look English is marked degenerate with reason
language, and it appears in the track's out-file. The check uses common function words for English, Spanish, French, German and Portuguese, and flags text that is mostly non-Latin script. It does not judge text shorter than 8 tokens.
Improvements
- Deployment files for
--self-check.testdata/selfcheck_en.flacandselfcheck_en.txtmust be installed next topilot.py. Their sha256 hashes are listed inrunner/testdata/README.md. Without these files,--self-checkexits with code2. - Documentation.
runner/README.mddocuments thelanguagedegenerate reason, the new self-check output fields, and exit code6.
Commits
- chore(earmark): bump chart version to 0.50.21 (5025a3e)
- fix(runner): pilot self-check on real English speech; flag non-English windows (#222) (870c5fc)
Full Changelog: v0.50.20...v0.50.21
Release 0.50.20
Install
Docker
docker pull ghcr.io/jedwards1230/earmark:0.50.20Helm
helm install earmark oci://ghcr.io/jedwards1230/charts/earmark --version 0.50.20Summary
This is a patch release that raises the default Helm resource requests and limits for earmark, and bumps the chart and app version to 0.50.20. The change addresses an out-of-memory kill of the ingest embed worker. No application code changed in this release.
Bug Fixes
- Ingest embed worker OOM kills: The ingest embed worker was OOMKilled at the previous 512Mi memory limit while processing a full re-embed backlog. The default memory limit is now 2Gi, and the default memory request is 256Mi. The values file notes that these settings apply to the ingest and MCP deployments.
Breaking Changes
-
Changed default resource values: The chart's default
resourcesblock is different, so deployments that rely on the defaults will now request and be allowed more resources.Setting Previous default New default requests.cpu100m 100m (unchanged) requests.memory128Mi 256Mi limits.cpu500m 1 limits.memory512Mi 2Gi Migration: If your cluster cannot accommodate the higher values, or you want the previous footprint, set
resourcesexplicitly in your own values override file to the previous values. Review this before upgrading if you run earmark on constrained nodes or enforce namespace quotas.
Commits
- chore(earmark): bump chart version to 0.50.20 (ced68b9)
- fix(chart): raise resource limit defaults past the embed OOM (#221) (293050f)
Full Changelog: v0.50.19...v0.50.20
Release 0.50.19
Install
Docker
docker pull ghcr.io/jedwards1230/earmark:0.50.19Helm
helm install earmark oci://ghcr.io/jedwards1230/charts/earmark --version 0.50.19v0.50.19 Release Notes (Patch)
This patch release hardens the ASR runner with transient-failure retries, resume from the last finished window, and mid-job yielding to GPU arbitration. It also adds a database migration (00018) and read-only status fields for gpu-arbiter's yield request.
Features
ASR runner: transient retry and classified failures
The Python runner now classifies each job failure as transient (DB connection loss, CUDA out of memory, unreachable books share, network I/O errors, ffmpeg/ffprobe timeouts) or permanent.
- Transient failures are retried in place with exponential backoff and jitter. Each class has its own retry budget.
- When retries run out, the job returns to
pendingwith its attempt kept and the error recorded. The claim that uses the last attempt fails the job instead, so jobs can't get stuck at the attempt cap. - After an I/O release, the runner holds new claims until the books share answers again. Heartbeat, self-update, and the GPU gate keep running during the hold.
- Permanent failures (for example, a missing file on a reachable share or undecodable audio) mark the job failed, as before.
- ffmpeg and ffprobe calls now have timeouts, so a hung read no longer stalls a job indefinitely.
ASR runner: resume from the last finished window
Chunked jobs write a local checkpoint after each window. A restart, crash, or retry resumes at the next window instead of 0 s.
- Checkpoints are stored in
RUNNER_CHECKPOINT_DIR(offdisables them) and are invalidated when the source file, model, or settings change. - Checkpoints are deleted when the job completes or fails permanently, and pruned after 7 days.
- The transcript is still written once, at the end of the job.
ASR runner: mid-job yield to gpu-arbiter
A new runner_control.arbiter_yield flag lets gpu-arbiter ask the runner to leave the GPU. The runner checks this between windows of a chunked job.
- An eviction costs at most one window of work.
- Pausing the runner now also stops a long job at its next window boundary, not after the whole file. The job resumes from its checkpoint when unpaused.
- Single-pass files, and chunked jobs with checkpointing disabled, still finish before parking.
Status API: arbiter yield visibility
/api/v1/status now includes:
pipeline.arbiterYield: whether gpu-arbiter is holding the runner off the GPUpipeline.arbiterYieldAt: the timestamp of the arbiter's last write, omitted until it has written once
These fields are read-only and independent of the batch phase.
Improvements
- Database migration 00018 adds
arbiter_yieldandarbiter_yield_attorunner_control. The change is additive and uses a constant default, so it rewrites no rows. - Heartbeat reliability: the per-job heartbeat reconnects after a failed connection or stamp, instead of going silent for the rest of the job. A silent heartbeat could let the stale sweep re-pend a running job.
- Claim-guarded job writes: the heartbeat,
_mark_done, and failure writes now check that the runner still holds the claim. A stale write can no longer overwrite a job that another claim has finished or reset._mark_donealso tolerates a replayed write when the job is alreadydone. - Fix: the runner's provenance-column check no longer fails when the connection returns rows as dictionaries.
Upgrade notes
- Apply the Earmark release containing migration 00018 before updating the runner pin (
asr_runner_windows_source_ref) in homelab-ansible. - The gpu-arbiter
gpu_gatechange that writesarbiter_yieldmust ship with or after that pin bump. The gate does not work against a schema without the column. - New runner settings are optional. Defaults are listed in
runner/README.mdanddocs/CONTRACT.md.
Commits
- chore(earmark): bump chart version to 0.50.19 (525e95f)
- feat(runner): ASR runner hardening — transient retry, resume from last window, mid-job yield (00018) (#216) (66164fe)
Full Changelog: v0.50.18...v0.50.19