Skip to content

Repository navigation

v1.1.2

Choose a tag to compare

View all tags
@weiwchu weiwchu released this 09 Oct 05:48
· 13 commits to main since this release
v1.1.2
a570328

FA-Bench v1.1.2 is a maintenance release. No change moves a published number, so the records/202609/ snapshot stands as scored for v1.1.0.

Fixed

  • Olign is named Olign 0.9, the version that produced its numbers. The records in both snapshots, the provenance block and the Olign adapter's docs had called it Olign 1.0. Only the name changes.
  • docs/paper.pdf is the revised paper. Its title is now "FA-Bench: A Benchmark for Phone- and Word-Level Timestamp Accuracy in Forced Alignment and ASR on Clean and Noisy Speech". Its word table names Olign 0.9, and the abstract and the introduction open with a new first sentence. The tables are unchanged.

Added

  • ElevenLabs forced alignment as a Track 1 row, elevenlabs_fa. Each utterance goes to ElevenLabs' /v1/forced-alignment endpoint with its reference transcript, and the row is scored on word boundaries beside MFA and Olign. It ships with enabled: false and no published numbers, because every request bills the account.
  • cloud_check.py takes --text, so it can check a forced aligner. cloud_cost.py prices the new row.
  • A one-column, double-spaced copy of the paper, docs/paper_for_reviewer.pdf, which the README's paper links now open. Its tables are set larger and write every number with its leading zero, 0.40 where the paper has .40, and its Fig. 1 is larger. evals/gen_paper_tables.py writes its tables in the same run as the paper's, so the two cannot disagree. The two-column docs/paper.pdf stays, and a new section at the end of the README links both and the paper on arXiv, arXiv 2609.32396.

Changed

  • The commercial cloud adapter can send the reference transcript with the audio, and the transcript then joins the response cache key. The Track 2 rows send none, so their keys and cached responses are unchanged.

The full changelog is at v1.1.1...v1.1.2

Assets 2
Loading