Easy ASR Bench v0.4.0

Easy ASR Bench v0.4.0

Release status: automated packaging checks may be present, but this is not an all-pass manual smoke release because required rows remain unverified.

What changed

• Added browser-local persistence plus JSON export/import for corrected-reference edits in final_results.html, so pasted corrections survive report reloads and can be moved to another browser. Open_Latest_Report.bat now prefers final_results.html reports before falling back to per-file compare.html, single-file runs now write a lightweight final_results.html wrapper, and README report guidance now presents final_results.html as the main report entry point.
• Added installed-app Start Menu shortcuts for running Easy ASR Bench, dropping audio/folders, opening the latest report, opening the output folder, editing config, repair, and uninstall; uninstall removes the shortcut folder while preserving user data by default.
• Consolidated missing runtime dependency prompts into one pre-processing decision for all selected model groups. Interactive runs can install all, skip affected models, print every repair command, or quit before inference starts; noninteractive runs now fail affected models early with a clear pre-processing message instead of prompting during a batch.
• Hardened release-smoke evidence integrity: runtime-matrix rows now record execution Git commit and clean/dirty state, merged smoke rows keep the target release commit separate from execution evidence, strict validation rejects stale execution commits unless an explicit exception is documented, and duplicate same-row evidence cannot hide a fail behind a later pass.
• Hardened the Hugging Face downloader: repo inspection resolves floating revisions to a Hub commit when available, choices carry total-byte metadata, large-download confirmation is based on byte size, common GGUF Q4-family packages get a blank-Enter default recommendation, a resumable hf_model_layout_repair_plan.json is written before the first file download, failures print the exact doctor repair command, closed stdin exits cleanly, gated/private errors include token guidance, and users are warned about Hub cache plus Models-folder disk usage.
• Implemented the configured update checks: setup runs the setup-scoped GitHub latest-release check after config initialization, app startup honors check_for_updates_on_run, and offline or malformed update responses report a non-fatal unavailable status.
• Added a top-level crash handler for unexpected app errors. Config-load, model-scan, report-write, and other unhandled failures now write Logs/crash_*.log and print a short support path with doctor diagnostics and the GitHub issue URL instead of leaving a raw traceback as the only user-facing output.
• Added a public bug-report issue template and support docs that request doctor JSON, latest logs, Windows/install details, and model/runtime type. Setup documentation now explains unsigned script warnings, SmartScreen/Defender/AV behavior, what setup downloads, checksum verification, local-only media/model handling, and safe quarantine troubleshooting.
• Pinned the v0.4.0 llama.cpp Python package path to the verified llama-cpp-python==0.3.28 version across CPU, CUDA, Vulkan prebuilt-wheel, and explicit Vulkan source-build flows so clean-machine repair does not silently drift to a different wheel.
• Added disk-space preflights before Hugging Face model downloads and large runtime dependency installs. Hugging Face downloads estimate cache plus Models-folder space when size metadata is available and require typed override on low space; dependency installs record the space check and block low-space installs unless dependency_install.allow_low_disk_space_install is explicitly enabled.
• Added doctor path diagnostics for risky Windows locations: non-ASCII install/user-profile paths and OneDrive or redirected paths now appear in doctor JSON/text output, with a runtime-matrix row covering the warning contract.
• Added repeat-last-run model selection persistence. Interactive model choices save compact candidate IDs and precision buckets in config.json; blank Enter reuses a still-valid saved selection, and drag/drop runs use the saved selection unattended while stopping with a clear stale-selection message if saved IDs no longer exist.
• Added batch resume checkpointing in Logs/batch_resume_manifest.json. Completed file/model pairs are recorded with input fast keys and a config/model signature, reruns skip files whose selected pairs are already complete, stale input/config changes invalidate the checkpoint, and interrupted batches write a partial final_results.html summary when completed rows exist.
• Clarified the first-run baseline: Systran/faster-whisper-tiny.en is presented as a small CPU-first, English-only, one-model sanity baseline rather than a ranking comparison pack, and first-run smoke JSON records those limits plus the size-gated downloader path.
• Added stale temp WAV cleanup on startup and doctor runs. Generated *_16k_mono.wav files older than advanced.stale_temp_wav_hours are removed from Temp, recent active files are preserved, and advanced.keep_temp_wavs=true disables the sweep.
• Closed the still-valid v0.3.9 release-audit gaps around release QA gates: the staged bad-checksum runtime row now runs on CI without a GitHub Actions skip, Windows release QA wrappers require an explicit vX.Y.Z tag instead of stale defaults, and the shipped default config now starts CPU-first with accelerator package installs disabled until explicitly opted in.
• Aligned llama.cpp Vulkan runtime planning with the installer path: machines with a Vulkan runtime now select the Vulkan prebuilt-wheel route even without the Vulkan SDK, GPU layers remain disabled until backend offload is verified, and explicit source builds use a separate requirements/llama_cpp_vulkan_source_build.txt file with SDK/build tooling opt-in.
• Fixed the direct --download-model CLI path so a successful Hugging Face model download immediately rescans and continues into the normal interactive or scan-only flow instead of telling the user to restart, and corrected the README batch-output tree to the actual final_results.html plus _data JSON layout.
• Added a generated support matrix at docs/support_matrix.generated.md that is rebuilt from release-smoke-v0.4.0.json, reports required-row pass counts, and keeps unproven CUDA/Vulkan/AMD/provider rows marked Not verified until release-smoke evidence marks them pass.
• Refreshed the local v0.4.0 Windows ZIP after post-audit hardening; the current package SHA256 is c10f0213f206545556ec5d6efede10f1c43b29c6ab4bd81f2cc2b184c9ab821b. Full pytest passed after the changes (609 passed), along with pip check, release-file validation, repo/ZIP physical-file validation, strict checksum rebuild, setup dry-run, setup JSON dry-run, tracked public hygiene, smoke public hygiene, and strict required-row smoke validation.
• Hardened the Windows Sandbox clean-bootstrap script with per-step JSON/stdout/stderr/exit-code sentinels, short dry-run preflight evidence, command-shape preservation across PowerShell jobs, and a real setup.bat --local --no-post-setup-menu bootstrap step before doctor repair. The first real clean Sandbox phase now fails fast on missing Python bootstrap support instead of hanging or falsely passing.
• Added a clean-Windows Python bootstrap fallback for environments such as Windows Sandbox where winget is unavailable: setup.bat --local now downloads and silently installs the official Python 3.12.10 x64 installer from python.org, then verifies py -3.12, python, or the per-user install path before creating the project venv.
• Bounded the clean-Windows Python fallback download by trying curl.exe with connect/overall timeouts before the PowerShell fallback, checking that the installer file is actually present and larger than 1 MB before executing it, and disabling PowerShell progress output so Sandbox bootstrap does not appear frozen during installer download.
• Added a release-hardening regression check that setup.bat is UTF-8 without a BOM after Sandbox caught cmd.exe interpreting the BOM as part of @echo off.
• Updated the Windows Sandbox startup script to run post-bootstrap runtime rows, full real-smoke, and release-smoke scripts through the newly created .venv\Scripts\python.exe instead of relying on the Sandbox process PATH to discover python.
• Hardened clean-VM dependency repair after Sandbox proof reached strict ASR/LLM rows: llama-cpp-python now falls back to the official prebuilt CPU wheel index instead of a local source build when accelerator wheels are not usable, OpenVINO provider repair skips unavailable compatibility versions before trying the next candidate, and clean-bootstrap first-run smoke parsing accepts final JSON payloads after diagnostic log lines.
• Tightened clean-bootstrap runtime-row parsing and provider fallback after the next Sandbox pass: setup repair-all-safe rows now parse schema-bearing JSON payloads from noisy doctor output, and auto-detected OpenVINO provider failures no longer block CPU fallback while explicit OpenVINO requests remain strict.
• Fixed the remaining clean-Sandbox llama-cpp fallback path so an unavailable accelerator wheel index installs from the prebuilt CPU wheel index instead of returning to the source-build requirements file on machines without native build tools.
• Fixed setup repair-all-safe runtime-row parsing to select the top-level repair-plan JSON payload, not nested runtime-resolution schema objects embedded later in the same doctor output.
• Added a clean-Windows Visual C++ Redistributable bootstrap step to setup.bat: setup now verifies the x64 runtime before doctor/self-test, prefers the winget package when available, and falls back to Microsoft’s supported x64 redistributable permalink with bounded download checks when winget is absent.
• Hardened clean-machine native backend probes after Sandbox validation: portable llama-mtmd-cli probes now run from the extracted tool directory so adjacent DLLs load correctly, and the faster-whisper/CTranslate2 isolated import probe allows a longer first-import window on fresh Windows environments.
• Hardened the same-media benchmark dependency preflight so a native-tool repair exception is rechecked before blocking; if the fallback repair produced a usable runtime, the row records the recovered exception and continues.
• Hardened isolated runtime rows that use their own cache folders: llama-mtmd-cli discovery now reuses a verified project-level native-tool cache and stages it into the row cache before falling back to a fresh native repair/download.
• Hardened isolated provider validation rows so an initial dependency conflict triggers one safe row-local dependency repair and provider recheck before the row is marked blocked.
• Hardened clean Sandbox release-smoke generation for environments without Git on PATH, and adjusted Sandbox step execution so native-command stderr is captured completely for diagnostics.
• Revalidated the Windows CUDA dGPU path for v0.4.0: Torch CUDA tensor smoke, ONNX Runtime CUDA tiny session, faster-whisper/CTranslate2 CUDA smoke, GGUF SmolLM llama.cpp CUDA smoke, and the combined NVIDIA CUDA row now have passing local evidence pending release-smoke merge.
• Hardened llama.cpp native DLL discovery on Windows by retaining os.add_dll_directory handles and preloading package-local ggml/llama DLLs before product and runtime-matrix imports, fixing direct GGUF reference rows and isolated GPU-offload probes after CUDA wheel installs.
• Updated faster-whisper CUDA dependency recovery to accept current NVIDIA Python wheel layouts that expose nvidia.cublas.bin and nvidia.cudnn.bin, while still accepting older .lib marker packages.
• Made the faster-whisper CUDA runtime row parse the final schema-bearing JSON payload from noisy subprocess output, and changed CUDA repair-command evidence to report the resolved llama.cpp CUDA wheel command instead of a stale static cu124 requirements file.
• Switched the public SmolLM GGUF validation fixture to QuantFactory/SmolLM-135M-GGUF after the previous HuggingFaceTB/SmolLM-135M-GGUF reference was unavailable during v0.4 validation; the local Q4_K_M fixture is hashed in runtime evidence.
• Capped automatic llama.cpp CUDA wheel selection at the verified cu125 index on newer NVIDIA drivers after cu130 and cu132 wheels installed on driver 591.59 but did not expose working GPU offload on this host. Explicit dependency_install.llama_cpp_cuda_tag overrides remain available for future wheel validation.
• Fixed the repair-all-safe backend probe path found during v0.4 required-row validation: core requirements now declare the jinja2 and jiwer imports that the core probe verifies, and isolated llama.cpp repair probes preload the same Windows DLL search path used by product GGUF imports before importing llama_cpp.
• Hardened the Windows Sandbox clean-bootstrap gate so a successful Sandbox launch no longer counts as a pass by itself. The row now passes only after mapped Sandbox evidence includes the clean Win11 setup row, clean-VM bootstrap row, release-smoke validation log, and sandbox smoke JSON; launch-only or missing mapped evidence remains blocked.
• Added Windows Sandbox contention detection to the clean-bootstrap gate after the validation host already had another project's Sandbox running. Easy-ASR now records a clear blocked row instead of launching a second Sandbox instance that cannot produce mapped completion evidence.
• Isolated Windows Sandbox deploy-row regression tests from real local Sandbox evidence so a completed release-smoke file cannot short-circuit blocked-path assertions on validation machines.
• Clarified release-verification docs that the Sandbox row requires mapped completion evidence and does not replace the separate Windows 10 existing-Python setup proof.
• Hardened Intel ONNX provider rows after validation on an Intel CPU/iGPU host: DirectML/OpenVINO hardware rows now try row-local provider repair when requested, fail instead of passing on CPU fallback, add OpenVINO DLL search paths before ONNX Runtime session creation, pin the OpenVINO runtime package to the ONNX Runtime compatibility release, and keep ONNX accelerator requirements inside shared NumPy/tokenizers bounds.
• Sanitized merged release-smoke evidence so public smoke artifacts keep provider/runtime facts while replacing exact local GPU model strings, CPU model strings, and user-profile paths with generic labels.
• Added a public-hygiene validator for tracked release files and generated release-smoke artifacts so local machine details, local-only transfer references, unrelated workspace paths, and Sandbox account-password diagnostics are caught before packaging or publication.
• Extended the public-hygiene validator with reachable-history scanning for pre-push cleanup; it reports commit/path/line and a generic category without echoing the matched local/private content.
• Added a safe public-history rewrite helper that fast-exports a selected ref into a separate candidate repository with local/private terms sanitized, allowing the candidate history to be validated before any local branch replacement or later force-push decision.
• Wired public-hygiene validation into the release workflows: branch/release gates scan tracked files and published smoke artifacts, while publish requires the checked-out release ref history to pass before assets can be uploaded.
• Added scripts/collect_win10_setup_evidence.py so the remaining Windows 10 existing-Python release gate has a single command that runs the runtime row, rejects Windows 11/blocked evidence, and can merge plus strict-validate the release-smoke artifact when a real Windows 10 pass is collected.
• Validated win10_existing_python_setup in a real Windows 10 22H2 VM with Python 3.12 already visible before setup dry-run. The local v0.4.0 release-smoke artifact now has strict required-row evidence for the Windows 10 existing-Python setup gate.
• Added scripts/validate_v040_release_readiness.py as a single release-readiness audit wrapper for strict smoke, public tree/smoke hygiene, and reachable-history hygiene. Against the sanitized candidate repo it now reports only the remaining Windows 10 row; against the current unreplaced main it reports the Windows 10 row plus history hygiene findings.
• Added an explicit intel_cpu_onnx_smoke required release row so Intel CPU validation is proven separately from generic CPU and Intel iGPU/OpenVINO rows. The row requires an Intel CPU host, runs the tiny Generic ONNX CTC fixture with CPUExecutionProvider, and fails if CPU provider evidence records fallback or a non-CPU active provider.
• Tightened win10_existing_python_setup so Windows 10 proof requires a real Windows 10 build below the Windows 11 build floor, preventing ambiguous Windows version reporting from satisfying the remaining OS-specific release gate on a Windows 11 host.
• Revalidated the v0.4.0 source/package gate on a Windows validation host with Python 3.12: release-file validation, physical-file validation, compileall, local setup dry-run, JSON setup dry-run, v0.4.0 ZIP build/metadata validation, ZIP physical validation, and the full pytest suite all passed (597 passed).
• Bumped the active validation/release line to v0.4.0, extended scripts/bump_version.py so future bumps update installer metadata, checksum ZIP names, Sandbox release-smoke scripts, installer runtime rows, and release verification docs, then regenerated and validated the local Easy-ASR-Bench-v0.4.0-win.zip package metadata.
• Fixed a syntax error in qa/run_real_tiny_model_smoke.py that prevented release validators from parsing the real faster-whisper smoke runner on the v0.4 validation machine; source, physical-file, and compile gates now pass after installing the validator's PyYAML dependency.
• Stabilized the v0.4.0 validation harness after the security-audit hardening pass: runtime-matrix Git evidence collection now avoids row-local monkeypatch leakage, the dependency-accepted failure-isolation row exercises the consolidated batch dependency prompt, and the full pytest suite passed with 657 passed.
• Refreshed the v0.4.0 release-smoke artifact for commit a6d434c: all 65 required release rows now pass strict validation with execution commit evidence, log/result hashes, and environment summaries. Current optional Intel DirectML/OpenVINO and NVIDIA CUDA provider rows were rerun at the same commit, and the generated public support matrix now marks the CUDA stack verified.

Automated Packaging Checks

• Built from commit: 954f6ad.
• Public setup verification path: setup.bat --dry-run --verify-release.
• GitHub Actions Release Gate must pass before a release should be promoted.
• release_file_validation: pass - C:\Users\PC\AppData\Local\Programs\Python\Python312\python.exe scripts/validate_release_files.py
• repo_physical_file_validation: pass - C:\Users\PC\AppData\Local\Programs\Python\Python312\python.exe scripts/validate_physical_files.py --repo .
• zip_physical_file_validation: pass - C:\Users\PC\AppData\Local\Programs\Python\Python312\python.exe scripts/validate_physical_files.py --zip E:\_github\Easy-ASR-Bench\dist\Easy-ASR-Bench-v0.4.0-win.zip
• version_coherence: pass - C:\Users\PC\AppData\Local\Programs\Python\Python312\python.exe scripts/check_release_version_coherence.py --tag v0.4.0

Manual Smoke Rows Marked Pass

• win11_clean_no_python_setup: pass
• windows_sandbox_clean_bootstrap_deploy: pass
• clean_vm_zero_dependency_bootstrap: pass
• win10_existing_python_setup: pass
• windows_vc_runtime_repair_contract: pass
• python_packaging_tools_repair_contract: pass
• transformers_cpu_dependency_repair_contract: pass
• media_tools_dependency_repair_contract: pass
• llama_mtmd_dependency_repair_contract: pass
• directml_provider_conflict_repair: pass
• install_path_with_spaces: pass
• setup_verify_release_bad_checksum: pass
• setup_dry_run_json: pass
• setup_doctor_strict: pass
• setup_repair_all_safe: pass
• repair_all_safe_failure_isolation: pass
• repair_all_safe_stale_cached_resolution: pass
• repair_plan_issue_classification_contract: pass
• setup_repair_model_layouts: pass
• repair_broken_venv: pass
• uninstall_preserve_user_data: pass
• empty_models: pass
• nested_models_scan: pass
• wav_mp3_mp4_media: pass
• corrupt_media_readable_error: pass
• no_audio_video_readable_error: pass
• compare_html_offline_large_transcript: pass
• compare_html_offline: pass
• report_atomic_write_failure_cleanup: pass
• watched_folder_partial_write_queue_contract: pass
• one_model_failure_continues: pass
• one_chunk_failure_continues: pass
• llm_reference_json_import: pass
• dependency_install_declined: pass
• hf_downloader_package_variant_taxonomy: pass
• model_fixture_quality_claims: pass
• same_media_multi_model_smollm_benchmark: pass
• same_media_multi_model_smollm_benchmark_directml: pass
• real_public_folder_batch_smollm_benchmark: pass
• nvidia_cuda_torch_onnx_faster_whisper_llama: pass
• nvidia_cuda_hardware_detection: pass
• torch_cuda_tensor_smoke: pass
• onnxruntime_cuda_tiny_session: pass
• faster_whisper_ctranslate2_cuda_smoke: pass
• llama_cpp_cuda_smollm_smoke: pass
• intel_cpu_onnx_smoke: pass
• intel_directml_onnx_smoke: pass
• intel_openvino_onnx_smoke: pass
• hf_whisper_safetensors_cpu: pass
• hf_whisper_sharded_safetensors_smollm_grading_cpu: pass
• hf_whisper_safetensors_quality_smollm_grading_cpu: pass
• real_public_video_hf_whisper_safetensors_smollm_grading_cpu: pass
• hf_safetensors_asr_quality_smollm_grading_cpu: pass
• faster_whisper_pkg_resources_repair: pass
• faster_whisper_ctranslate2_candidate_fallback_repair: pass
• faster_whisper_vc_runtime_repair: pass
• faster_whisper_cpu: pass
• real_public_video_openai_whisper_pt_smollm_grading: pass
• faster_whisper_cuda_unavailable_cpu_fallback: pass
• transformers_cuda_unavailable_cpu_fallback: pass
• whisper_cpp_ggml_speech_smollm_grading: pass
• real_public_video_whisper_cpp_ggml_smollm_grading: pass
• openai_whisper_pt_checksum_verified: pass
• openai_pt_unverified_blocked: pass
• openai_whisper_cuda_unavailable_cpu_fallback: pass
• generic_onnx_manifest_cpu: pass
• generic_onnx_cuda_unavailable_cpu_fallback: pass
• generic_onnx_openvino_unavailable_cpu_fallback: pass
• real_public_video_generic_onnx_ctc_smollm_grading_cpu: pass
• generic_onnx_without_manifest_rejected: pass
• gguf_asr_mmproj_pair: pass
• mismatched_audio_asr_gguf_mmproj_rejected: pass
• gguf_text_llm_reference_only: pass
• standalone_safetensors_incomplete: pass

Not Verified In Release Smoke

• first_run_smoke_json: not_run
• install_path_risk_warnings: not_run
• setup_double_click_equivalent: not_run
• setup_dry_run_verify_release: not_run
• update_preserves_user_data: not_run
• destructive_uninstall_requires_phrase: not_run
• bad_checksum_fails_before_execution: not_run
• tampered_installer_fails_before_execution: not_run
• interrupted_download_rollback: not_run
• broken_venv_repair: not_run
• empty_models_folder: not_run
• nested_models_folders: not_run
• wav_mp3_mp4_no_audio_corrupt_media: not_run
• batch_continues_after_one_model_or_chunk_fails: not_run
• dependency_install_accepted: not_run
• hf_downloader_supported_outcome_taxonomy: not_run
• cpu_model_smoke: not_run
• llama_cpp_vulkan_smollm_smoke: not_run
• amd_directml_onnx_smoke: not_run
• vulkan_runtime_no_sdk: not_run
• vulkan_runtime_with_sdk: not_run
• hf_safetensors_asr: not_run
• hf_whisper_safetensors: not_run
• hf_whisper_safetensors_smollm_grading_cpu: not_run
• real_public_media_hf_whisper_safetensors_smollm_grading_cpu: not_run
• hf_safetensors_asr_smollm_grading_cpu: not_run
• sharded_safetensors_index: not_run
• faster_whisper_ctranslate2: not_run
• real_tiny_faster_whisper_report_smoke: not_run
• real_tiny_faster_whisper_smollm_grading: not_run
• real_public_media_faster_whisper_smollm_grading: not_run
• real_public_video_faster_whisper_smollm_grading: not_run
• real_public_media_openai_whisper_pt_smollm_grading: not_run
• whisper_cpp_ggml: not_run
• whisper_cpp_ggml_smollm_grading: not_run
• real_public_media_whisper_cpp_ggml_smollm_grading: not_run
• openai_whisper_pt_unknown_blocked: not_run
• generic_onnx_ctc_manifest_v1: not_run
• generic_onnx_smollm_grading_cpu: not_run
• generic_onnx_smollm_grading_directml: not_run
• generic_onnx_ctc_quality_smollm_grading_cpu: not_run
• real_public_media_generic_onnx_ctc_smollm_grading_cpu: not_run
• multi_file_onnx_ar_nar: not_run
• audio_asr_gguf_mmproj: not_run
• real_public_media_gguf_asr_mmproj_smollm_grading: not_run
• incomplete_audio_asr_gguf_mmproj_rejected: not_run
• gguf_reference_llm: not_run
• smollm_reference_grading_report: not_run
• hf_text_llm_safetensors_unsupported: not_run
• known_unsupported_asr_families_explained: not_run

Release assets

• Easy-ASR-Bench-v0.4.0-win.zip: sha256:6d842389289829c4fbdce454392004e64105a265cb3fbdf4cfc54e310abca43d
• install.ps1: sha256:6e0cfb08a92a64378b539da0d2a89271bdb931d67057dc004a9fd31bab3d1279
• manifest.json: sha256:57eee2b7b273def9b7fd3bdc2c22d79a2f9a9728bb9f40e9c0b7751ae405d149
• setup.bat: sha256:ca2a60348fc334268f708bfb3e39894d444babc13fbe24a1eb60ffee476bf761

Known limits

• Optional model dependency groups install only when needed.
• VRAM metrics use Windows GPU Adapter Memory counters when available and Torch CUDA allocator metrics as a labeled fallback; unavailable telemetry is reported explicitly.
• Unsafe pickle-backed .pt checkpoints remain blocked unless explicitly trusted.