Skip to content

fix(runner): runtime-error detection is English-only, so it does nothing under a translated locale #998

Description

@Chemaclass

Summary

bashunit::runner::detect_runtime_error recognises a shell error by matching English phrases — command not found, unbound variable, permission denied, and about twenty more. bash translates those diagnostics, so under any locale with bash's message catalogue installed, none of them matches and a genuine runtime error is not classified at all.

LC_ALL=C      /tmp/x.sh: line 1: foo: command not found      → classified
LC_ALL=es_ES  /tmp/x.sh: línea 1: foo: orden no encontrada   → not classified

How it surfaced

While fixing #992 I added a regression guard asserting that a real shell error is still reported as an Error. It passed on every English job and failed on Ubuntu - Spanish Locale and Ubuntu - Japanese Locale, with the fixture exiting 0 — the error was never detected, so the test passed and the run was green.

I pinned that guard to LC_ALL=C, because it exists to test classification mechanics rather than localization. That is the right scope for it and the wrong place to leave the underlying gap unrecorded.

Impact

A user running under a translated locale gets no runtime-error classification. Their test still fails — the exit code and assertion counts are unaffected — but it is reported as a plain Failed with no indication that the cause was a shell error rather than a wrong value. That is precisely the distinction the Error channel exists to draw, and it is the one that saves debugging time.

The repo already runs Spanish, Brazilian and Japanese locale jobs, so this is a supported configuration, not a hypothetical.

Why phrase matching was chosen, and what it costs

Matching on message text is the reason this is locale-dependent. Two directions worth weighing:

  1. Force the C locale around the captured command. Setting LC_ALL=C for the test subshell would make diagnostics predictable, but it changes what the code under test sees — a test asserting on localized output would break. Almost certainly wrong.
  2. Stop relying on message text where possible. Exit codes carry some of this unambiguously (127 not found, 126 not executable, 128+N killed by signal N) and are locale-independent. That would not cover every phrase in the list, but it would cover the common ones without any translation dependency.
  3. Accept and document it. If the classifier stays English-only, say so, so nobody is surprised that the Error channel goes quiet on their machine.

My preference is 2 for the codes that carry the information, and 3 for the remainder.

Constraints

  • Bash 3.0+; case and parameter expansion only.
  • Per-test failure path — no new fork. See .claude/rules/perf-fork-budget.md.
  • The prefix rule added in fix(runner): a quoted shell-error phrase is no longer a runtime error #997 (: line N: , or its translated equivalent, on the same line as the phrase) is itself partly text-shaped — línea and still match its generic form, but any change here should keep that working.
  • Failure output is compared verbatim across the acceptance suite; regenerate snapshots deliberately and read the diff.

Acceptance criteria

  • A genuine command not found is classified as an Error under a translated locale, or the limitation is documented explicitly
  • Existing English classification does not regress — covered by the tests added in fix(runner): a quoted shell-error phrase is no longer a runtime error #997
  • The LC_ALL=C pin on test_a_real_shell_error_is_still_reported is revisited: kept deliberately, or removed if classification becomes locale-independent
  • make sa · make lint · ./bashunit --parallel --simple --strict tests/ · bash build.sh bin -v

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions