Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
- npm registry distribution: `npm install -g bashunit` (#244)
- `bashunit::env::supports_color` and `bashunit::io::clear_screen` helpers (#247)
- LCOV reports now include `FN`, `FNDA`, `FNF` and `FNH` function records, consumed by `genhtml`, Codecov and Coveralls
- LCOV reports now include `BRDA`, `BRF` and `BRH` branch records for `if`/`elif`/`else` chains and `case` patterns (see `adrs/adr-007-branch-coverage-mvp.md`)
- `BASHUNIT_COVERAGE_SHOW_FUNCTIONS=true` adds a per-function coverage block to the text report
- `BASHUNIT_COVERAGE_SHOW_UNCOVERED=true` adds an "Uncovered Lines" block to the text report, with consecutive line numbers compressed into ranges

Expand Down
90 changes: 90 additions & 0 deletions adrs/adr-007-branch-coverage-mvp.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Branch Coverage MVP via Static Branch-Point Detection

* Status: accepted
* Date: 2026-05-04

## Context and Problem Statement

Coverage today reports line-level execution only. Standard tooling (genhtml, Codecov, Coveralls) consumes branch records via the LCOV `BRDA`/`BRF`/`BRH` fields, which let reviewers see whether `else`/`elif` arms and individual `case` patterns were exercised. Adding true branch coverage to a Bash framework is non-trivial because:

1. Bash exposes no native instrumentation comparable to gcov branch counters.
2. The DEBUG trap fires on commands, not on branch decisions.
3. `BASH_COMMAND` reflects the *next* command, not the boolean outcome of a conditional.

We need a path that yields useful, mostly-correct branch metrics in LCOV reports without breaking Bash 3.0+ compatibility or the cost profile of the existing line tracker.

## Decision Drivers

* Bash 3.0+ compatibility (no associative arrays, no `[[`, no Bash 4-only features).
* Reuse existing line-hit data; do not double the runtime cost of coverage.
* LCOV output must be consumable by genhtml, Codecov and Coveralls without custom processing.
* Implementation must fit in `src/coverage.sh` and remain testable with the existing unit-test patterns.
* Behavior must be predictable enough to pin in tests; "best-effort heuristic" outputs are not acceptable.

## Considered Options

1. **Static branch-point detection plus line-hit inference** — parse the source file for branch-introducing constructs (`if`/`elif`/`else`, `case` patterns), compute the line range owned by each outcome, then mark the outcome as "taken" iff any line inside its range was hit.
2. **Runtime decision tracing via `BASH_COMMAND`** — record the actual command being executed in the DEBUG trap and reconstruct decisions taken (`if X` followed by execution of either then-block or else-block).
3. **Patch-based instrumentation** — preprocess source files to insert hit recorders inside each branch arm, run tests against the instrumented copy, post-process the data file.

## Decision Outcome

Chosen option: **Option 1 (static branch-point detection plus line-hit inference)**.

It reuses the existing line-hit data file with no DEBUG-trap changes. Bash 3.0+ compatibility is preserved because the parser is a single pass over the source with brace counting, identical in shape to the existing `extract_functions` walker. The output maps cleanly to LCOV `BRDA` records, and the contract ("an arm is taken iff any executable line inside it was hit") is precise enough to write unit tests against.

### Positive Consequences

* Zero runtime cost beyond the existing line tracker. Branch records are computed during report generation, not during test execution.
* Reuses `is_executable_line` and `get_all_line_hits`, which already tolerate Bash 3.0 limitations.
* LCOV output remains a single file, consumed unchanged by downstream tools.

### Negative Consequences

* Branch detection is line-presence based, not outcome based. A `then` arm whose only statement is a comment-line will register as `not taken` even if the conditional fired (because there are no executable lines inside). This is documented as a known limitation.
* Implicit `else` (when an `if/elif` chain has no explicit `else`) is reported only when at least one explicit arm exists; the synthetic "fall-through" outcome is omitted from this MVP and may be added in a follow-up.
* Compound conditionals (`if A && B`) are reported as a single binary decision, not per sub-expression.

## Pros and Cons of the Options

### Option 1: Static + line-hit inference (chosen)

* Good, because reuses existing data and code paths.
* Good, because matches the implementation pattern of `extract_functions` already shipping in the codebase.
* Good, because output is deterministic and easy to test.
* Bad, because cannot distinguish "arm executed but produced no executable lines" from "arm not executed".

### Option 2: Runtime DEBUG-trap decision tracing

* Good, because reflects actual runtime behavior.
* Bad, because `BASH_COMMAND` semantics across Bash 3.x and 5.x diverge for `((...))`, `[[...]]` and pipelines, requiring per-version logic.
* Bad, because increases per-line overhead; the existing tracker already has measurable cost.
* Bad, because subshell context loss (already documented for line coverage) extends to branches taken inside `$(...)`.

### Option 3: Source-rewrite instrumentation

* Good, because most accurate signal possible.
* Bad, because requires either running tests against a rewritten source tree or hooking `source` to redirect to instrumented copies — both invasive and brittle.
* Bad, because debugging stack traces and line numbers no longer match the user's source.
* Bad, because doubles the code surface and breaks the "DEBUG-trap only" simplicity model.

## Scope of MVP

Included:

* `if`/`elif`/`else` chains: each arm is one outcome.
* `case` statements: each pattern is one outcome.
* LCOV `BRDA:<line>,<block>,<branch>,<taken>` lines.
* `BRF:<count>` and `BRH:<count>` per file.

Deferred (potential follow-ups):

* Synthetic "implicit-else" outcomes for `if/elif` chains without an explicit `else`.
* Per-sub-expression decisions inside `if A && B`.
* `&&` / `||` short-circuit branches outside `if`.
* Loop-entry decisions (`while`/`until`).

## Links

* Builds on the function extractor introduced in `src/coverage.sh` (see `bashunit::coverage::extract_functions`).
* LCOV format reference: <https://manpages.debian.org/unstable/lcov/geninfo.1.en.html>
132 changes: 132 additions & 0 deletions docs/coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,13 @@ end_of_record
|-------|-------------|---------|
| `TN:` | Test Name (usually empty) | `TN:` |
| `SF:` | Source File path | `SF:/home/user/project/src/math.sh` |
| `FN:` | Function: `start_line,name` | `FN:5,multiply` |
| `FNDA:` | Function call data: `count,name` (1 if any line in body was hit, else 0) | `FNDA:1,add` |
| `FNF:` | Functions Found | `FNF:2` |
| `FNH:` | Functions Hit | `FNH:1` |
| `BRDA:` | Branch data: `decision_line,block,arm,taken` | `BRDA:12,0,1,1` |
| `BRF:` | Branches Found | `BRF:6` |
| `BRH:` | Branches Hit | `BRH:4` |
| `DA:` | Line Data: `line_number,hit_count` | `DA:15,3` (line 15 hit 3 times) |
| `LF:` | Lines Found (total executable lines) | `LF:25` |
| `LH:` | Lines Hit (lines with hits > 0) | `LH:20` |
Expand Down Expand Up @@ -351,6 +358,131 @@ These lines are not counted toward coverage:
- Control flow keywords (`then`, `else`, `fi`, `do`, `done`, `esac`, `in`)
- Case statement patterns (`--option)`, `*)`) and terminators (`;;`, `;&`, `;;&`)

## Branch Coverage

Beyond line and function coverage, bashunit emits **branch coverage** records in the LCOV report so reviewers can see whether each `else`/`elif` arm and each `case` pattern was exercised. Branch records are produced automatically; no extra flags are needed.

### What Counts as a Branch

| Construct | Arms |
|-----------|------|
| `if X; then ... fi` | 1 (the `then` body) |
| `if X; then ... else ... fi` | 2 (`then` + `else`) |
| `if X; then ... elif Y; then ... else ... fi` | 3 (one per arm) |
| `case X in a) ... ;; b) ... ;; *) ... ;; esac` | one per pattern |

An arm is reported as **taken** iff at least one executable line inside its range was hit by tests.

### Verbose Output Helpers

Two opt-in environment variables enrich the text report when investigating coverage gaps:

::: code-group
```bash [Per-function block]
BASHUNIT_COVERAGE_SHOW_FUNCTIONS=true bashunit tests/ --coverage
```
```bash [Uncovered lines block]
BASHUNIT_COVERAGE_SHOW_UNCOVERED=true bashunit tests/ --coverage
```
```bash [Both]
BASHUNIT_COVERAGE_SHOW_FUNCTIONS=true \
BASHUNIT_COVERAGE_SHOW_UNCOVERED=true \
bashunit tests/ --coverage
```
:::

The default text report stays compact; opt in only when triaging.

### Worked Example

Given `src/route.sh`:

```bash
#!/usr/bin/env bash
function route() {
if [ "$1" = "GET" ]; then
echo "fetch"
elif [ "$1" = "POST" ]; then
echo "create"
else
echo "405"
fi
}
```

If tests only call `route GET`, the LCOV record looks like:

```
TN:
SF:/path/to/src/route.sh
FN:2,route
FNDA:1,route
FNF:1
FNH:1
BRDA:3,0,0,1
BRDA:3,0,1,0
BRDA:3,0,2,0
BRF:3
BRH:1
DA:3,1
DA:4,1
DA:5,0
DA:6,0
DA:7,0
DA:8,0
LF:6
LH:2
end_of_record
```

**Reading the branch records:**
- `BRDA:3,0,0,1`: decision on line 3, block 0, arm 0 (`then`/GET), taken.
- `BRDA:3,0,1,0`: same decision, arm 1 (`elif`/POST), not taken.
- `BRDA:3,0,2,0`: same decision, arm 2 (`else`/405), not taken.
- `BRF:3` `BRH:1`: 3 branches found, 1 taken.

### Visualizing with genhtml

LCOV's `genhtml` renders branch coverage alongside line and function coverage:

::: code-group
```bash [Generate]
bashunit tests/ --coverage
genhtml --branch-coverage coverage/lcov.info -o coverage/html
```
:::

The resulting site shows a red/green diamond next to each branch decision, mirroring `gcov`'s C/C++ output.

### CI Integration

Codecov and Coveralls pick up the new records without configuration. To require branch coverage in PR gates:

::: code-group
```yaml [Codecov]
coverage:
status:
project:
default:
target: 80%
patch:
default:
target: 80%
threshold: 0%
flags:
- branch
```
:::

### Limitations

- An arm whose body has no executable lines (only comments or braces) registers as not-taken even when the conditional fired.
- Implicit `else` (an `if`/`elif` chain without an explicit `else`) reports only the explicit arms; the synthetic fall-through outcome is omitted.
- Compound conditionals (`if A && B`) are reported as a single binary decision, not per sub-expression.
- `&&`/`||` short-circuit branches outside `if` and loop-entry decisions (`while`/`until`) are not tracked.

See `adrs/adr-007-branch-coverage-mvp.md` for the design rationale and the rejected alternatives.

## Limitations

### External Commands
Expand Down
Loading
Loading