Skip to content

test: add mutation testing coverage for survived and untested mutants#76

Closed
Copilot wants to merge 7 commits intomainfrom
copilot/add-test-coverage-for-mutants
Closed

test: add mutation testing coverage for survived and untested mutants#76
Copilot wants to merge 7 commits intomainfrom
copilot/add-test-coverage-for-mutants

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 8, 2026

Adds targeted tests to kill mutants identified by mutation testing reports. All changes are test-only — no source code was modified.

Protocol & Models

  • Direct tests for _decode_temperature, _check_length, _encode_temperature (mod-2 mutant), _decode_temp_unit, _decode_flame_effect (bitwise AND vs OR), _encode_heat_settings (boost boundary)
  • kebab_name tests verifying lowercase + underscore-to-hyphen conversion

Auth

  • Direct calls to _build_app() verifying each MSAL constructor kwarg individually (CLIENT_ID, AUTHORITY, validate_authority=False, token_cache)
  • Direct calls to _save_cache() verifying mkdir(parents=True, exist_ok=True) and write_text content
  • Logger format/arg verification via caplog

Client

  • Parametrized _parse_fire_features tests covering all 24 feature fields individually (catches key-name and default-value mutations)
  • get_fire_overview feature sourcing tests (FireDetails vs WifiFireOverview fallback)
  • Field default tests, status boundary checks (199/200/299/300), write_parameters payload structure

CLI

  • build_parser description/help text assertions
  • Precise _display_* output verification (timer arithmetic // 60 / % 60, error hex/binary formatting, feature labels)
  • _masked_input terminal handling (asterisks, backspace sequences, stdin.read(1) calls)

TUI Widgets

  • Exact label string assertions for all _format_* functions (kills XX-prefix/suffix and case mutations)
  • _build_fire_art structural verification (frame elements, flame chars, LED/media styles, heat wave chars)
  • _expand_flame gap distribution logic

TUI App

  • _resolve_version subprocess.run arg verification for all three git commands (describe, rev-parse, status) including kwargs (capture_output=True, text=True, timeout=2) and all branch paths

Remaining mutants

~3,400 mutants remain labeled "no tests" by mutmut due to its test selection algorithm failing to map integration-style tests to specific source functions. The code paths are exercised by existing tests but mutmut cannot associate them. Fixing this requires mutmut configuration changes or source modifications, not additional tests.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 6 commits March 8, 2026 15:26
Add TestKebabName class to test_models.py for kebab_name() mutants.
Add test classes to test_protocol.py for _decode_temperature,
_check_length, _encode_temperature, _decode_temp_unit,
_decode_flame_effect, and _encode_heat_settings mutants.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add TestTokenAuthCallableMutant, TestBuildAppDirect, and TestSaveCacheDirect
test classes that call methods directly (bypassing asyncio.to_thread) to
defeat 27 surviving mutation testing mutants in auth.py.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add comprehensive direct unit tests for _parse_fire_features,
get_fire_overview, get_fires, _request, write_parameters, turn_on,
turn_off, and __init__/__aenter__/__aexit__ to enable mutmut v3
test selection to associate tests with specific functions.

Key additions:
- TestParseFireFeaturesDirect: parametrized tests for all 24 features
- TestGetFireOverviewFeatureSource: FireDetails vs WifiFireOverview fallback
- TestGetFiresFieldMapping: every Fire field mapping verified
- TestRequestBoundaryStatus: status 199/200/299/300 boundary checks
- TestWriteParametersPayload: exact payload structure verification
- TestTurnOnDirect/TestTurnOffDirect: mode and temperature assertions
- TestContextManagerDirect: session creation and lifecycle
- TestGetFireOverviewFieldDefaults: individual field default tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add 112 new tests targeting survived mutants in cli.py:

- build_parser: verify description text, help strings, argument
  configs, and subparser names (kills ~81 mutants)
- _display_* functions: precise output assertions for headers,
  labels, format strings, and arithmetic operations (kills ~56 mutants)
- _masked_input: verify prompt writing, asterisk output, backspace
  handling, terminal settings save/restore, and flush calls
  (kills ~23 mutants)
- main: verify logging level selection and args forwarding
  (kills ~2 mutants)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add ~60 new tests targeting survived mutants across all format functions,
_expand_flame, _build_fire_art, and _ClickableValue. Tests verify exact
label strings, action names, value formatting, item counts, and ASCII art
structural elements.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add 12 tests verifying exact subprocess.run arguments, kwargs,
branch conditions, strip() application, and fallback paths to kill
58 survived mutants in the TUI app._resolve_version function.

Co-authored-by: deviantintegral <255023+deviantintegral@users.noreply.github.com>
Copilot AI changed the title [WIP] Add additional test coverage to address mutation testing test: add mutation testing coverage for survived and untested mutants Mar 8, 2026
@deviantintegral
Copy link
Copy Markdown
Owner

@copilot compare this PR against PR #78. Report on what the differences are, and suggest which one is the better version, if either.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 9, 2026

@copilot compare this PR against PR #78. Report on what the differences are, and suggest which one is the better version, if either.

Here's my comparison of this PR (#80) vs PR #78:

Size: PR #80 is larger (+3,114 lines across 7 files) vs PR #78 (+1,786/-190 across 9 files).

Overlapping coverage (both PRs target the same mutants):

Unique to PR #78:

  • test_fireplace_visual.py (+491 lines, new file) — Dedicated tests for _expand_flame and _build_fire_art covering heat rows, styles, and flame geometry in a standalone file.
  • test_b2c_login.py (+7) — Exact log record assertions.
  • test_tui_screens.py (+2/-1) — Minor fix.

Unique to PR #80:

  • test_auth.py (+275 lines) — TestBuildAppDirect, TestSaveCacheDirect, TestTokenAuthCallableMutant — direct unit tests calling _build_app() and _save_cache() without going through get_token().

Quality differences:

CI status: PR #78 passes all CI checks. This PR (#80) also passes.

Recommendation: PR #78 is the better version. It's more concise (-1,328 fewer added lines), takes a surgical approach of tightening existing tests rather than duplicating setup, and covers test_fireplace_visual.py as a dedicated test file. PR #80's main advantage is the test_auth.py coverage (27 mutants) and more granular CLI help text tests, which could be cherry-picked into PR #78 if desired.

deviantintegral added a commit that referenced this pull request Mar 9, 2026
Cherry-pick auth tests from Copilot PR #76 that complement our existing
mutation testing coverage with direct unit tests for auth internals.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
deviantintegral added a commit that referenced this pull request Mar 9, 2026
Cherry-pick auth tests from Copilot PR #76 that complement our existing
mutation testing coverage with direct unit tests for auth internals.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants