Skip to content

test: eliminate testing theater and improve test assertions - #33

Merged
loadinglucian merged 1 commit into
mainfrom
fix/eliminate-testing-theater
Oct 10, 2025
Merged

test: eliminate testing theater and improve test assertions#33
loadinglucian merged 1 commit into
mainfrom
fix/eliminate-testing-theater

Conversation

@loadinglucian

@loadinglucian loadinglucian commented Oct 10, 2025

Copy link
Copy Markdown
Owner

Replaced weak assertions that only check 'something returned' with strong assertions that verify correct behavior and valid data.

Changes:

  • VersionServiceTest: Verify version matches valid patterns (semver, git-describe, commit hash)
  • FilesystemServiceTest: Verify getCwd() returns actual valid directory path
  • ConsoleOutputTraitTest: Remove arbitrary length assertion (fragile)
  • TestHelpersTest: Execute command to verify proper configuration
  • ServerRepositoryTest: Test behavior instead of just types

Impact:

  • Before: 7 tests could pass even if functionality broke
  • After: 0 tests with testing theater

All 267 tests passing with 82.8% coverage maintained.

Summary by CodeRabbit

  • Tests
    • Strengthened repository tests by validating server names in results.
    • Enhanced filesystem tests to ensure the working directory is a valid directory.
    • Updated version tests to accept semantic, branch-prefixed, or commit-hash formats.
    • Refined test helpers to focus on file content and verify command execution with success exit codes.
    • Simplified console output tests for separator rendering.

No user-facing features changed; these updates improve test coverage and reliability.

Replaced weak assertions that only check 'something returned' with
strong assertions that verify correct behavior and valid data.

Changes:
- VersionServiceTest: Verify version matches valid patterns (semver, git-describe, commit hash)
- FilesystemServiceTest: Verify getCwd() returns actual valid directory path
- ConsoleOutputTraitTest: Remove arbitrary length assertion (fragile)
- TestHelpersTest: Execute command to verify proper configuration
- ServerRepositoryTest: Test behavior instead of just types

Impact:
- Before: 7 tests could pass even if functionality broke
- After: 0 tests with testing theater

All 267 tests passing with 82.8% coverage maintained.
@coderabbitai

coderabbitai Bot commented Oct 10, 2025

Copy link
Copy Markdown
Contributor

Walkthrough

Test updates adjust assertions across repository, services, helpers, and console trait tests: shifting from type checks to value checks, tightening validity conditions (cwd, version pattern), expanding command testing with CommandTester and exit code assertions, and simplifying an output assertion. No production code or public APIs were changed.

Changes

Cohort / File(s) Summary of changes
Server repository tests
tests/Unit/Repositories/ServerRepositoryTest.php
Changed assertion from instance-of ServerDTO to checking first element’s name equals web1; retained subsequent name assertions.
Filesystem service tests
tests/Unit/Services/FilesystemServiceTest.php
Replaced non-empty string check for getCwd() with directory existence check via isDirectory, asserting truthiness with a message.
Version service tests
tests/Unit/Services/VersionServiceTest.php
Replaced generic non-empty version assertion with regex-based assertion supporting semver, dev/branch prefixes, and git hashes; also asserts non-empty.
Test helpers
tests/Unit/TestHelpersTest.php
Removed existence assertions around mocked filesystem; focused on readFile content. Expanded mockCommandContainer test: arranged inputs, executed via Symfony CommandTester, asserted command instance type and exitCode === Command::SUCCESS.
Console output trait tests
tests/Unit/Traits/ConsoleOutputTraitTest.php
Simplified separator line assertion to only require presence of ╭───────; removed output length check.

Sequence Diagram(s)

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • bigpixelrocket/deployer-php#23 — Touches the same ConsoleOutputTrait test file; both simplify the separator assertion.
  • bigpixelrocket/deployer-php#22 — Modifies the same FilesystemService cwd test to validate directory existence.
  • bigpixelrocket/deployer-php#25 — Introduces/affects ServerRepository and ServerDTO behavior referenced by the updated ServerRepositoryTest assertions.

Poem

I thump my paws, tests hop in line,
From types to names, the checks refine.
Cwd’s a burrow, version’s a trail,
Commands return success without fail.
A subtle curve, “╭───────” on display—
Green carrots gleam: all tests OK! 🥕✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title concisely highlights the primary change of removing ineffective “testing theater” and strengthening test assertions, matching the PR objectives. It clearly communicates the focus on improving test validity without extraneous detail. A teammate scanning history can quickly understand the main purpose of the changeset.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix/eliminate-testing-theater

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1f61497 and f82e64e.

📒 Files selected for processing (5)
  • tests/Unit/Repositories/ServerRepositoryTest.php (1 hunks)
  • tests/Unit/Services/FilesystemServiceTest.php (1 hunks)
  • tests/Unit/Services/VersionServiceTest.php (1 hunks)
  • tests/Unit/TestHelpersTest.php (1 hunks)
  • tests/Unit/Traits/ConsoleOutputTraitTest.php (1 hunks)
🧰 Additional context used
📓 Path-based instructions (3)
**/*.php

📄 CodeRabbit inference engine (.cursor/rules/00-main.mdc)

**/*.php: Eliminate single-use private methods by inlining them directly
Cache expensive computed values by initializing them in the constructor instead of recomputing
Prefer direct property access over method calls when appropriate to avoid call overhead
Organize code into comment-separated sections; prefer alphabetical ordering when it does not conflict with logical grouping

**/*.php: Adhere to PSR-12 coding style in all PHP files
Declare strict_types=1 at the top of every PHP file
Prefer PHP 8.x features (union types, match, attributes, readonly) where appropriate
Always import classes with use statements; avoid fully qualified class names in code bodies
All methods must declare explicit return types; use proper generics in types/docblocks (e.g., Collection<int, User>)
Use dependency injection instead of manually resolving or instantiating classes
Prefer Symfony component classes (e.g., Filesystem, Process) over native PHP functions for testability
All object creation must use $container->build(ClassName::class) instead of new, except for value objects/DTOs/pure data structures
In production code, access the Container via constructor injection, not via static/global access
Add minimal DocBlock comments with descriptions, parameters, and return types for classes and functions
Use comments as visual separators for sections/subsections with a single newline between header, subheader, and paragraph; avoid obvious or stale comments
Run rector on changed PHP files before completing a task
Run pint on changed PHP files to fix code style before completing a task

Files:

  • tests/Unit/Services/FilesystemServiceTest.php
  • tests/Unit/Services/VersionServiceTest.php
  • tests/Unit/TestHelpersTest.php
  • tests/Unit/Traits/ConsoleOutputTraitTest.php
  • tests/Unit/Repositories/ServerRepositoryTest.php
{tests/**,test/**,**/*@(Test|Spec).php}

📄 CodeRabbit inference engine (.cursor/rules/00-main.mdc)

Do not run or edit tests unless explicitly instructed

Files:

  • tests/Unit/Services/FilesystemServiceTest.php
  • tests/Unit/Services/VersionServiceTest.php
  • tests/Unit/TestHelpersTest.php
  • tests/Unit/Traits/ConsoleOutputTraitTest.php
  • tests/Unit/Repositories/ServerRepositoryTest.php
tests/**/*.php

📄 CodeRabbit inference engine (.cursor/rules/01-architecture.mdc)

tests/**/*.php: In tests, direct Container instantiation and bind() for mocks is allowed and encouraged for isolation
Do not run PHPStan on test files; tests are excluded from static analysis

tests/**/*.php: Unit tests must instantiate services manually (no DI container)
Command/integration tests must use mockCommandContainer() for building commands and overriding services
Only use container auto-wiring in tests to verify DI configuration or multi-service integration (edge cases)
Keep test files under 1.8x the size of the source they test (without sacrificing readability)
Test core business logic; avoid testing the framework itself
Prefer dataset-driven testing using ->with([...]) for multiple scenarios
Consolidate related assertions (e.g., expect($x)->toBe(...)->and($y)->toBe(...))
Mock only external dependencies; keep unit tests isolated from filesystem/HTTP/processes
Avoid performance tests unless performance is the primary concern
Use the AAA pattern in tests (Arrange, Act, Assert; optional Cleanup)
In exception tests, use a combined // ACT & ASSERT step when the act triggers the assertion
Organize tests with describe() blocks, beforeEach() setup, and shared helpers/traits for DRY
Forbidden assertions in tests: type-only or generic checks (e.g., toBeInstanceOf, toBeArray, not->toBeNull, expect(true)->toBeTrue) and sleep(...); prefer time mocking
Preferred assertions: assert observable behavior and interactions (e.g., domain values, validator outcomes, mock expectations)
Unit tests: mock all external dependencies, test single units in isolation, and complete in milliseconds
Integration tests: use real file operations and external processes; cover CLI commands and full workflows
Do not require PHPStan compliance in tests; avoid excessive phpdoc solely to satisfy types in tests

Files:

  • tests/Unit/Services/FilesystemServiceTest.php
  • tests/Unit/Services/VersionServiceTest.php
  • tests/Unit/TestHelpersTest.php
  • tests/Unit/Traits/ConsoleOutputTraitTest.php
  • tests/Unit/Repositories/ServerRepositoryTest.php
🧠 Learnings (6)
📓 Common learnings
Learnt from: CR
PR: bigpixelrocket/deployer-php#0
File: .cursor/rules/02-tests.mdc:0-0
Timestamp: 2025-10-10T11:10:25.074Z
Learning: Applies to tests/**/*.php : Preferred assertions: assert observable behavior and interactions (e.g., domain values, validator outcomes, mock expectations)
Learnt from: CR
PR: bigpixelrocket/deployer-php#0
File: .cursor/rules/02-tests.mdc:0-0
Timestamp: 2025-10-10T11:10:25.074Z
Learning: Applies to tests/**/*.php : Consolidate related assertions (e.g., expect($x)->toBe(...)->and($y)->toBe(...))
📚 Learning: 2025-10-10T11:10:25.074Z
Learnt from: CR
PR: bigpixelrocket/deployer-php#0
File: .cursor/rules/02-tests.mdc:0-0
Timestamp: 2025-10-10T11:10:25.074Z
Learning: Applies to tests/**/*.php : Mock only external dependencies; keep unit tests isolated from filesystem/HTTP/processes

Applied to files:

  • tests/Unit/TestHelpersTest.php
📚 Learning: 2025-10-10T11:10:25.074Z
Learnt from: CR
PR: bigpixelrocket/deployer-php#0
File: .cursor/rules/02-tests.mdc:0-0
Timestamp: 2025-10-10T11:10:25.074Z
Learning: Applies to tests/TestHelpers.php : When BaseCommand gains a new service, update mockCommandContainer() in tests/TestHelpers.php by adding a parameter, constructing/binding it, and wiring it to the container

Applied to files:

  • tests/Unit/TestHelpersTest.php
📚 Learning: 2025-10-10T11:10:25.074Z
Learnt from: CR
PR: bigpixelrocket/deployer-php#0
File: .cursor/rules/02-tests.mdc:0-0
Timestamp: 2025-10-10T11:10:25.074Z
Learning: Applies to tests/**/*.php : Command/integration tests must use mockCommandContainer() for building commands and overriding services

Applied to files:

  • tests/Unit/TestHelpersTest.php
📚 Learning: 2025-10-10T11:09:41.967Z
Learnt from: CR
PR: bigpixelrocket/deployer-php#0
File: .cursor/rules/01-architecture.mdc:0-0
Timestamp: 2025-10-10T11:09:41.967Z
Learning: Applies to tests/**/*.php : In tests, direct Container instantiation and bind() for mocks is allowed and encouraged for isolation

Applied to files:

  • tests/Unit/TestHelpersTest.php
📚 Learning: 2025-10-10T11:10:25.074Z
Learnt from: CR
PR: bigpixelrocket/deployer-php#0
File: .cursor/rules/02-tests.mdc:0-0
Timestamp: 2025-10-10T11:10:25.074Z
Learning: Applies to tests/**/*.php : Unit tests: mock all external dependencies, test single units in isolation, and complete in milliseconds

Applied to files:

  • tests/Unit/TestHelpersTest.php
🧬 Code graph analysis (3)
tests/Unit/Services/FilesystemServiceTest.php (1)
app/Services/FilesystemService.php (1)
  • isDirectory (88-91)
tests/Unit/TestHelpersTest.php (3)
tests/TestHelpers.php (1)
  • mockCommandContainer (373-401)
app/Container.php (1)
  • build (59-85)
tests/Fixtures/TestConsoleCommand.php (2)
  • TestConsoleCommand (25-236)
  • execute (61-92)
tests/Unit/Repositories/ServerRepositoryTest.php (1)
app/Repositories/ServerRepository.php (1)
  • all (83-93)
🔇 Additional comments (6)
tests/Unit/Traits/ConsoleOutputTraitTest.php (1)

129-129: Appropriate simplification to remove fragile constraint.

Checking for the presence of the box-drawing header ╭─────── is sufficient to verify that the separator renders correctly. Removing the arbitrary length constraint (> 40) makes the test more resilient to formatting changes without sacrificing meaningful validation.

tests/Unit/Repositories/ServerRepositoryTest.php (1)

57-57: Excellent shift from type-checking to value-based assertion.

Verifying that the first element's name is 'web1' is far more meaningful than merely checking the type. This change ensures the test validates actual behavior and would catch data corruption or ordering issues that the previous type-only assertion would miss.

As per coding guidelines: preferred assertions focus on observable behavior and domain values rather than type-only checks.

tests/Unit/Services/VersionServiceTest.php (1)

15-17: Strong improvement using regex pattern validation.

The new assertion validates that versions conform to expected formats (semantic versions, branch prefixes, or git commit hashes) rather than merely checking for non-emptiness. This regex-based validation would catch malformed versions that the previous weak assertion (not->toBeEmpty) would have missed entirely.

Pattern coverage:

  • Semantic versions: v1.2.3 or 1.2.3
  • Branch prefixes: dev-, main-, master-
  • Git hashes: 7-40 hexadecimal characters
tests/Unit/Services/FilesystemServiceTest.php (1)

83-85: Excellent validation of actual filesystem state.

The new assertion verifies that getCwd() returns a valid, existing directory path rather than merely checking it's a non-empty string. This stronger validation ensures the method returns meaningful, correct data that can actually be used for filesystem operations.

tests/Unit/TestHelpersTest.php (2)

166-166: Appropriate focus on content validation.

Removing the existence assertion and focusing solely on content validation ('test content') is more meaningful. The content assertion implicitly verifies that the file can be read (existence), while also validating the actual data returned by the mock.


172-182: Excellent addition of command execution verification.

The expanded test now properly verifies that commands built by mockCommandContainer() are not just structurally correct but actually executable. Testing the exit code (Command::SUCCESS) ensures the command completes successfully, which is far more meaningful than merely checking the instance type.

This aligns perfectly with the PR's goal of testing observable behavior rather than structural properties.


Comment @coderabbitai help to get the list of available commands and usage tips.

@loadinglucian
loadinglucian merged commit 2ebe602 into main Oct 10, 2025
5 checks passed
@loadinglucian
loadinglucian deleted the fix/eliminate-testing-theater branch October 10, 2025 11:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant