Implement filter functionality

[roxblnfk]

Test Filtering Feature

This document describes the test filtering functionality in Testo, which allows selective test execution based on various criteria.

Overview

Testo provides a flexible filtering system that operates in multiple stages:

1. Stage 1 (Suite Filter): Determines configuration scopes that define file scanning locations
2. Stage 2 (Path Filter): Applied at the finder level - scans directories and locates files
3. Stage 3 (File Filter): Pre-filters test files before loading them for reflection analysis
4. Stage 4 (Test Filter): Filters individual test cases and methods after reflection analysis

This multi-stage approach optimizes performance by progressively narrowing down the test set, skipping unnecessary operations at each level.

Filter Types

The filtering system supports three types of filters, each working independently:

1. Name Filter (--filter)

Filters tests by method or function names. Supports three formats:

• FQN (Fully Qualified Name): Namespace\ClassName or Namespace\functionName
• Method: ClassName::methodName or Namespace\ClassName::methodName
• Fragment: Simple names like methodName, functionName, or ShortClassName

2. Path Filter (--path)

Filters test files by glob patterns with wildcard support (*, ?, [abc]).

3. Suite Filter (--suite)

Filters tests by test suite name.

Filter Logic

Combining Multiple Filters

• Same type filters use OR logic: Multiple values of the same filter type are combined with OR

  • Example: --filter=test1 --filter=test2 matches tests where name is test1 OR test2

• Different type filters use AND logic: Different filter types are combined with AND

  • Example: --filter=test1 --path="tests/Unit/*" matches tests where name is test1 AND file is in tests/Unit/

Formula: AND(OR(filters), OR(paths), OR(suites))

Name Filter Behavior

When filtering by name (--filter), the behavior depends on the filter format:

Method Format (ClassName::methodName)

When using the method format with :: separator:

• Only checks if the specified method exists in the matching class
• Result: Test case is included with only the specified method
• Other methods in the same class are excluded
• Example: --filter=UserTest::testLogin includes only testLogin method from UserTest class

FQN or Fragment Format

When using FQN (with \) or simple fragment (no separators):

1. Class name match: If the test case class name matches the filter

   • Result: Entire test case is included with all its test methods

2. Class name doesn't match: If the test case class name doesn't match

   • The system then checks individual methods/functions
   • If any methods/functions match: Test case is included with only matched methods
   • If no methods/functions match: Test case is skipped entirely

Examples:

• --filter=UserTest → includes entire UserTest class with all methods
• --filter=Tests\Unit\UserTest → includes entire class with all methods
• --filter=testLogin → includes any class that has testLogin method, with only that method

Usage Examples

Basic Filtering

# Run all tests in default location
./bin/testo run

# Run tests from specific directory
./bin/testo run tests/Unit

# Run tests matching glob patterns (wildcards supported)
./bin/testo run --path="tests/Unit/*Test.php" --path="tests/Integration/*Test.php"

Filter by Test Name

# Filter specific test methods or functions by name (OR logic)
./bin/testo run --filter=testUserAuthentication --filter=testDatabaseConnection

# Filter specific methods in classes (using short name or FQN)
./bin/testo run --filter="UserTest::testAuthentication"
./bin/testo run --filter="Tests\Unit\UserTest::testAuthentication"

Filter by Suite

# Filter by test suite name (OR logic)
./bin/testo run --suite=Unit --suite=Integration

Complex Filtering

# Combine filters with AND logic between types
# Runs tests that match (UserTest::testCreate OR UserTest::testUpdate) AND (Critical suite)
./bin/testo run --filter=UserTest::testCreate --filter=UserTest::testUpdate --suite=Critical

# Complex filtering: path AND filter AND suite
# Runs tests in Unit directory that match testImportant AND are in Critical suite
./bin/testo run --path="tests/Unit/*" --filter=testImportant --suite=Critical

CI Integration

# Run with TeamCity output format for CI
./bin/testo run --teamcity

# Run tests with custom config
./bin/testo run --config=./testo.php

---

Implementation Details

Multi-Stage Filtering Pipeline

Stage 1: Suite Filter (Configuration Level)

Applied at the configuration level to determine which test suites to run:

• Filters configuration scopes based on --suite option
• Each suite defines file scanning locations and patterns
• Determines the initial set of directories to scan

Stage 2: Path Filter (Finder Level)

Applied at the file finder level during directory scanning:

• Uses --path option with glob patterns
• Matches file paths against wildcard patterns (*, ?, [abc])
• Returns list of files to be processed
• Works with Symfony Finder component

Stage 3: File Filter (FilterInterceptor - FileLocatorInterceptor)

Implemented in FilterInterceptor::locateFile():

public function locateFile(TokenizedFile $file, callable $next): ?bool

• Performs quick pre-filtering based on tokenized file data
• Skips files that don't contain any matching classes, methods, or functions
• Uses lightweight tokenization instead of full reflection for performance
• Checks against --filter patterns before loading reflections

Stage 4: Test Filter (FilterInterceptor - CaseLocatorInterceptor)

Implemented in FilterInterceptor::locateTestCases():

public function locateTestCases(FileDefinitions $file, callable $next): CaseDefinitions

• Filters loaded test definitions based on class and method names
• Implements the hierarchical filtering logic:
  • For method format (::) - filters specific methods only
  • For FQN/fragment format - checks class name first, then methods
• Returns filtered test case definitions ready for execution

Pattern Matching

The filter uses whole-word matching with regex:

private static function has(string $needle, string $haystack): bool
{
    return \preg_match('/\\b' . \preg_quote($needle, '/') . '\\b$/', $haystack) === 1;
}

This ensures:

• User matches App\User but not App\UserManager
• test matches testMethod but not latestMethod

Performance Considerations

1. Multi-stage progressive filtering: Each stage narrows down the test set before moving to the next, more expensive operation
2. Suite-level pre-filtering: Stage 1 limits the scope of directories to scan based on configuration
3. Path-based file selection: Stage 2 uses efficient glob matching to select files before parsing
4. Tokenization over reflection: Stage 3 uses lightweight tokenization instead of full reflection for initial filtering
5. Early exit: Returns immediately on first match in file filtering stage
6. Lazy evaluation: Only loads and processes files that pass Stages 2 and 3
7. Hierarchical test filtering: Stage 4 checks class names before iterating through methods

Command Reference

Arguments

• path (optional): Path to test directory or file

Options

• --filter: Filter methods or functions to be run (repeatable)

  • Supports methodName, ClassName::methodName, and Namespace\ClassName::methodName formats

• --path: Glob patterns for test files to be run (repeatable)

  • Supports wildcards: *, ?, [abc]

• --suite: Filter test suites by name (repeatable)

• --teamcity: Enable TeamCity CI output format

• --config: Path to the configuration file

Examples by Use Case

Run specific test method

./bin/testo run --filter=UserTest::testLogin

Run all tests in a class

./bin/testo run --filter=UserTest

Run tests across multiple files

./bin/testo run --path="tests/Unit/User*Test.php" --path="tests/Unit/Auth*Test.php"

Run critical tests in specific directory

./bin/testo run --path="tests/Unit/*" --suite=Critical

Run specific methods from multiple classes

./bin/testo run --filter=UserTest::testLogin --filter=AuthTest::testLogin

CI/CD Pipeline

# Run unit tests with TeamCity reporting
./bin/testo run --suite=Unit --teamcity

# Run integration tests with specific config
./bin/testo run --suite=Integration --config=./ci-testo.php --teamcity