FEAT: Replace trial cloning with configurable trial populations

[behnam-o]

[This change and #121 are related and will conflict with each other. will publish it after the other one is merged]

Summary

Replace pytest item cloning for @pytest.mark.trial with an explicit
trial_config fixture.

Tests now control population execution while RAMPART supplies:

• The marker-declared population size and threshold
• A --rampart-trials N CLI override for population size
• Collection-time validation of trial declarations
• An error when a trial-marked test does not consume trial_config

This avoids clone-specific pytest and xdist behavior while preserving
population-level result reporting and legacy worker-payload compatibility.

Changes

• Add the immutable TrialConfig public type
• Add the trial_config fixture
• Add the --rampart-trials N option
• Remove collection-time test cloning
• Validate trial markers during collection
• Update xdist tests for one-item, multiple-result populations
• Update trial documentation and examples

Example

@pytest.mark.trial(n=10, threshold=0.8)
async def test_injection_resistance(adapter, trial_config):
    results = [
        await execution.execute_async(adapter=adapter)
        for _ in range(trial_config.n)
    ]

    pass_rate = sum(result.safe for result in results) / trial_config.n
    assert pass_rate >= trial_config.threshold

[Copilot]

Pull request overview

This PR updates RAMPART’s pytest @pytest.mark.trial semantics to stop cloning pytest items and instead supply trial population configuration through a dedicated trial_config fixture (with an optional --rampart-trials N CLI override). This avoids clone/xdist scheduling edge-cases while keeping population-style reporting.

Changes:

• Introduces an immutable TrialConfig type and trial_config fixture, plus --rampart-trials N to override population depth (n) without changing the declared threshold.
• Removes collection-time trial cloning and replaces it with collection-time validation (including enforcing that trial-marked tests consume trial_config).
• Updates unit/integration/xdist tests and refreshes documentation/examples to reflect population execution being controlled by the test.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/unit/pytest_plugin/test_xdist_aggregation.py	Updates xdist aggregation tests to reflect “single pytest item that records multiple Results” populations.
tests/unit/pytest_plugin/test_trial.py	Adds unit tests for trial marker parsing/validation and CLI override resolution.
tests/unit/pytest_plugin/test_trial_integration.py	Adds subprocess/integration tests verifying fixture behavior and failure modes.
tests/unit/pytest_plugin/test_plugin.py	Removes cloning-related unit tests that no longer apply.
rampart/reporting/sink.py	Updates population_summary docstring to reflect multi-Result-per-test behavior.
rampart/pytest_plugin/plugin.py	Adds --rampart-trials, adds trial_config fixture, removes cloning logic, and enforces trial marker validation + fixture consumption.
rampart/pytest_plugin/_trial.py	New module implementing TrialConfig, CLI parsing helper, and marker/override resolution.
rampart/pytest_plugin/_session.py	Updates docstrings to clarify trial-spec paths are legacy/compatibility only.
rampart/pytest_plugin/init.py	Exposes TrialConfig on the public pytest plugin API.
pyproject.toml	Updates marker description to the new population/fixture semantics.
docs/usage/xdist.md	Updates xdist guidance to reflect that @trial no longer affects scheduling; adds --rampart-trials guidance.
docs/usage/pytest-integration.md	Updates marker semantics, adds trial_config fixture docs, updates examples and sample output.
docs/usage/configuration.md	Documents --rampart-trials N alongside other pytest options.
docs/usage/ci-integration.md	Updates CI guidance/examples to use trial_config and CLI override.
docs/usage/authoring-tests.md	Updates authoring guidance/examples to show trial_config usage in class-based tests.
docs/probes/behavioral.md	Updates probe example to execute a population via trial_config.
docs/glossary.md	Updates “Trial” definition to match population + trial_config semantics.
docs/getting-started/quickstart.md	Updates quickstart example/output to use trial_config and population execution pattern.
docs/getting-started/installation.md	Updates expected marker help output text.
docs/concepts/overview.md	Updates overview wording for trial marker semantics.
docs/attacks/xpia.md	Updates attack example to execute a population via trial_config.
docs/api/pytest-plugin.md	Adds _trial module docs entry (but currently misreferences TrialGroupResult).