Skip to content

Add fixture tag selection#42

Merged
mavam merged 7 commits into
mainfrom
topic/fixture-selection
May 11, 2026
Merged

Add fixture tag selection#42
mavam merged 7 commits into
mainfrom
topic/fixture-selection

Conversation

@mavam
Copy link
Copy Markdown
Member

@mavam mavam commented May 11, 2026

🔍 Problem

  • --match can select tests by path, but there was no stable way to select tests by fixture behavior.
  • Running only container-backed or Docker Compose tests required relying on naming conventions.

🛠️ Solution

  • Add --fixture-tag and the matching fixture_tags Python API parameter.
  • Add fixture tag metadata with inference from tagged abstractions.
  • Tag container_runtime as container and the built-in Docker Compose fixture as container and docker-compose.
  • Add a changelog entry for the new selector.

💬 Review

  • The main design point is that tags come from abstractions by default, while explicit tags= remains available for custom fixtures.
  • --fixture-tag intersects with path and --match selection and expands suites the same way as --match.

Checks:

uv run ruff check
uv run ruff format --check
uv run mypy
uv run pytest
📚 Docs PR: tenzir/docs#305

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 73a9a67ef6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/tenzir_test/run.py Outdated
@mavam mavam force-pushed the topic/fixture-selection branch from 73a9a67 to a84acf4 Compare May 11, 2026 18:10
Allow test runs to filter by fixture tags with -F/--fixture-tag. Tags are inferred from tagged fixture abstractions, with container_runtime providing the container tag and docker-compose carrying both container and docker-compose tags.

Assisted-by: GPT-5 (Codex)
@mavam mavam force-pushed the topic/fixture-selection branch from a84acf4 to 76135bb Compare May 11, 2026 18:10
Keep tests with malformed configuration in the selected set when filtering by fixture tag. The normal execution path now reports those parse errors instead of silently dropping them from targeted runs.

Assisted-by: GPT-5 (Codex)
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

collected_paths = _expand_suites(collected_paths)

P1 Badge Apply all selectors before expanding suites

Expanding suites immediately after --match filtering lets tests bypass intersection semantics when --fixture-tag is also used: if test A in a suite matches the path/pattern selector and test B in the same suite matches the fixture tag selector, the first expansion adds B before tag filtering, then the second expansion re-adds A, so the suite runs even though no individual test matched both selectors. This contradicts the documented "intersection" behavior for combined selectors and can make targeted runs execute unrelated tests.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

mavam added 3 commits May 11, 2026 20:34
Filter by path, match pattern, and fixture tag before expanding suites. This preserves documented intersection semantics for combined selectors.

Assisted-by: GPT-5 (Codex)
Infer fixture tags from tagged provider modules as well as directly tagged functions. This keeps direct imports from the shared container runtime helpers selectable with --fixture-tag container.

Assisted-by: GPT-5 (Codex)
Record the pull request number in the unreleased fixture tag selection changelog metadata.

Assisted-by: GPT-5 (Codex)
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 39584e9db7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/tenzir_test/run.py Outdated
Fixture-tag prefiltering now preserves tests whose frontmatter raises a YAML parser error, so tagged runs queue the test and report the configuration failure instead of aborting discovery.

Assisted-by: GPT-5 (Codex)
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c853bb4744

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/tenzir_test/fixtures/__init__.py
Treat a bare string passed to fixture tag registration as one tag instead of iterating it character by character. This keeps --fixture-tag selection working for common custom fixture declarations.

Assisted-by: GPT-5 (Codex)
@mavam mavam merged commit 84cb79f into main May 11, 2026
3 checks passed
@mavam mavam deleted the topic/fixture-selection branch May 11, 2026 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant