fix(tests): fix integration test filter with dynamic job determination

[njbrake]

Description

Claude Code helped me with this refactor and the nice explanation. I've never used the github 'actions' folder before but this seems like a nice way to organize the logic to avoid the workflow file blowing up. I also opted to put it into the action.yml instead of elevating it to a separate shell script because it seemed like logic that was only relevant for github actions, and wouldn't be used by people outside of that env.

Problem

The GitHub Actions workflow filtering logic was causing integration tests to be skipped incorrectly when using the filter input parameter with workflow_dispatch.

Root Cause

The workflow used simple substring matching with contains() to determine whether jobs should run:

if: github.event.inputs.filter == '' || contains(needs.expected-providers.outputs.providers, github.event.inputs.filter)

This approach had a critical issue: test function filters were broken. When passing pytest test filters like "test_completion" or "test_streaming", the jobs would be skipped entirely because these strings don't exist in the provider list. The jobs would skip before pytest ever had a chance to run.

The workflow conflated two different types of filtering:

• Job-level: "Should we set up infrastructure for these providers?"
• pytest-level: "Which specific test functions should run?"

Example Failure Scenario

# User wants to run all completion tests across all providers
gh workflow run tests-integration.yaml --field filter="test_completion"

Expected: Both test jobs run, pytest filters to only completion tests
Actual: Both jobs skip because contains("anthropic,bedrock,...,minimax", "test_completion") returns false

Solution

Created a reusable composite action (.github/actions/determine-jobs) that intelligently analyzes the filter to determine which test jobs should run:

1. Provider matching: Checks if the filter contains any provider name from either the integration or local provider lists

2. Smart fallback: If the filter doesn't match any provider name, assumes it's a test function filter and runs all relevant jobs, letting pytest handle the filtering

   • "test_completion" → runs both jobs, pytest filters the tests ✅
   • "test_streaming" → runs both jobs, pytest filters the tests ✅

3. Provider-specific optimization: When a provider is explicitly named, only the relevant job runs

   • "minimax" → only runs integration tests job ✅
   • "ollama" → only runs local tests job ✅

Testing Scenarios

Filter Input	Integration Job	Local Job	Behavior
(empty)	✅ Runs	✅ Runs	All tests
minimax	✅ Runs	❌ Skips	Only minimax tests
ollama	❌ Skips	✅ Runs	Only ollama tests
test_completion	✅ Runs	✅ Runs	All completion tests
test_streaming and minimax	✅ Runs	❌ Skips	Minimax streaming tests
llama	✅ Runs	❌ Skips	All llama provider tests

Changes

• Created .github/actions/determine-jobs/action.yml - Composite action with smart filter analysis logic
• Updated determine-jobs-to-run job to use the new composite action
• Updated run-integration-tests job condition (already correctly configured)
• Updated run-local-integration-tests job condition (already correctly configured)

Benefits

• ✅ pytest test function filters now work correctly
• ✅ Provider name filters work correctly
• ✅ Jobs only run when they need to, saving CI resources
• ✅ Logic is extracted into a reusable, testable composite action
• ✅ Workflow YAML is cleaner and easier to maintain
• ✅ The workflow is more intuitive and matches user expectations

PR Type

🐛 Bug Fix ## Relevant issues

Checklist

• I have added unit tests that prove my fix/feature works
• New and existing tests pass locally
• Documentation was updated where necessary
• I have read and followed the contribution guidelines```

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
see 39 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.