Skip to content

fix: add pytest.skip guards for tests requiring external API keys #5

@charlie83Gs

Description

@charlie83Gs

Problem

Some tests fail in CI because they require external API keys that aren't available in the CI environment. Instead of gracefully skipping, they fail with assertion errors or connection errors.

Known affected tests

  • libs/kt-providers/tests/integration/test_brave_provider.py::test_brave_provider_is_available — asserts is_available() is True without checking if BRAVE_KEY is set
  • Potentially other tests using OPENROUTER_API_KEY, SERPER_KEY, OPENAI_API_KEY

Current workaround

All tests/integration/ directories are excluded from CI.

Proposed solution

1. Consistent skip guards

Every test that requires an external API key should skip gracefully:

async def test_brave_provider_is_available() -> None:
    settings = get_settings()
    if not settings.brave_key:
        pytest.skip("BRAVE_KEY not set")
    # ... rest of test

2. Audit all tests for missing guards

Search for tests that reference API keys or external services without skip guards:

grep -rn "brave_key\|openrouter\|serper_key\|openai_api_key" libs/*/tests/ services/*/tests/

3. Optional: CI job with secrets for main branch

For tests that genuinely need API keys, add a separate CI job that:

  • Only runs on push to main (not PRs from forks)
  • Has API keys configured as GitHub secrets
  • Runs the full integration test suite including external API tests

Acceptance criteria

  • All tests that require API keys have pytest.skip() guards
  • No test fails with a missing key error — it either skips or passes
  • pytest --co (collect-only) succeeds for all test directories without any keys set

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions