Problem
Some tests fail in CI because they require external API keys that aren't available in the CI environment. Instead of gracefully skipping, they fail with assertion errors or connection errors.
Known affected tests
libs/kt-providers/tests/integration/test_brave_provider.py::test_brave_provider_is_available — asserts is_available() is True without checking if BRAVE_KEY is set
- Potentially other tests using
OPENROUTER_API_KEY, SERPER_KEY, OPENAI_API_KEY
Current workaround
All tests/integration/ directories are excluded from CI.
Proposed solution
1. Consistent skip guards
Every test that requires an external API key should skip gracefully:
async def test_brave_provider_is_available() -> None:
settings = get_settings()
if not settings.brave_key:
pytest.skip("BRAVE_KEY not set")
# ... rest of test
2. Audit all tests for missing guards
Search for tests that reference API keys or external services without skip guards:
grep -rn "brave_key\|openrouter\|serper_key\|openai_api_key" libs/*/tests/ services/*/tests/
3. Optional: CI job with secrets for main branch
For tests that genuinely need API keys, add a separate CI job that:
- Only runs on push to
main (not PRs from forks)
- Has API keys configured as GitHub secrets
- Runs the full integration test suite including external API tests
Acceptance criteria
Problem
Some tests fail in CI because they require external API keys that aren't available in the CI environment. Instead of gracefully skipping, they fail with assertion errors or connection errors.
Known affected tests
libs/kt-providers/tests/integration/test_brave_provider.py::test_brave_provider_is_available— assertsis_available() is Truewithout checking ifBRAVE_KEYis setOPENROUTER_API_KEY,SERPER_KEY,OPENAI_API_KEYCurrent workaround
All
tests/integration/directories are excluded from CI.Proposed solution
1. Consistent skip guards
Every test that requires an external API key should skip gracefully:
2. Audit all tests for missing guards
Search for tests that reference API keys or external services without skip guards:
3. Optional: CI job with secrets for main branch
For tests that genuinely need API keys, add a separate CI job that:
main(not PRs from forks)Acceptance criteria
pytest.skip()guardspytest --co(collect-only) succeeds for all test directories without any keys set