Summary
The test suite has 3 failures, 97 skipped tests, and 289 deprecation warnings. While the 99.9% pass rate indicates platform health, we need to improve test reliability and coverage by fixing configuration issues.
Test Failures (3)
1. Async Task Execution Flakiness
Files: tests/test_parallel_task.py:595, tests/test_parallel_task.py:731
Tests:
TestAsyncModeExecution::test_async_mode_execution_completes
TestAsyncModeActivities::test_async_mode_creates_task_activity
Problem: These tests depend on a live Claude Code agent successfully completing a task. When the agent fails (missing API key, out of context, transient Docker issue), both tests fail.
Fix:
2. Subscription State Leak
File: tests/test_subscriptions.py:528
Test: TestSubscriptionAutoAssign::test_create_agent_no_subscriptions
Problem: Test expects agent to have no subscription, but subscription from prior test is still assigned.
Fix:
Skipped Tests to Enable (97 total)
Priority 1: Missing Test Environment Config
| Category |
Count |
Fix Required |
| Agent Server Direct |
14 |
Set TEST_AGENT_NAME env var in test config |
| Internal API |
7 |
Set INTERNAL_API_SECRET in test environment |
| Git tests |
10 |
Configure a test agent with git template |
| SSH access |
7 |
Configure SSH service for test env |
Priority 2: Missing Python Packages
| Package |
Tests Enabled |
Command |
docker |
6 (test_log_archive.py) |
pip install docker |
markdown-it-py |
13 (unit/test_slack_rich_text.py) |
pip install markdown-it-py |
Priority 3: Test Design Improvements
| Category |
Count |
Action |
| Public links |
4 |
Require running agent with public URL - consider mocking |
| Public user memory |
9 |
Same as above |
| Setup tests |
3 |
Can only run on fresh install - document as expected |
| Slack integration |
3 |
Mock Slack or configure test workspace |
| Validation DB |
3 |
Verify VALIDATE-001 tables exist in test DB |
Deprecation Warnings (289)
datetime.utcnow() Deprecation (Python 3.12+)
Replace with datetime.now(datetime.UTC) in:
Pydantic V2 Deprecations
Acceptance Criteria
Summary
The test suite has 3 failures, 97 skipped tests, and 289 deprecation warnings. While the 99.9% pass rate indicates platform health, we need to improve test reliability and coverage by fixing configuration issues.
Test Failures (3)
1. Async Task Execution Flakiness
Files:
tests/test_parallel_task.py:595,tests/test_parallel_task.py:731Tests:
TestAsyncModeExecution::test_async_mode_execution_completesTestAsyncModeActivities::test_async_mode_creates_task_activityProblem: These tests depend on a live Claude Code agent successfully completing a task. When the agent fails (missing API key, out of context, transient Docker issue), both tests fail.
Fix:
pytest.mark.dependencyto cascade skip when execution fails2. Subscription State Leak
File:
tests/test_subscriptions.py:528Test:
TestSubscriptionAutoAssign::test_create_agent_no_subscriptionsProblem: Test expects agent to have no subscription, but subscription from prior test is still assigned.
Fix:
clear_agent_subscriptioncall in fixture or test setupSkipped Tests to Enable (97 total)
Priority 1: Missing Test Environment Config
TEST_AGENT_NAMEenv var in test configINTERNAL_API_SECRETin test environmentPriority 2: Missing Python Packages
dockerpip install dockermarkdown-it-pypip install markdown-it-pyPriority 3: Test Design Improvements
Deprecation Warnings (289)
datetime.utcnow()Deprecation (Python 3.12+)Replace with
datetime.now(datetime.UTC)in:src/scheduler/service.pysrc/scheduler/database.pysrc/backend/logging_config.pysrc/backend/services/backlog_service.pysrc/backend/services/ssh_service.pytests/scheduler_tests/*.pyPydantic V2 Deprecations
AgentStatus,Activity,Executionmodels insrc/backend/models.pyto use Pydantic V2 config syntaxAcceptance Criteria