Skip to content

Conversation

@alexander-alderman-webb
Copy link
Contributor

Description

The AI test suite takes the longest, so splitting the workflow up reduces the time for CI to complete.

Issues

Reminders

strategy:
fail-fast: false
matrix:
python-version: ["3.10","3.11","3.12","3.13","3.14","3.14t"]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Python version matrix includes unsupported combinations

The workflow Python version matrices include versions unsupported by some frameworks in each group. The generator script unions all Python versions across grouped frameworks, causing tests to fail for nonexistent tox environments. For example, pydantic_ai lacks Python 3.11, 3.14, and 3.14t in tox.ini, yet the Agents workflow will attempt py3.11-pydantic_ai. Similarly, langgraph lacks 3.11, and mcp lacks 3.14 and 3.14t in their respective workflows.

Additional Locations (2)

Fix in Cursor Fix in Web

@codecov
Copy link

codecov bot commented Nov 25, 2025

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
28630 1 28629 2319
View the top 1 failed test(s) by shortest run time
tests.integrations.ray.test_ray::test_errors_in_ray_actors
Stack Traces | 0.106s run time
.../integrations/ray/test_ray.py:229: in test_errors_in_ray_actors
    ray.init(
.tox/py3.9-ray-v2.7.2/lib/python3.9.../ray/_private/client_mode_hook.py:103: in wrapper
    return func(*args, **kwargs)
.tox/py3.9-ray-v2.7.2/lib/python3.9.../ray/_private/worker.py:1536: in init
    _global_node = ray._private.node.Node(
.tox/py3.9-ray-v2.7.2/lib/python3.9.../ray/_private/node.py:303: in __init__
    self._ray_params.update_pre_selected_port()
.tox/py3.9-ray-v2.7.2/lib/python3.9.../ray/_private/parameter.py:348: in update_pre_selected_port
    raise ValueError(
E   ValueError: Ray component dashboard_agent_http is trying to use a port number 52365 that is used by other components.
E   Port information: {'gcs': 'random', 'object_manager': 'random', 'node_manager': 'random', 'gcs_server': 60768, 'client_server': 'random', 'dashboard': 'random', 'dashboard_agent_grpc': 52365, 'dashboard_agent_http': 52365, 'dashboard_grpc': 'random', 'runtime_env_agent': 64941, 'metrics_export': 57935, 'redis_shards': 'random', 'worker_ports': 'random'}
E   If you allocate ports, please make sure the same port is not used by multiple components.

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Copy link
Contributor

@sentrivana sentrivana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏🏻

@alexander-alderman-webb alexander-alderman-webb merged commit 7b7ea33 into master Nov 26, 2025
155 checks passed
@alexander-alderman-webb alexander-alderman-webb deleted the webb/split-ai-test-suite branch November 26, 2025 08:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants