Skip to content

Conversation

github-actions[bot]
Copy link
Contributor

@github-actions github-actions bot commented Oct 2, 2025

Update our test matrix with new releases of integrated frameworks and libraries.

How it works

  • Scan PyPI for all supported releases of all frameworks we have a dedicated test suite for.
  • Pick a representative sample of releases to run our test suite against. We always test the latest and oldest supported version.
  • Update tox.ini with the new releases.

Action required

  • If CI passes on this PR, it's safe to approve and merge. It means our integrations can handle new versions of frameworks that got pulled in.
  • If CI doesn't pass on this PR, this points to an incompatibility of either our integration or our test setup with a new version of a framework.

🤖 This PR was automatically created using a GitHub action.

@github-actions github-actions bot requested a review from a team as a code owner October 2, 2025 15:23
@github-actions github-actions bot added Component: CI Dealing with Github Actions Component: Tests Dealing with tests labels Oct 2, 2025
@sentrivana sentrivana enabled auto-merge (squash) October 2, 2025 15:25
Copy link

codecov bot commented Oct 2, 2025

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
23711 2 23709 2304
View the top 2 failed test(s) by shortest run time
tests.integrations.openai_agents.test_openai_agents::test_tool_execution_span
Stack Traces | 0.132s run time
.../integrations/openai_agents/test_openai_agents.py:467: in test_tool_execution_span
    assert agent_span["data"]["gen_ai.request.available_tools"] == available_tools
E   assert '[{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true, "tool_input_guardrails": null, "tool_output_guardrails": null}]' == '[{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true}]'
E     
E     - [{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true}]
E     + [{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true, "tool_input_guardrails": null, "tool_output_guardrails": null}]
E     ?                                                                                                                                                                                                                                                                                                                                                                                                                                      +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
tests.integrations.openai_agents.test_openai_agents::test_tool_execution_span
Stack Traces | 0.157s run time
.../integrations/openai_agents/test_openai_agents.py:467: in test_tool_execution_span
    assert agent_span["data"]["gen_ai.request.available_tools"] == available_tools
E   assert '[{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true, "tool_input_guardrails": null, "tool_output_guardrails": null}]' == '[{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true}]'
E     
E     - [{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true}]
E     + [{"name": "simple_test_tool", "description": "A simple tool", "params_json_schema": {"properties": {"message": {"title": "Message", "type": "string"}}, "required": ["message"], "title": "simple_test_tool_args", "type": "object", "additionalProperties": false}, "on_invoke_tool": "<function agents.tool.function_tool.<locals>._create_function_tool.<locals>._on_invoke_tool>", "strict_json_schema": true, "is_enabled": true, "tool_input_guardrails": null, "tool_output_guardrails": null}]
E     ?                                                                                                                                                                                                                                                                                                                                                                                                                                      +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@sentrivana sentrivana merged commit ffc88f5 into master Oct 2, 2025
113 checks passed
@sentrivana sentrivana deleted the toxgen/update branch October 2, 2025 16:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component: CI Dealing with Github Actions Component: Tests Dealing with tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant