CAT: improve test timeouts #31810
CAT: improve test timeouts #31810
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
|
Current dependencies on/for this PR:
This comment was auto-generated by Graphite. |
alafanechere
force-pushed
the
augustin/10-25-CAT_improve_test_timeouts
branch
from
b7c0857
to
d9e52b6
Compare
| @@ -24,7 +24,7 @@ RUN poetry install --no-root --only main --no-interaction --no-ansi | |||
| COPY . /app | |||
| RUN poetry install --only main --no-cache --no-interaction --no-ansi | |||
|
|
|||
| LABEL io.airbyte.version=2.0.2 | |||
| LABEL io.airbyte.version=2.1.1 | |||
| # Running tests in parallel can sometime delay the execution of the tests if downstream services are not able to handle the load. | ||
| # This is why we set a timeout on tests that call command that should return quickly, like spec | ||
| @pytest.mark.default_timeout(ONE_MINUTE) |
There was a problem hiding this comment.
Is it possible for us to know if we've set an option to run concurrently? I.e. set timeouts based on if we're running concurrently or not?
No worries if not, just a suggestion :)
There was a problem hiding this comment.
I know the pytest.xdist exposes a worker_id fixture that can help us check within the test execution if we're running concurrently or not. But these timeout are not controlled by the test themselves but at the pytest orcherstration level. If we want to set the timeout more dynamically I think we'd have to change something here where we tweak tests configuration and have access to fixture value. I think it might be early optimization at the moment. Let's consider this change if we observe more timeout-related problems.
| @@ -517,7 +519,7 @@ def test_oauth_flow_parameters(self, actual_connector_spec: ConnectorSpecificati | |||
| diff = paths_to_validate - set(get_expected_schema_structure(spec_schema)) | |||
| assert diff == set(), f"Specified oauth fields are missed from spec schema: {diff}" | |||
|
|
|||
| @pytest.mark.default_timeout(60) | |||
| @pytest.mark.default_timeout(ONE_MINUTE) | |||
What
Closes #31809
Running test in parallel can sometime cause a increased load on the dagger-engine which can delay some test execution.
We've seen
TestDiscoverytimeout on source postgres (cf #31809)How
Testing
Running
airbyte-ci connecor --name=source-postgres test