Skip to content

chore: parallelize tests by module#1939

Merged
eakmanrq merged 3 commits intomainfrom
eakmanrq/improve_test_perf_parallelize
Jan 9, 2024
Merged

chore: parallelize tests by module#1939
eakmanrq merged 3 commits intomainfrom
eakmanrq/improve_test_perf_parallelize

Conversation

@eakmanrq
Copy link
Copy Markdown
Collaborator

@eakmanrq eakmanrq commented Jan 9, 2024

Low effort way to improve test performance. Cut times locally in half. Changed Circle from Small to Medium and nearly cut the test time (just test) in half.

Biggest issue now is that we load balance the tests across the workers by module. This is because some module assume we aren't running in parallel and therefore share resources across tests. This results in a skew across the workers. Fix shouldn't be too bad but taking the easy win for now.

@eakmanrq eakmanrq force-pushed the eakmanrq/improve_test_perf_parallelize branch from 0579eea to a1f33bb Compare January 9, 2024 02:22
Comment thread pytest.ini Outdated
@eakmanrq eakmanrq force-pushed the eakmanrq/improve_test_perf_parallelize branch 2 times, most recently from 3a0f043 to caff61b Compare January 9, 2024 02:45
@eakmanrq eakmanrq force-pushed the eakmanrq/improve_test_perf_parallelize branch from caff61b to c62ea6b Compare January 9, 2024 02:53
@eakmanrq eakmanrq requested a review from izeigerman January 9, 2024 16:51
Comment thread tests/conftest.py
Comment thread tests/conftest.py
@eakmanrq eakmanrq merged commit 57098a2 into main Jan 9, 2024
@eakmanrq eakmanrq deleted the eakmanrq/improve_test_perf_parallelize branch January 9, 2024 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants