Load testing #50
Load testing #50
Conversation
tests/test_load.py
Outdated
|
|
||
| factory, network = global_execution_env | ||
| session_1 = network.sessions[0].copy() | ||
| session_2 = network.sessions[0].copy() |
There was a problem hiding this comment.
| session_2 = network.sessions[0].copy() | |
| session_2 = network.sessions[1].copy() |
There was a problem hiding this comment.
Thank you! That was a big typo. I fixed it everywhere.
There was a problem hiding this comment.
Thanks @jmorel ! Really good idea to implement these 3 use cases.
The 3 tests have exactly the same pattern.
Would it be possible to create a single test depending on a parametrized fixture called compute_plans (without sacrificing readability)?
https://docs.pytest.org/en/latest/fixture.html#parametrizing-fixtures
Not sure it will be better, but it looks fun to try ;)
tests/test_load.py
Outdated
| algo=algo, | ||
| dataset=dataset, | ||
| data_samples=[dataset.train_data_sample_keys[0]], | ||
| in_models=[previous_tuple] if previous_tuple else None, |
There was a problem hiding this comment.
Really minor and won't change anything:
| in_models=[previous_tuple] if previous_tuple else None, | |
| in_models=[previous_tuple] if previous_tuple else [], |
tests/test_load.py
Outdated
|
|
||
|
|
||
| @pytest.mark.parametrize('compute_plan_size', COMPUTE_PLAN_SIZES) | ||
| def test_load_single_node_1_branch(compute_plan_size, global_execution_env): |
There was a problem hiding this comment.
This is not really clear what's the goal of this test.
The goal of this test is to check that we can create a compute plan with a "lot of tuples".
That's the reason why it is cancelled shortly after creation.
Strictly speaking, I'm not sure it's a load test. This test should not take long to complete.
tests/test_load.py
Outdated
|
|
||
| cp = session.cancel_compute_plan(cp.compute_plan_id) | ||
| assert cp.status == assets.Status.canceled | ||
|
|
||
| cp = cp.future().wait() | ||
| assert cp.status == assets.Status.canceled | ||
|
|
||
| first_tuple = cp.list_traintuple()[0] | ||
| assert first_tuple.rank == 0 | ||
| assert first_tuple.status == assets.Status.done |
There was a problem hiding this comment.
I think the goal of this test is only to check that compute plan can be added and that the first tuple has been executed properly.
What about replacing the selected lines with:
cp = session.cancel_compute_plan(cp.compute_plan_id).future().wait()
assert cp.status == assets.Status.canceledIt will make the test simpler to read and it will be easier to understand what is tested.
Same for the 3 other tests.
tests/test_load.py
Outdated
|
|
||
| factory, network = global_execution_env | ||
| session_1 = network.sessions[0].copy() | ||
| session_2 = network.sessions[0].copy() |
There was a problem hiding this comment.
| session_2 = network.sessions[0].copy() | |
| session_2 = network.sessions[1].copy() |
jmorel
force-pushed
the
load_testing
branch
2 times, most recently
from
e498ccf
to
1558845
Compare
|
Since this is not necessary anymore and since we don't plan on running it regularly there's no point in keeping the PR open. I'm closing it. |
A few tests to try injecting larger and larger compute plans into the platform and making sure the platform can handle them.
Use
pytest tests/test_load.py -rs -v --durations=0to run these tests only.