-
Notifications
You must be signed in to change notification settings - Fork 6.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix running integration tests with spaces in it's names #51514
Fix running integration tests with spaces in it's names #51514
Conversation
This is an automated comment for commit baf6d18 with description of existing statuses. It's updated for the latest CI running
|
d4135cd
to
7835ed3
Compare
7835ed3
to
a1f5dbb
Compare
a1f5dbb
to
227eb94
Compare
One of jobs got killed:
|
What has happened with the test for |
All tests had been failed because of the runner got killed (see above):
|
This PR was expected to find a failed test for UDF, but it didn't: #51964 |
Interesting, why the runner was killed? |
What's wrong with
? |
@azat I cannot find |
@azat This group https://s3.amazonaws.com/clickhouse-test-reports/51514/7e8ccade3517f55e111c6dbfac653dcd7ed18019/integration_tests__release__[1_4].html |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.
0b26dbf
to
ce061ba
Compare
Nice catch, there was an error due to single quotes in quotes:
Same:
Both had be addressed |
Now it solves the problem! |
ce061ba
to
b03d813
Compare
|
b03d813
to
4e7a7b5
Compare
Still cannot parse tests with spaces,
Rebased (sigh). |
But why they had not marked as "not failed"? #50926 - The problem is new parsing that appends the
|
4e7a7b5
to
936d7c0
Compare
936d7c0
to
aa90e11
Compare
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Previously they were simply ignored, and not only them, but all the tests in that group, here is an example [1] [1]: https://s3.amazonaws.com/clickhouse-test-reports/51448/4ed462ac7834a8180f92ca7d7d3c076e687bfca9/integration_tests__asan__[4_6].html Note, that I've replaced PYTEST_OPTS with handled by pytest itself PYTEST_ADDOPTS, since "sh -c $PYTEST_OPTS" does not allows you to preserve the arguments correctly. v5: use double quotes for PYTEST_ADDOPTS value Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
It had been fixed and now it is allowed. Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
After proper parsing more failed tests pops up: - https://s3.amazonaws.com/clickhouse-test-reports/51514/aa90e11241a97e0fd3f36c5ea0b57db50e299d2d/integration_tests__asan__analyzer__[4_6].html - https://s3.amazonaws.com/clickhouse-test-reports/51514/aa90e11241a97e0fd3f36c5ea0b57db50e299d2d/integration_tests__asan__analyzer__[5_6].html Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
aa90e11
to
ee5e639
Compare
…ilures Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
b04e72c
to
baf6d18
Compare
pytest_opts = " ".join(args.pytest_args).replace("'", "\\'") | ||
tests_list = " ".join(args.tests_list) | ||
# NOTE: since pytest options is in the argument value already we need to additionally escape '"' | ||
pytest_opts = " ".join( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@azat this broke sending extra arguments to pytest like -k
because everything is put under the same quote and parsed as test name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should work, maybe you are using old env variable name? The new one is PYTEST_ADDOPTS
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
did you try running an integration test using runner and filtering, e.g. test_keeper_back_to_back -k test_concurrent_watches
?
AFAIS, the entire pytest_args
are quoted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
$ DOCKER_HOST=unix:///run/docker-privileged.sock tests/integration/runner --disable-net-host --binary $(which clickhouse) -- test_keeper_back_to_back -k test_concurrent_watches
2023-08-07 18:50:57,015 [ 25841 ] INFO : ClickHouse root is not set. Will use /src/ch/worktrees/clickhouse-upstream (runner:42, check_args_and_update_paths)
2023-08-07 18:50:57,015 [ 25841 ] INFO : Base configs dir is not set. Will use /src/ch/worktrees/clickhouse-upstream/programs/server (runner:77, check_args_and_update_paths)
2023-08-07 18:50:57,015 [ 25841 ] INFO : Cases dir is not set. Will use /src/ch/worktrees/clickhouse-upstream/tests/integration (runner:90, check_args_and_update_paths)
2023-08-07 18:50:57,015 [ 25841 ] INFO : src dir is not set. Will use /src/ch/worktrees/clickhouse-upstream/src (runner:97, check_args_and_update_paths)
2023-08-07 18:50:57,015 [ 25841 ] INFO : base_configs_dir: /src/ch/worktrees/clickhouse-upstream/programs/server, binary: /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse, cases_dir: /src/ch/worktrees/clickhouse-upstream/tests/integration (runner:99, check_args_and_update_paths)
clickhouse_integration_tests_volume
Running pytest container as: 'docker run -it --rm --name clickhouse_integration_tests_9xl5v8 --privileged --dns-search='.' --volume=/src/ch/clickhouse/.cmake-llvm16/programs/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/src/ch/clickhouse/.cmake-llvm16/programs/clickhouse:/clickhouse --volume=/src/ch/clickhouse/.cmake-llvm16/programs/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/src/ch/worktrees/clickhouse-upstream/programs/server:/clickhouse-config --volume=/src/ch/worktrees/clickhouse-upstream/tests/integration:/ClickHouse/tests/integration --volume=/src/ch/worktrees/clickhouse-upstream/src/Server/grpc_protos:/ClickHouse/src/Server/grpc_protos --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS=" test_keeper_back_to_back -k test_concurrent_watches -vvv" clickhouse/integration-tests-runner:latest '.
Start tests
========================================================================================================================================= test session starts =========================================================================================================================================
platform linux -- Python 3.10.6, pytest-7.4.0, pluggy-1.2.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /ClickHouse/tests/integration
configfile: pytest.ini
plugins: xdist-3.3.1, repeat-0.9.1, random-0.2, order-1.0.0, timeout-2.1.0
timeout: 900.0s
timeout method: signal
timeout func_only: False
collected 9 items / 8 deselected / 1 selected
test_keeper_back_to_back/test.py::test_concurrent_watches PASSED [100%]
============================================================================================================================= 1 passed, 8 deselected in 67.29s (0:01:07) ==============================================================================================================================
Previously they were simply ignored, and not only them, but all the
tests in that group, here is an example 1
Changelog category (leave one):