Try to stabilize test 02346_full_text_search.sql #46344
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
robot-clickhouse
added
the
pr-not-for-changelog
devcrafter
approved these changes
Feb 13, 2023
|
Should we disable this feature with Ordinary engine then? |
|
Hm, probably the same issue happens with Atomic databases as well, but we don't see it because the error is logged by a background thread, so |
tavplubix
requested changes
Feb 13, 2023
|
Probably some file related to indexes was not added to |
Failure, e.g. https://s3.amazonaws.com/clickhouse-test-reports/0/f3fa3d868cb9957eeca6b5d5fd653234f5db5b45/stateless_tests__release__databaseordinary_.html : "Cannot quickly remove directory /var/lib/clickhouse/data/test_ggwgonnk/tab/delete_tmp_all_1_1_0 by removing files; fallback to recursive removal. Reason: Code: 566. DB::ErrnoException: Cannot rmdir /var/lib/clickhouse/data/test_ggwgonnk/tab/delete_tmp_all_1_1_0, errno: 39, strerror: Directory not empty. (CANNOT_RMDIR) (version 23.2.1.1427 (official build))" The issue did not reproduce in a local debug or release builds. Then noticed that it happens only with "DatabaseOrdinary" test profiles: https://play.clickhouse.com/play?user=play#U0VMRUNUIGNoZWNrX3N0YXJ0X3RpbWUsIGNoZWNrX25hbWUsIHRlc3RfbmFtZSwgcmVwb3J0X3VybApGUk9NIGNoZWNrcwpXSEVSRSBjaGVja19zdGFydF90aW1lID49IG5vdygpIC0gSU5URVJWQUwgOTYgSE9VUgogICAgQU5EIHB1bGxfcmVxdWVzdF9udW1iZXIgPSAwCiAgICBBTkQgdGVzdF9zdGF0dXMgIT0gJ1NLSVBQRUQnCiAgICBBTkQgdGVzdF9zdGF0dXMgTElLRSAnRiUnCiAgICBBTkQgY2hlY2tfc3RhdHVzICE9ICdzdWNjZXNzJwogICAgQU5EIHBvc2l0aW9uKHRlc3RfbmFtZSwgJzAyMzQ2X2Z1bGxfdGV4dF9zZWFyY2gnKSA+IDAKT1JERVIgQlkgY2hlY2tfc3RhcnRfdGltZQ== "Ordinary" databases were deprecated in ca. 2020 and nowadays can no longer be created unless a special setting is on. There is actually a fallback path (recursive removal) so it is arguable why this is logged as error. For now disabling 02346_full_text_search for ordinary databases.
|
Hmm, good points. Yes, the inverted index adds extra files to the persistence. Let me try a real fix... |
rschu1ze
force-pushed
the
rs/stabilize-full_text_seaerch_test
branch
from
e14db0a
to
95c50b8
Compare
|
The immediate problem is solved in a bit hacky way. I did not find a more natural place where the checksums are populated. |
See |
rschu1ze
added a commit
that referenced
this pull request
Feb 26, 2023
The (experimental) inverted index writes/reads files different from the
standard files written by the other skip indexes. The original problem
was that with database engine "ordinary", DROP TABLE of a table with
inverted index finds unknown files in persistence and complains. The
same will happen with engine "atomic" but deferred. As a hotfix, the
error was silenced by explicitly adding the four files created in a
specific test to the deletion code.
This PR tries a cleaner solution where all needed files are provided via
the normal checksum structure. One drawback remains which is that the
affected files were written earlier and we don't have their checksums
available. Therefore, the inverted index is currently excluded from
CHECK TABLE.
Minimal repro:
SET allow_experimental_inverted_index = 1;
DROP TABLE IF EXISTS tab;
CREATE TABLE tab(s String, INDEX af(s) TYPE inverted(2)) ENGINE = MergeTree() ORDER BY s;
INSERT INTO tab VALUES ('Alick a01');
CHECK TABLE tab;
DROP TABLE IF EXISTS tab;
run ./clickhouse-test with --db-engine Ordinary
rschu1ze
added a commit
that referenced
this pull request
Feb 27, 2023
A better alternative to #46344
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Failure, e.g. https://s3.amazonaws.com/clickhouse-test-reports/0/f3fa3d868cb9957eeca6b5d5fd653234f5db5b45/stateless_tests__release__databaseordinary_.html :
The issue did not reproduce in a local debug or release builds.
I noticed that it happens only with "DatabaseOrdinary" test profiles: https://play.clickhouse.com/play?user=play#U0VMRUNUIGNoZWNrX3N0YXJ0X3RpbWUsIGNoZWNrX25hbWUsIHRlc3RfbmFtZSwgcmVwb3J0X3VybApGUk9NIGNoZWNrcwpXSEVSRSBjaGVja19zdGFydF90aW1lID49IG5vdygpIC0gSU5URVJWQUwgOTYgSE9VUgogICAgQU5EIHB1bGxfcmVxdWVzdF9udW1iZXIgPSAwCiAgICBBTkQgdGVzdF9zdGF0dXMgIT0gJ1NLSVBQRUQnCiAgICBBTkQgdGVzdF9zdGF0dXMgTElLRSAnRiUnCiAgICBBTkQgY2hlY2tfc3RhdHVzICE9ICdzdWNjZXNzJwogICAgQU5EIHBvc2l0aW9uKHRlc3RfbmFtZSwgJzAyMzQ2X2Z1bGxfdGV4dF9zZWFyY2gnKSA+IDAKT1JERVIgQlkgY2hlY2tfc3RhcnRfdGltZQ==
"Ordinary" databases were deprecated in ca. 2020 and nowadays can no longer be created unless a special setting is on.
There is actually a fallback path (recursive removal) and I am not sure why this is logged as "error" instead of "warning".
For now disabling 02346_full_text_search for ordinary databases.
Changelog category (leave one):