Fix using current database while checking access rights. #11920
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alexey-milovidov
approved these changes
Jun 24, 2020
vitlibar
added
pr-bugfix
|
The bug was introduced in #9065, it didn't exist until 20.4. So there is no need to backport it to 20.3 or 20.1 |
alesapin
approved these changes
Jun 24, 2020
|
https://clickhouse-test-reports.s3.yandex.net/11920/29a9d4187f569cc781cffbc712c1a3ddb3fcfb82/stress_test_(thread).html#fail1 in CI fest. Integration tests failure is OK (bad experiment with docker image). |
abyss7
added a commit
that referenced
this pull request
Jun 25, 2020
…f7a4489166d9ef4db68ac7d528287 Cherry pick #11920 to 20.5: Fix using current database while checking access rights.
alesapin
added a commit
that referenced
this pull request
Jul 2, 2020
…cess rights. (#11963) * Simple github hook * Add concurrent benchmark to performance test After the main test, run queries from `website.xml` in parallel using `clickhouse-benchmark`. This can be useful to test the effects of concurrency on performance. Comparison test can miss some effects because it always runs queries sequentially, and many of them are even single-threaded. * typo * fixup * add flag to continue on errors * fixup * fixup * Compilable setColumns, setConstraints, setIndices * Working setColumns, setConstraints, setIndices * Move set*Key methods to StorageInMemoryMetadata * Working set*Keys methods * TTL methods in StorageInMemoryMetadata * All set methods in metadata * Metadata in read and write methods of IStorage * make max global thread pool setting configurable This PR adds a server level config for overriding the default max number of threads in global thread pool that is currently allowed (10,000). This might be useful in scenarios where there are a large number of distributed queries that are executing concurrently and where the default number of max threads might not be necessarily be sufficient. * add max_thread_pool_size setting to tests This adds the `max_thread_pool_size` config to tests/server-test.xml file. * docs for max_thread_pool_size This adds the docs for the new server level setting `max_thread_pool_size`. * remove extra vertical space * fixup * Copy some methods to metdata * Get column dependencies in StorageInMemoryMetadata * Move getSampleBlockNonMaterialized to StorageInMemoryMetadata * getSampleBlockWithVirtuals in StorageInMemoryMetadata * Compileable getSampleBlockWithColumns in StorageInMemoryMetadata * Seems to be working getSampleBlockWithColumns in StorageInMemoryMetadata * Buildable getSampleBlock in StorageInMemoryMetadata * getConstraints() in StorageInMemoryMetadata (suspicious commit, but pretend to work) * Fix race condition * Secondary indices in StorageInMemoryMetadata * Partition key in StorageInMemoryMetadata * Compilable sorting key in metadata * Fix race condition * Sampling key in StorageInMemoryMetadata * Primary key in storage metadata * Missed change for primary key * TTL in storage in memory metadata * Settings changes in StorageInMemoryMetadata * Select query in metadata * Check methods in metadata * Better getVirtuals method * getColumns in StorageInMemoryMetadta (only compilable) * Fix segmentation fault * Fix build * Fix crash * Fix storage merge * Truncate with metadata * Fix SystemTables * Really atomic metadata * Fix style * Fix storage buffer metadata * Better locks in Storages * Less locks * Fix style * Remove unused method * Remove unused method better * Remove int contention * Disable optimize_skip_unused_shards if sharding_key has non-deterministic func Example of such functions is rand() And this patch disables only optimize_skip_unused_shards, i.e. INSERT code path does not changed, so it will work as before. * optimize_skip_unused_shards=2 will disable it for nested distributed queries P.S. Looks like settings can be converted between SettingUInt64 and SettingBool without breaking binary protocol. FWIW maybe it is a good idea to change the semantics of the settings as follow (but I guess that changing semantic is not a good idea, better to add new settings and deprecate old ones): - optimize_skip_unused_shards -- accept nesting level on which the optimization will work - force_skip_optimize_shards_nesting -- accept nesting level on which the optimization will work * Add logging of adjusting conditional settings for distributed queries * Add missing DROP TABLE in 01319_mv_constants_bug * Improve 01319_optimize_skip_unused_shards_no_nested Before there is no check that optimize_skip_unused_shards was working for the first level, use cluster with unavalable shard to guarantee this. * Fix locks * Add missed check * Move partition key initialization into registerMergeTree * Remove unused constructor and rename method * Better naming * Better logging in storages * Remove unused variable * report the number of errors * fixup * fixup * Send logs on fatal errors (development) * Set send_logs_level to fatal by default * Update tests * Turn off traps * Add settings to control nesting level for shards skipping optimization - optimize_skip_unused_shards_nesting (allows control nesting level for shards skipping optimization) - force_skip_optimize_shards_nesting (allows control nesting level for checking was shards skipped or not) - deprecates force_skip_optimize_shards_no_nested * Fix memory accounting via HTTP interface function perf_test() { time yes '127.1:8123/?wait_end_of_query=1' | head -n10000 | xargs -P10000 curl -s -d 'select 1' | grep -x -c 1 } function server() { local limit=$1 && shift clickhouse-server "$@" -- --max_server_memory_usage=$limit } perf_test: - before this patch with 1G limit: succeed only ~800 queries - after this patch with 1G limit: succeed ~8000 queries - before this patch with 10G limit: succeed only ~3000 queries - after this patch with 10G limit: succeed ~10000 queries Fixes: #11153 * Add a test for memory tracking via HTTP interface * Update executeQuery.cpp * Make LIVE VIEW test less wrong (it was depending on the timing of progress messages) * Do not run 01238_http_memory_tracking under sanitizers * Tune 01238_http_memory_tracking to make it more optimal w/o the patch the test fails with, since it succeed only 512 queries. * Merge with master * Ad tests with long select and simulatenous alter * Better comments * Fix 01213_alter_rename_column_zookeeper * report * Fix bad test for LIVE VIEW * Disable MySQL tests under tsan * Better 00980_alter_settings test * Support sumWithOverflow as SimpleAggregateFunction * Clang-tidy fixes * Fix header for nullable prewhere column. * Update tests. * Fix 01091_num_threads flaps (by adding unique identifier to the queries) Actually there were two issues: - missing \n, so it picked the wrong query anway - no unique identifier, hence it may works incorrectly (and actually, event clickhouse-test script executes 'SELECT 1' query) * Initialize GlobalThreadPool explicitly * Do not render progress too much * Stop tests on fatal errors * jemalloc: fix HAVE_CPU_SPINWAIT for linux aarch64 Fixes: #11163 * Style * constexpr intExp10 * Fix bugs in DateTime64 parsing * Added a test * Fix wrong code * Fix strange code * Update tests * jemalloc: jemalloc_preamble.h is common * jemalloc: fix JEMALLOC_NOTHROW/JEMALLOC_CXX_THROW for freebsd/darwin * jemalloc: add freebsd pregenerated header * jemalloc: add darwin pregenerated header * jemalloc: enable by default for freebsd/darwin * jemalloc: drop superfluous include directory * jemalloc: explicitly disable pthread_setname_np on freebsd (requires 12.1) * Add drop replica alter support * fix merge state Fix style && build * Add tests and docs Fix integration-tests * only drop inactive replica update doc Increase timeout to release the zookeeper Ephemeral nodes Fix code comment use PartitionManager make integrations test passed * add SYSTEM DROP REPLICA * fix: can not drop local table and replace ALTER with SYSTEM fix test_drop_replica fix drop replica '/path/to/zk/' ending in '/' and update doc * add drop relica database and the whole replica fix removeReplicaByZKPath fix bug: add zkpath empty judge fix: rewrite code delete useless code. fix:ast fromat fix bug add test_drop_replica add drop_replica doc add drop databse checkAccess refactor dropReplica update tests add static method StorageReplicatedMergeTree::dropReplicaByZkPath update doc and delete useless code fix conflict fix doc fix doc fix StorageReplicatedMergeTree::dropReplica fix bug delete useless code * rewrite static StorageReplicatedMergeTree::dropReplica * format comment * SHA256 add FixedString support * fixup * Update index.md * fix of incorrect example was `curl -sS "<address>?param_id=2¶m_phrase=test" -d "SELECT * FROM table WHERE int_column = {id:UInt8} and string_column = {phrase:String}"` * Add missed lock * jemalloc: disable percpu_arena on non-linux (not supported anyway) And since it does not supported it produces the following warning: <jemalloc>: No getcpu support: percpu_arena:percpu <jemalloc>: perCPU arena getcpu() not available. Setting narenas to 8. However it is uses narenas=ncpu instead of default narenas=ncpu*4 * Optimize code * jemalloc: disable it for darwin Right now it aborts (Abort trap: 6) on osx, the issue does not looks like [1], but should be investigated, so disable for now. What is interesting that it works under debugger, so I guess it is indeed the order of symbol replacements... [1]: jemalloc/jemalloc#420 * fix bug for query "select *, abc.*" should failed. (#11753) * Remove redundat sync replica * test commit * Hook for push events * add test fix fix fix * memory usage settings * Update README.md * Update README.md * Merge pull request #11071 (#11874) * Add's hasSubstr array function * Add's example in the english doc of hasSubStr * Updates conditions to avoid prematurely calling isEqual * Remove feathericons Authored-by: Ryad ZENINE <ryad.zenine@contentsquare.com> * allow implicit initialization * Fix bad code * Allow ClickHouse to run on Android * Fix documentation * Add docs about non-linux binaries (#11889) * Add docs about non-linux binaries * Better * Update install.md * Update install.md * Update docs/en/getting-started/install.md Co-authored-by: Ivan Blinkov <github@blinkov.ru> * Update docs/en/getting-started/install.md Co-authored-by: Ivan Blinkov <github@blinkov.ru> * Update docs/en/getting-started/install.md Co-authored-by: Ivan Blinkov <github@blinkov.ru> * Review fixes * Better instruction * Fix reference * Better * Better links Co-authored-by: Ivan Blinkov <github@blinkov.ru> * Fix anotehr one nullable prewhere column. * minor improvements * Update StorageSystemTables.cpp * Added a test * fix style check - removed unused LOGICAL_ERROR * Fix if function with NULLs (#11807) * cleanup few unused headers The following files had some unused headers and caught my eyes, so cleaning them up: ```bash programs/obfuscator/Obfuscator.cpp src/Databases/DatabaseAtomic.cpp ``` * sync with goals * Remove unused code * Remove already fixed bug * update SELECT description (#11907) Signed-off-by: Slach <bloodjazman@gmail.com> * [docs] extra F.A.Q. content (#11898) * more content for F.A.Q. * more content * normalize * more content * maybe fix the docs check * absolute img urls * fix tests * Keep alias after substitution of query parameters #11914 * show build commands * Added a test * Fix using current database while checking access rights. * Better github-hook * Do not update PVS Studio docker automatically. They keep removing the old image every couple of weeks, and the entire docker update fails. * for the love of God just update this container already, i'm begging you * Update pvs-studio version * Fix jemalloc under OSX (by registering it as default zone explicitly) In case of OSX jemalloc register itself as a default zone allocator. But when you link statically then zone_register() will not be called, and even will be optimized out: $ nm clickhouse.patched | grep -c zone_register 0 Fix this, by manually calling it. v2: extern C * Rewrite arithmetic in aggregate functions optimisation (#11899) * Put clickhouse-local data to /tmp by default This fixes #9848 Also fixes #11926 * Update features.html * fixpu * Less layers in docker file * Less layers in docker file * Update docker/test/performance-comparison/perf.py Co-authored-by: alesapin <alesapin@gmail.com> Co-authored-by: Alexander Kuzmenkov <akuzm@yandex-team.ru> Co-authored-by: Bharat Nallan <bharatnc@gmail.com> Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com> Co-authored-by: Alexey Milovidov <milovidov@yandex-team.ru> Co-authored-by: Nikolai Kochetov <nik-kochetov@yandex-team.ru> Co-authored-by: sundy-li <543950155@qq.com> Co-authored-by: amudong <xiaoxiong5288@gmail.com> Co-authored-by: feng lv <fenglv15@mails.ucas.ac.cn> Co-authored-by: Alexander Kuzmenkov <36882414+akuzm@users.noreply.github.com> Co-authored-by: Ivan Blinkov <github@blinkov.ru> Co-authored-by: Alexandr Kondratev <theg4sh@gmail.com> Co-authored-by: hexiaoting <hewenting_ict@163.com> Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com> Co-authored-by: Alexander Tokmakov <avtokmakov@yandex-team.ru> Co-authored-by: Artem Zuikov <chertus@gmail.com> Co-authored-by: Vxider <lb@vxider.com> Co-authored-by: Nikolai Kochetov <KochetovNicolai@users.noreply.github.com> Co-authored-by: Eugene Klimov <bloodjazman@gmail.com> Co-authored-by: Vitaly Baranov <vitbar@yandex-team.ru> Co-authored-by: robot-clickhouse <robot-clickhouse@yandex-team.ru>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en
Changelog category:
Changelog entry:
Use the correct current database for checking access rights after statement
USE database.