Fix overflow in function "neighbor" #12019
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
akuzm
approved these changes
Jun 29, 2020
robot-clickhouse
added a commit
that referenced
this pull request
Jul 7, 2020
…785b05c8ce456e92c1f58ccb2b671 Cherry pick #12019 to 20.5: Fix overflow in function "neighbor"
alexey-milovidov
added a commit
that referenced
this pull request
Jul 7, 2020
* Add settings to control nesting level for shards skipping optimization - optimize_skip_unused_shards_nesting (allows control nesting level for shards skipping optimization) - force_skip_optimize_shards_nesting (allows control nesting level for checking was shards skipped or not) - deprecates force_skip_optimize_shards_no_nested * fix review * fix * logger * Update executeQuery.cpp * restore immediate stacktrace output * Support parse uuid without separator * try fix build failure * report * fix read extra bytes when with separator * add perf test for distinct combinator * more optimal aggregate functions with both 'if' and 'distinct' combinators * try fix test failure * Initialize GlobalThreadPool explicitly * apply review suggestions * fixup * memory usage settings * allow implicit initialization * Fix documentation * Fix anotehr one nullable prewhere column. * fix style check - removed unused LOGICAL_ERROR * restore old toStringImpl signature * send build_id to sentry as well + fix some minor issues * remove unused imports * clang format file * construct path using boost::filesystem::path * cleanup few unused headers The following files had some unused headers and caught my eyes, so cleaning them up: ```bash programs/obfuscator/Obfuscator.cpp src/Databases/DatabaseAtomic.cpp ``` * Change MySQL global variables query to globalVariable function * add column header to test * lost unused * [experiment] maybe fix warnings in integration tests * fix arcadia build * Fix JDBC @@session.variables * Remove already fixed bug * Slightly modernize code around ZooKeeper * Fix @@session.variables AS * try to rename py files to less common names * add __init__.py * update SELECT description (#11907) Signed-off-by: Slach <bloodjazman@gmail.com> * [docs] extra F.A.Q. content (#11898) * more content for F.A.Q. * more content * normalize * more content * maybe fix the docs check * absolute img urls * fix tests * Keep alias after substitution of query parameters #11914 * show build commands * Added a test * Add SELECT @@Version fake value '5.7.30' #11089 * fix + bump tests * Fix unitialized memory * Fix using current database while checking access rights. * fix style check - clang format for lambdas * Better github-hook * Do not update PVS Studio docker automatically. They keep removing the old image every couple of weeks, and the entire docker update fails. * bump CI * for the love of God just update this container already, i'm begging you * Update Dockerfile * Update pvs-studio version * Fix jemalloc under OSX (by registering it as default zone explicitly) In case of OSX jemalloc register itself as a default zone allocator. But when you link statically then zone_register() will not be called, and even will be optimized out: $ nm clickhouse.patched | grep -c zone_register 0 Fix this, by manually calling it. v2: extern C * Rewrite arithmetic in aggregate functions optimisation (#11899) * remove clang-format tag comments * Put clickhouse-local data to /tmp by default This fixes #9848 Also fixes #11926 * trigger ci * fix some logical errors * Update features.html * Consider allocatedBytes() instead of bytes() in Storage{Buffer,Memory}. * fix query parameter visitor * fixpu * Update 00945_bloom_filter_index.sql * Less layers in docker file * Less layers in docker file * Update docker/test/performance-comparison/perf.py * rename test back to test.py * client exit with unrecognized arguments format code fix * remove useless const * save * Update canonized values in 00753_system_columns_and_system_tables. * add ssd to integration test * one more test * Fix one reason of test flakiness * Fix race condition in extractAllGroups * extract JOIN in own plan step * fix mistake reported by @Slach * remove useless logic * fix crash * Revive mmap IO * More verbose CMake in build docker * Avro UUID support * remove a trick with expression continuation * Respect direct_io/mmap settings while reading secondary indices * Minor modification * Added a comment #11949 * Added a test * Add CANNOT_PARSE_UUID extern const * Fix estimation of the number of marks for various thresholds * better ExpressionAction::execute() * performance comparison * add page fault perf events * better error messages * Add CPU frequencies to system.asynchronous_metrics * Update AsynchronousMetrics.cpp * keep ArrayJoin optimisation * Update path to performance tests build * tests with distributed * place left join keys in before_join actions * Added tests for #8692 * review fixes * fix totals * Update path to perf test package * use std::filesystem::path * Minor cleanup in Client.cpp before fuzzing * Extend word break config to all non-alphanumeric chars * Update AsynchronousMetrics.h * more cleanup * add test update test fix * remove unused imports + bump tests * Update 01345_array_join_LittleMaverick.sql * Update ExpressionActions.h * Fixes * Add comment * Remove empty lines * Update system.md (#11945) * Update system.md Translate the doc to Chinese * Update system.md * fix the title's translattion error (#11939) @zhang2014 I use MergeTree replaced Chinese version "合并“， and fix an error. * Add __init__.py * Remove strange header * Trying to fix ya.make * Update replxx submodule * Add a test for #10102 * disable tests in arcadia * Added a test from #5131 * performance comparison * Support Enums type for MySQL engine #3985 * performance comparison * Add integration test for MySQL enums type * Fix bad log message at server startup * Insert enum column values for test * Better diagnostics of "Replica {} appears to be already active" message * Fix typos * Added a test for #8550 * Update extended-roadmap.md * Update test * Fix FPE, step 1 * Make it more correct * Add a test * Avoid floating point error in geohashesInBox #11369 * Avro: allow missing fields * update * Fix build * Revert "[experiment] maybe fix warnings in integration tests" * Add xeus-clickhouse (#12010) xeus-clickhouse is a Jupyter kernal for ClickHouse * Fix test * Fix array size overflow in generateRandom * Added a test * Update ErrorCodes.cpp * More hardening * More hardening * Fix access rights: cannot grant DDL when allow_ddl=0 * Add tests * Add test test_allow_ddl. * Added failing test * Remove the usage of obsolete setting * Whitespace * Added a test * Allow to ALTER partition key in some cases * Slightly more safe * Only allow Enum extension within the same width * Update test * Added failing test for Replicated table * Fix clang-tidy * Whitespace * Write partition key changes to ZooKeeper * Typos * Implement for ReplicatedMergeTree * Added test reference * Added missing ORDER BY in test * Fix integration test * Fix error in addMonths with floating point argument * Fix error in addMonths with floating point argument * Fix overflow in function "neighbor" * Whitespace * Whitespace * Feature 8074 (#12014) * refs #8074 1: start to stream * refs final check format and spell * refs final check format and spell * refs #8074 fix up link to sql-reference/functions/other-functions * Bump idna from 2.9 to 2.10 in /docs/tools Bumps [idna](https://github.com/kjd/idna) from 2.9 to 2.10. - [Release notes](https://github.com/kjd/idna/releases) - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst) - [Commits](kjd/idna@v2.9...v2.10) Signed-off-by: dependabot-preview[bot] <support@dependabot.com> * DOCS-636 Synchronized some edits from EN to RU (#11941) * [review][clickhousedocs][translate] DOCSUP-1351: Edit and translate to Russian (#129) * DOCSUP-1351: Add blockSerializedSize (RU) * DOCSUP-1351: Fix blockSerializedSize description by function description template. * DOCSUP-1351: PR8756- Update Distributed engine section. * DOCSUP-1351: PR9076-Show Queries more like MySQL did. * DOCSUP-1351: PR#9919-system.tables improvements (total_rows/total_bytes/storage_policy) * DOCSUP-1351: (ru) blockSerializedSize remake by template. * DOCSUP-1351: PR#9598-Add function arrayReduceInRange and update arrayReduce function. * DOCSUP-1351: PR#9598 Fix arrayReduce and arrayReduceInRanges sections by template. * DOCSUP-1351: Some fixes. * DOCSUP-1351: Some fixes. Co-authored-by: romanzhukov <romanzhukov@yandex-team.ru> * CLICKHOUSEDOCS-636: Minor changes. Co-authored-by: Sergei Shtykov <bayonet@yandex-team.ru> Co-authored-by: Roman Bug <rrrrrroman@gmail.com> Co-authored-by: romanzhukov <romanzhukov@yandex-team.ru> * Temporary enable debug logs when releasing docs * [docs] improve stable releases discovery * Automated backporting machinery (#12029) * Add automated cherry-pick machinery * Added script for backporting machinery * Implement dry-run mode * Comment old methods * Fix python2 * Add ya.make for sandbox tasks * Remove old stuff Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com> Co-authored-by: Nikita Vasilev <vasnikserg@yandex.ru> Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru> Co-authored-by: Ivan Blinkov <github@blinkov.ru> Co-authored-by: zhang2014 <coswde@gmail.com> Co-authored-by: Alexander Kuzmenkov <akuzm@yandex-team.ru> Co-authored-by: Anton Popov <pad11rus@gmail.com> Co-authored-by: Nikolai Kochetov <nik-kochetov@yandex-team.ru> Co-authored-by: bharatnc <bharatnc@gmail.com> Co-authored-by: BohuTANG <overred.shuttler@gmail.com> Co-authored-by: Nikolai Kochetov <KochetovNicolai@users.noreply.github.com> Co-authored-by: Eugene Klimov <bloodjazman@gmail.com> Co-authored-by: Alexander Kuzmenkov <36882414+akuzm@users.noreply.github.com> Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com> Co-authored-by: alesapin <alesapin@gmail.com> Co-authored-by: Vitaly Baranov <vitbar@yandex-team.ru> Co-authored-by: Artem Zuikov <chertus@gmail.com> Co-authored-by: Alexander Tokmakov <avtokmakov@yandex-team.ru> Co-authored-by: Maxim Akhmedov <max42@yandex-team.ru> Co-authored-by: feng lv <fenglv15@mails.ucas.ac.cn> Co-authored-by: Nikita Mikhailov <jakalletti@jakalletti-build.sas.yp-c.yandex.net> Co-authored-by: Andrew Onyshchuk <andryk.rv@gmail.com> Co-authored-by: Tom Bombadil <565258751@qq.com> Co-authored-by: Yuntao Wu <wuyuntao2015@163.com> Co-authored-by: Wang Fenjin <wangfenj@gmail.com> Co-authored-by: Mikhail Gaidamaka <magnoly.nsk@gmail.com> Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Co-authored-by: BayoNet <da-daos@yandex.ru> Co-authored-by: Sergei Shtykov <bayonet@yandex-team.ru> Co-authored-by: Roman Bug <rrrrrroman@gmail.com> Co-authored-by: romanzhukov <romanzhukov@yandex-team.ru> Co-authored-by: Ivan <5627721+abyss7@users.noreply.github.com>
alesapin
pushed a commit
that referenced
this pull request
Jul 10, 2020
Fix overflow in function "neighbor" (cherry picked from commit 3223757)
alesapin
pushed a commit
that referenced
this pull request
Jul 10, 2020
Fix overflow in function "neighbor" (cherry picked from commit 3223757)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
A query with function
neighboras the only returned expression may return empty result if the function is called with offset-9223372036854775808. This fixes #11367.