[typo](docs)Fix Docs Error Urls#13153
Closed
FreeOnePlus wants to merge 252 commits intoapache:masterfrom
Closed
Conversation
github-actions
bot
added
the
kind/docs
…veLimits rule (apache#12624) This PR added a condition check for MergeConsecutiveLimits rule: the input upper limit should not have valid offset info.
…ter on hash join (apache#12475) test sql: TPC-H q21 ``` select count(*) from lineitem l3 right anti join lineitem l1 on l3.l_orderkey = l1.l_orderkey and l3.l_suppkey <> l1.l_suppkey; ``` if we have other join conjuncts, we have to put all slots from left and right into `slotReferenceMap` instead of `hashjoin.getOutput()` After splitting intermediate tuple and output tuple, we meet several issues in regression test. And hence, we make following changes: 1. since translating project will replace underlying hash-join node's output tuple, we add PhysicalHashJoin.shouldTranslateOutput 2. because PhysicalPlanTranslator will merge filter and hashJoin, we add PhysicalHashJoin.filterConjuncts and translate filter conjuncts in physicalHashJoin 3. In this pr, we set HashJoinNode.hashOutputSlotIds properly when using nereids planner. 4. in order to be compatible with BE, in substring function, nullable() returns true
…benchmark (apache#12651) For eliminate all unessential cross join in TPC-H benchmark, this PR: 1. push all predicates that can be push down through join before do ReorderJoin rule. Then we could eliminate all cross join that can be eliminated in ReorderJoin rule since this rule need matching a LogicalFilter as a root pattern. (Q2, Q15, Q16, Q17, Q18) 2. enable expression optimization rule - extract common expression. (Q19) 3. fix cast translate failed. (Q19)
Co-authored-by: wudi <>
…r-overflow when TBrokerOpenReaderResponse too large (apache#12658)
…on trait (apache#12671) This pr did these things: 1. Change the nullable mode of 'from_unixtime' and 'parse_url' from DEPEND_ON_ARGUMENT to ALWAYS_NULLABLE, which nullable configuration was missing previously. 2. Add some new interfaces for origin NullableMode. This change inspired by the grammar of scala's mix-in trait, It help us to quickly understand the traits of function without read the lengthy procedural code and save the work to write some template code, like `class Substring extends ScalarFunction implements ImplicitCastInputTypes, PropagateNullable`. These are the interfaces: - PropagateNullable: equals to NullableMode.DEPEND_ON_ARGUMENT - AlwaysNullable: equals to NullableMode.ALWAYS_NULLABLE - AlwaysNotNullable: equals to NullableMode.ALWAYS_NOT_NULLABLE - others ComputeNullable: equals to NullableMode.CUSTOM 3. Add `GenerateScalarFunction` to generate nereids-style function code from legacy functions, but not actual generate any new function class yet, because the function's trait is not ready for use. I need add some traits for the legacy function's CompareMode and NonDeterministic, this thought is the same as ComputeNullable.
When the flush is triggered when the load channel exceeds the mem limit, if the flush fails, an error message is returned and the load is terminated. Usually flush failure is -238 error code. Because the memtable is frequently flushed after the load channel exceeds the mem limit, the number of segments exceeds the max value.
…ases (apache#12160) Fix _delete_sign_idx and _seq_col_idx when append_column or build_schema when load. Tablet schema cache support recycle when schema sptr use count equals 1. Add a http interface for flink-connector to sync ddl. Improve tablet->tablet_schema() by max_version_schema.
…lloc allocator cache (apache#12688) tcmalloc/jemalloc allocator cache does not participate in the mem check as part of the process physical memory. because new/malloc will trigger mem hook when using tcmalloc/jemalloc allocator cache, but it may not actually alloc physical memory, which is not expected in mem hook fail. in addition: The value of tcmalloc/jemalloc allocator cache is used as a mem tracker, the parent is the process mem tracker, which is updated every 1s. Modify the process default mem_limit to 90%. expect mem tracker to effectively limit the memory usage of the process.
…he#12670) Speed up dict data read and not datetime. same target apache#12636
…when flush memtable (apache#12668)
turn on all TPC-H sf1 test cases except Q2. Q2 caused dead loop in Join reorder. Will turn on Q2 after fix it.
After the consume mem tracker exceeds the mem limit in the mem hook, the boost stacktrace will be printed. A query/load will only be printed once, and the process tracker will only be printed once per second. After the process memory reaches the upper limit, the boost stacktrace will be printed every second. The observed phenomena are as follows: After query/load is canceled, the memory increases instantly; tcmalloc profile total physical memory is less than perf process memory; The process mem tracker is smaller than the perf process memory;
…nation rule (apache#11793) Every time a new broker load comes in, Doris will update the start time of Kerberos authentication, but this logic is wrong. Because the authentication duration of Kerberos is calculated from the moment when the ticket is obtained. This PR change the logic: 1. If it is kerberos, check fs expiration by create time. 2.Otherwise, check fs expiration by access time
…table (apache#12644) This PR fix: 2 Backends. Create tables with colocation group, 1 replica. Decommission one of Backends. The tablet on decommissioned Backend is not reduced. This is a bug of ColocateTableCheckerAndBalancer.
…ARCHAR and STRING type (apache#12637) The predicate column type for char, varchar and string is PredicateColumnType<TYPE_STRING>, so _base_evaluate method should convert the input column to PredicateColumnType<TYPE_STRING> always.
…pache#13116) 1. Fix issue apache#13115 2. Modify the method of `get_next_block` or `GenericReader`, to return "read_rows" explicitly. Some columns in block may not be filled in reader, if the first column is not filled, use `block->rows()` can not return real row numbers. 3. Add more checks for broker load test cases.
… size (apache#13112) String column lengh is 2GB, if we allocate memory according to column length, string would consume a lot of memory. It also misleads memory tracker.
Now, every preare put a runtime filter controller, so it takes the mutex lock on the controller map. Init of bloom filter takes some time in allocate and memset. If we run p1 tests with -parallel=20 -suiteParallel=20 -actionParallel=20, then we get error message like 'send fragment timeout 5s'. The patch fixes the problem in the following 2 ways: 1. Replace one mutex block with 128s. 2. If a plan fragment does not have a runtime filter, it does not need to take the locks.
…pache#12848) Related pr: apache#11582 This pr is the new jdbc scan node and scanner.
…er is more than one character (apache#13066)
fix data cache sidebar error
…ult (apache#13076) Flushing memtable is cpu bound, so 2 thread for a disk is tool small.
…2890) * [improve](Nereids): simplify onCondition check. * feature: support project Alias for join reorder.
…h new file scanner (apache#13135) Fix some logic about broker load using new file scanner, with parquet format: 1. If columns are specified in load stmt, but none of them are in parquet file, error will be thrown like `err: No columns found in file`. See `parquet_s3_case4` 2. If the first column of table are not in table, the result number of rows is wrong. See `parquet_s3_case8` 3. If column specified in `columns` in load stmt does not exist in file and table, error will be thrown like: `failed to find default value expr for slot: x1`. See `parquet_s3_case2`
github-actions
bot
added
area/load
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
baseUrl: https://doris.apache.org/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHOW-ALTER-TABLE-MATERIALIZED-VIEW
errorUrl:https://doris.apache.org/zh-CN/docs/sql-manual/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-MATERIALIZED-VIEW.md
fixUrl:https://doris.apache.org/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-MATERIALIZED-VIEW.md
Proposed changes
Issue Number: close #xxx
Problem summary
Describe your changes.
Checklist(Required)
Further comments
If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...