merge main #135
Merged
merge main #135
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fix bvt test Approved by: @heni02
- reduce the size of tombstone files Approved by: @XuPeng-SH
由于索引表总是将主键序列化进去，导致ndv很高，索引表的过滤度估计严重错误，会导致优化器错判tp/ap语句 现在改成利用原始过滤条件的过滤度去计算prefix_eq函数的过滤度 Approved by: @aunjgr
…rigin#16633) Add a log statement to improve traceability when the transfer is not needed Approved by: @XuPeng-SH
…gin#16685) make sure all pipeline run in single parallel for tp query Approved by: @ouyuanning, @aunjgr
handle null in convertRowsIntoBatch Approved by: @daviszhen
Fix enumtype system variable check Approved by: @daviszhen
support query replica count of special cn Approved by: @reusee, @daviszhen
把收发数据的功能从build算子中拆开，拆成merge+build,为后续的重构做准备 Approved by: @m-schen, @ouyuanning, @aunjgr
fileservice: longer timeout for http client Approved by: @zhangxu19830126
1. Reuse latest partition state. Approved by: @badboynt1, @m-schen, @XuPeng-SH
rm tag 16601 and 16597 Approved by: @heni02
optimize top operator in pipeline for tp query, don't need mergetop Approved by: @m-schen
optimize limit operator in pipeline for tp query, don't need toplimit Approved by: @m-schen
…es (matrixorigin#16694) add global system variable and session variable account isolation cases Approved by: @aressu1985
Add issue 16613 cases Approved by: @aressu1985
add case for hex() and unhex(). Approved by: @heni02
optimize group operator in pipeline for tp query, don't need mergegroup Approved by: @m-schen
issue上的问题是事务状态异常。 在出问题的调用栈上，增加事务状态的检测逻辑。 txnIsValid 判断事务状态是否异常。 Approved by: @badboynt1, @m-schen, @ouyuanning, @triump2020, @qingxinhome, @aunjgr
…in#16722) add optimizer hint exectype to force query to be ap or tp Approved by: @ouyuanning
support prefix bloom filter for object reader and writer Approved by: @XuPeng-SH
fileservice: remove IOEntry.ReadFromOSFile memorycache: remove RCBytes Approved by: @zhangxu19830126
optimize limit operator in pipeline for tp query, don't need mergeoffset Approved by: @m-schen
add issue 16139 cases Approved by: @aressu1985
SQL执行时将事务WriteOffset与当前语句绑定，解决读数据万圣节问题 MO Checkin Regression test susccess: https://github.com/matrixorigin/ci-test/actions/runs/9362961560 https://github.com/matrixorigin/ci-test/actions/runs/9379340928 Approved by: @daviszhen, @badboynt1, @m-schen, @reusee, @zhangxu19830126, @XuPeng-SH, @aunjgr, @triump2020
handle `ctrl+c` failed to cancel during restore data Approved by: @daviszhen
…gin#16737) fix a bug that cause ap performance regression on multi cn Approved by: @ouyuanning
把mergeorder算子拆成merge+mergeorder,把接收数据的功能独立出来。 对于tp query，pipeline直接改成scan->order->mergeorder即可，不需要通过connector-merge进行连接 Approved by: @ouyuanning, @m-schen
refine go runtime metrics dashboard Approved by: @zhangxu19830126
malloc: refactor config malloc: add chainDeallocator, FuncDeallocator; optimize metrics allocator malloc: optimize metrics allocator malloc: enable metrics default Approved by: @zhangxu19830126
skip stats for create view Approved by: @daviszhen, @aunjgr
add local disk s3 fs object storage for testing purposes Approved by: @zhangxu19830126
block reader supports between filters. Approved by: @XuPeng-SH, @heni02, @aunjgr
) limit和offset 的数据类型改为uint64。 Approved by: @m-schen, @reusee, @ouyuanning, @badboynt1, @aunjgr, @aressu1985
fix shard service panic Approved by: @reusee
Fix hung when add txn error trace Approved by: @iamlinjunhong
…origin#16209)" (matrixorigin#16754) Revert "resubmit pipeline client max connections is too large (matrixorigin#16209)" Approved by: @badboynt1, @daviszhen
…igin#16760) fix timeout mechanism when TN service startup: 5m timeout for total wait and 5s timeout for each request. Approved by: @zhangxu19830126
add XuPeng-SH as vector owner Approved by: @fengttt
…rigin#16756) 1. `>, >=, <, <=` 2. `between and, prefix in, prefix between, prefix eq` 3. `in, eq` Approved by: @XuPeng-SH, @aressu1985
Remove list checkpoint meta file Approved by: @XuPeng-SH
fix fd leak in LocalFS.read Approved by: @fengttt
…n#16766) Remove unnecessary projects from Master Index. ```sql mysql> explain analyze SELECT tbl.a100 FROM tbl WHERE tbl.a75 = 'I2nJ0RqIQu'; +---------------------------------------------------------------------------------------------------------------------------------------------------------+ | AP QUERY PLAN ON MULTICN(10 core) | +---------------------------------------------------------------------------------------------------------------------------------------------------------+ | Project | | Analyze: timeConsumed=0ms waitTime=6ms inputRows=2 outputRows=1 InputSize=48bytes OutputSize=24bytes MemorySize=0bytes | | -> Join | | Analyze: timeConsumed=2ms waitTime=44ms inputRows=2 outputRows=1 InputSize=48bytes OutputSize=24bytes MemorySize=0bytes | | Join Type: INDEX | | Join Cond: (tbl.a100 = #[1,0]) | | Runtime Filter Build: #[-1,0] | | -> Table Scan on a.tbl [ForceOneCN] | | Analyze: timeConsumed=0ms waitTime=0ms inputBlocks=1 inputRows=1 outputRows=1 InputSize=48bytes OutputSize=24bytes MemorySize=50bytes | | Filter Cond: (tbl.a75 = 'I2nJ0RqIQu') | | Block Filter Cond: (tbl.a75 = 'I2nJ0RqIQu') | | Runtime Filter Probe: tbl.a100 | | -> Table Scan on a.__mo_index_secondary_019003d8-3fd8-7455-b750-bd977ca13178 [ForceOneCN] | | Analyze: timeConsumed=2ms waitTime=0ms inputBlocks=6 inputRows=49152 outputRows=1 InputSize=3mb OutputSize=24bytes MemorySize=696320bytes | | Filter Cond: prefix_eq(#[0,0], 'F74 FI2nJ0RqIQu ') | | Block Filter Cond: prefix_eq(#[0,0], 'F74 FI2nJ0RqIQu ') | +---------------------------------------------------------------------------------------------------------------------------------------------------------+ 16 rows in set (0.01 sec) mysql> explain analyze SELECT tbl.a100 FROM tbl WHERE tbl.a37 = '3Tfm6CEXy5' AND tbl.a94 = '6PRBdXpsVB'; +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | AP QUERY PLAN ON MULTICN(10 core) | +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Project | | Analyze: timeConsumed=0ms waitTime=15ms inputRows=3 outputRows=1 InputSize=72bytes OutputSize=24bytes MemorySize=0bytes | | -> Join | | Analyze: timeConsumed=4ms waitTime=72ms inputRows=2 outputRows=1 InputSize=48bytes OutputSize=24bytes MemorySize=0bytes | | Join Type: INDEX | | Join Cond: (tbl.a100 = #[1,0]) | | Runtime Filter Build: #[-1,0] | | -> Table Scan on a.tbl [ForceOneCN] | | Analyze: timeConsumed=0ms waitTime=0ms inputBlocks=1 inputRows=1 outputRows=1 InputSize=72bytes OutputSize=24bytes MemorySize=75bytes | | Filter Cond: (tbl.a94 = '6PRBdXpsVB'), (tbl.a37 = '3Tfm6CEXy5') | | Block Filter Cond: (tbl.a94 = '6PRBdXpsVB'), (tbl.a37 = '3Tfm6CEXy5') | | Runtime Filter Probe: tbl.a100 | | -> Join | | Analyze: timeConsumed=4ms probe_time=[total=0ms,min=0ms,max=0ms,dop=10] build_time=[4ms] waitTime=57ms inputRows=2 outputRows=1 InputSize=48bytes OutputSize=24bytes MemorySize=361187bytes | | Join Type: INNER | | Join Cond: (#[0,0] = #[1,0]) | | -> Table Scan on a.__mo_index_secondary_019003d8-3fd8-7455-b750-bd977ca13178 [ForceOneCN] | | Analyze: timeConsumed=3ms waitTime=0ms inputBlocks=4 inputRows=32768 outputRows=1 InputSize=2mb OutputSize=24bytes MemorySize=679936bytes | | Filter Cond: prefix_eq(#[0,0], 'F36 F3Tfm6CEXy5 ') | | Block Filter Cond: prefix_eq(#[0,0], 'F36 F3Tfm6CEXy5 ') | | -> Table Scan on a.__mo_index_secondary_019003d8-3fd8-7455-b750-bd977ca13178 [ForceOneCN] | | Analyze: timeConsumed=3ms waitTime=0ms inputBlocks=6 inputRows=49152 outputRows=1 InputSize=3mb OutputSize=24bytes MemorySize=696320bytes | | Filter Cond: prefix_eq(#[0,0], 'F93 F6PRBdXpsVB ') | | Block Filter Cond: prefix_eq(#[0,0], 'F93 F6PRBdXpsVB ') | +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 24 rows in set (0.00 sec) ``` Approved by: @badboynt1
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Which issue(s) this PR fixes:
issue #
What this PR does / why we need it: