Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: synchronize the code to the community #472

Merged
merged 39 commits into from
Feb 2, 2024
Merged

chore: synchronize the code to the community #472

merged 39 commits into from
Feb 2, 2024

Conversation

shilinlee
Copy link
Member

How Has This Been Tested?

  • Test team
  • UT
  • IT

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

openGemini added 11 commits February 2, 2024 13:26
@shilinlee
fix(log): write interface default type content type fix
1a8ad6a 
Match-id-d6bad274f6aca6d1a42228233df86dcbc9f252c4

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(index): support AND/OR and compound expr query for indexes
8f6d7b2 
Match-id-08ed00a071ccfb600a3ab9655d4894180632684d

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
fix: full text index missing data
f24a259 
Match-id-92c31a1b3757cd4137c4fc6c2fe4efe1991d20a9

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
fix: mulitpt query of ts-server bug
d505b46 
Match-id-027bb1d75852ba8bc99e9623ee50807cb74fcf15

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(logstore): add bloomfilter full_text query
5759de9 
Match-id-31f3df98cbdf2287b28a832b2b0846e2c99c65e8

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(storage): add lz4 compression
a9edad8 
Match-id-a234617a40a13dee40f6f8e5360940781e9727ab

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
fix: resolved the problem that memdata is more queried
1a5360c 
Match-id-77cbfc5372ffa4fec927ca3e2d0b55f294b8aa7d

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
fix(logstore): fix bug of filter data get nil
cbd7cc1 
Match-id-ac9b4b151606496852cff30bb03d2386bd084c0e

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(logstore): remove the wildcard in pipe query
ac44dc3 
Match-id-f917e4daf92897fe1c563153974ab041a5f4417f

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
fix: add the rpn element by full text index
fc63fbe 
Match-id-0371ec6d9e6bd4a65f36ceaeacfed3dac4a017c5

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
perf: fileloopcursor use a new record pool
fbf9cc1 
Match-id-a88d877677480c88cd90c2052e9be50c86b70536

Signed-off-by: shilinlee <836160610@qq.com>
openGemini added 18 commits February 2, 2024 13:33
@shilinlee
chore: format the etc fstab file
bf6d81d 
Match-id-291744f61ac4e32b39bdab39d6134bc4ce95cbc3

Signed-off-by: shilinlee <836160610@qq.com>

Signed-off-by: openGemini <>
@shilinlee
feat(logstore): fix the EXTRACT error in log-query interface
e2c7695 
Match-id-af92948367fd98071fdb35a4b4dbc003871c0708

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
perf: the query performance is optimized with valueIndexRange
b50e5d3 
Match-id-12d63e6fd79913b0621f442a65f3b80f1b25080d

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
perf: optimize pre-aggregated data merge logic
22e9589 
Match-id-e2c0bfb40d4161377fc43b1160bbcebc1ad14217

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(ToCloud): check if a shard can move from dfv to obs
b373328 
Match-id-f760528aad93836a77baf62155a51311f364716a

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(logstore): fix the EXTRACT error in log-query interface
128ead7 
Match-id-c5aed0cfb111cf7f9567e54f807eb75dc1be3741

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
fix(logstore): fix bug of filter data
bff0e81 
Match-id-4d70cde7b1fe605f81724e45c0c952b9e973cf1c

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat: query api adaptation
0737497 
Match-id-af3b434f92f059bae470df7cec23d911b29dfed4

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(metric): match obs reader to read same filed
1c9f37d 
Match-id-9cf6c1b7c547952017b73a5ae4307f257a9b6ec3

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(storage): add seqId col to data
ec2a019 
Match-id-beaca247213ac506ddd46c55f07cd6df4299763b

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(metric): match obs reader to read same filed
b22e99e 
Match-id-9e46cb24844fe46fda02703dbc4e33615db6115a

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(storage): cut preSchema while product is logkeeper
e95efaa 
Match-id-12bc587e6118eca402658810c8faac071b631e36

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
fix: control the reader concurrency for subquery
8898bdb 
Match-id-6f02c61d06999197af67ec4d4f9821c3018e4b6b

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(ToCloud): return http 400 when measurement name len is over 255
9acb84e 
Match-id-f0cbd05b673d2c2940fa27208a1a1fcd9142098c

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(metric): fix calculation errors about the HashAgg operator
19829a2 
Match-id-c4edd636523631e394928cbabf31d9b00148a1fb

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(ToCloud): fix batch process when where is a parse error
b405727 
Match-id-4874aff0f6d97758acccd84ea10f679897819d3c

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
chore: add sherlock configs for cloud deployment
d58721e 
Signed-off-by: l00601186 <shilin.lee@huawei.com>

merge enable-sherlock into develop

chore: add sherlock configs for cloud deployment

Created-by: lishilin 00601186
Author-id: 537952
MR-id: 29555336
Commit-by: l00601186
Merged-by: zhouyijian 00618253
E2E-issues: US2022090800590
Description: chore: add sherlock configs for cloud deployment

Signed-off-by: l00601186 <shilin.lee@huawei.com>

See merge request: CTO_Technical_Innovation/Database/Gemini/openGemini!2718

Match-id-ec60316f99867b5c26feb17caf1d420fbcd27056

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(ToCloud): update mst length check
ad794e2 
Match-id-efd70503a63904080f54b90e747fbfbf18fe28e9

Signed-off-by: shilinlee <836160610@qq.com>
openGemini added 8 commits February 2, 2024 13:34
@shilinlee
chore: add a pre-aggregated storage configuration
c32d043 
Match-id-348f2e6bda2494341834da1660a5ce2c792506aa

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
fix(logstore): move the code of indexType to lib
8f53614 
Match-id-4cb7b39c4b6b36b407f4443d7720991f13f1b2fd

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
fix: cancel aggpushdownsubquery rule when no preagg
3d923f4 
Match-id-fccff8abbc2dba9ced76b1610b212a312073db7b

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(convergence): add cursor for page query
80d110e 
Match-id-e583ffc5d02256a183876b831a9152a3dea985e6

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(granfana): add pipe parser for query interface
ae89039 
Match-id-57b629b32e69a0f6a0ad71de6814f44f84d01932

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
fix: missing pre-aggregated data for the time column
4444bb0 
Match-id-24ca841409bddb40eadc3984c40219011b6d4c61

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
chore: count the number of rows only in range mode
19013bd 
Match-id-745513e6aae9144b5633af7f5b56341f2948119e

Signed-off-by: shilinlee <836160610@qq.com>
@shilinlee
feat(timecluster): adaptation the time cluster query
e89abf2 
Match-id-5b642549d42d1ef3dc18f4cf1b9681cedc0baaef

Signed-off-by: shilinlee <836160610@qq.com>
openGemini added 2 commits February 2, 2024 13:36
@shilinlee
fix: sql node does not read the pre-aggregation configuration item
2284bea 
Match-id-2432e503d52edb8619c346a615c00a2a15511e3f

Signed-off-by: shilinlee <836160610@qq.com>

Signed-off-by: openGemini <>
@shilinlee
chore: remove fstab uuid format script
fb725a5 
Match-id-33846641876a1d2266b54eb50dce8dfcfbe0e940

Signed-off-by: shilinlee <836160610@qq.com>
@codecov Codecov
Copy link

codecov bot commented Feb 2, 2024

Codecov Report

Attention: 310 lines in your changes are missing coverage. Please review.

Comparison is base (e2b367a) 74.72% compared to head (fb725a5) 74.70%.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #472      +/-   ##
==========================================
- Coverage   74.72%   74.70%   -0.03%     
==========================================
  Files         556      558       +2     
  Lines      117150   118230    +1080     
==========================================
+ Hits        87541    88324     +783     
- Misses      24165    24386     +221     
- Partials     5444     5520      +76

@huang-feiteng huang-feiteng merged commit 77c3c50 into openGemini:main Feb 2, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@huang-feiteng huang-feiteng huang-feiteng approved these changes

@xiangyu5632 xiangyu5632 xiangyu5632 approved these changes

@scuzyj scuzyj Awaiting requested review from scuzyj

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@shilinlee @huang-feiteng @xiangyu5632