Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge internal 11-23 #207

Merged
merged 6 commits into from Dec 30, 2022
Merged

Merge internal 11-23 #207

merged 6 commits into from Dec 30, 2022

Conversation

lgqss
Copy link
Collaborator

@lgqss lgqss commented Dec 29, 2022

Notice:

滚动升级顺序:BaikalMeta=>BaikalStore=>BaikalDB(本次升级需要严格按照这个顺序,否则会出现兼容性问题)
回滚步骤反过来:BaikalDB=>BaikalStore=>BaikalMeta
如没特殊说明,后续升级步骤都应该按上述顺序进行

New Features:

  • 内部现在适配的是gcc12,rocksdb 7.7.3(c++17),protobuf_3-18-0-1,本次开源版本依赖未变,有能力同学可以适配下bazel依赖
  • 支持注释调整执行并发,防止脚本类大sql打挂集群,/*{"single_store_concurrency":1}*/
  • mysql协议增加keepalive,降低客户端挂掉后事务残留时间
  • 支持mysql建表语句,改成同步建表,建表后即可查询
  • fulltext支持utf8编码
  • 支持修改表的charset,ALTER TABLE tb DEFAULT CHARSET= gbk;
  • 支持hash partition自定义表达式,partition功能达到可用状态,举例:PARTITION BY HASH (key & 0xABB % 16) PARTITIONS 16
  • 支持单表多集群(resource_tag)概念,多副本可以在不同集群,例如:"dists": [ { "resource_tag":"a1","logical_room":"x", "count":2}, { "resource_tag":"a1","logical_room":"y", "count":1}, { "resource_tag":"a2","logical_room":"x", "count":1}, { "resource_tag":"a3","logical_room":"y", "count":1}]
  • baikaldb支持访问指定的resource_tag,-fetcher_resource_tag=a1
  • baikaldb支持follower read读取最新数据,配置-use_read_index=true
  • 分裂优化,分裂耗时更短
  • ddl流程优化
  • 支持\0转义
  • 支持按user设置不同的锁超时时间
  • store和db均支持sign黑名单拒绝:handle schema_conf {table_name} add_blacklist {sign}
  • 迁移限速判断l0 sst数量,降低stall概率
  • not null无default值时,插入null兼容mysql非严格模式,例如int类型会写入0,string类型写入空串
  • binlog表增加字段binlog_row_cnt bigint(20) NOT NULL DEFAULT '1'

Bug Fixes:

  • agg空数据修复
  • 修复多项事务bug,完善幂等处理,事务更稳定
  • 修复a join b using(id)语法
  • 内存限制bugfix
  • 时间类型,时间函数等修复和兼容mysql
  • insert ignore LOCK bugfix
  • subquery txn_id bugfix
  • substring_index空串死循环修复
  • 修复update set a=a+1并发问题
  • double to string增加精度
  • insert select 修复时间类型

Performance Improvements:

  • 支持multiget,默认打开:-scan_use_multi_get=true
  • baikaldb索引计算优化
  • baikalstore减少数据复制
  • binlog支持multiget
  • join性能优化,降低planrouter的开销
  • ttl cache优化
  • or,and支持多叉树
  • a = 1or a= 2改写成a in (1,2);(a=1 and b=1) or (a=2 and b=2)改成成(a,b) in ((1,1),(2,2)),满足后续索引选择需求

@lgqss lgqss merged commit e669e4f into baidu:master Dec 30, 2022
@codeoos
Copy link

codeoos commented Jan 31, 2023

请问内部protobuf版本是3.18.0还是3.18.1呢?protobuf并没有3.18.0.1这个版本

@lgqss lgqss deleted the merge_11-23 branch June 20, 2023 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants