Release notes 2.0.0-GA

Release date: Jan 4, 2022

New Feature

• External Table
  • [Experimental Function]Support for Hive external table on S3
  • DecimalV3 support for external table #425
• Implement complex expressions to be pushed down to the storage layer for computation, thus gaining performance gains
• Primary Key is officially released, which supports Stream Load, Broker Load, Routine Load, and also provides a second-level synchronization tool for MySQL data based on Flink-cdc

Improvement

• Arithmetic operators optimization
  • Optimize the performance of dictionary with low cardinality #791
  • Optimize the scan performance of int for single table #273
  • Optimize the performance of count(distinct int) with high cardinality #139 #250 #544#570
  • Execution level optimization and refinement Group by 2 int / limit / case when / not equal
  • Optimize Group by 2 int / limit / case when / not equal in implementation-level
• Memory management optimization
  • Refactor the memory statistics and control framework to accurately count memory usage and completely solve OOM
  • Optimize metadata memory usage
  • Solve the problem of large memory release stuck in execution threads for a long time
  • Add process graceful exit mechanism and support memory leak check #1093

Bugfix

• Fix the problem that the Hive external table is timeout to get metadata in a large amount.
• Fix the problem of unclear error message of materialized view creation.
• Fix the implementation of like in vectorization engine #722
• Repair the error of parsing the predicate is in alter table #725
• Fix the problem that the curdate function can not format the date.

Release notes 1.19.5

Improvement

• Improve shuffle bucket plan (#2184)
• Add a num_segment threshold when loading multiple big files (#2067)

Bugfix

• Upgrade Log4j2 to 2.17.0 (#2284) (#2290)
• Fix query hive empty partition Error #707 (#2082)

Release notes 1.19.4

Imporvement

• support cast(varchar as bitmap) (#1941)
• Modify Hive external table scan scheduling strategy (#1394) (#1807)

Bugfix

• Fix cross join lose predicate in JoinAssociativityRule (#1918)
• Fix cast to decimal(0,0) bug (#1709) (#1738)
• Fix replicate join in plan fragment builder (#1727)
• Fix several planner cost calculation

Release notes 1.19.3

Improvement

Upgrade jprotobuf to enhance security (#1506)

Major Bugfix

Fix some CBO bad cases and correctness issues.
Fix grouping sets with same column bug(#1395) (#1119)
Fix some date function issues (#1385) (#1627)
Fix streaming aggregation issue（#1584）

Release notes 1.19.2

Improvement

• bucket shuffle join support right join and full outer join(#1209) (#1234)

Major Bugfix

• Support push down predicate through repeat node(#1410) (#1417)
• Fix routine load data losing bug when FE master changed (#1074) (#1272)
• Fix create view failed with union (#1083)
• Fix some Hive external table stability issues(#1408)
• Fix select group by view error (#1231)

Release notes 1.19.1

Improvement

• Optimize the performance of show frontends. # 507 # 984
• Add monitoring of slow queries and meta logs. # 502 # 891
• Optimize the fetching of Hive external metadata to achieve parallel fetching.# 425 # 451

BugFix

• Fix the problem of Thrift protocol compatibility, so that the Hive external table can be connected with Kerberos. # 184 # 947 # 995 # 999
• Fix several bugs in view creation. # 972 # 987# 1001
• Fix the problem that FE cannot be upgraded in grayscale. # 485 # 890

Release notes 1.19.0

New Feature

• Implement Global Runtime Filter, which can enable runtime filter for shuffle join.
• CBO Planner is enabled by default, improved colocated join, bucket shuffle, statistical information estimation, etc.
• [Experimental Function] Primary Key model release: To better support real-time/frequent update features, StarRocks has added a new table type: primary key model. The model supports Stream Load, Broker Load, Routine Load, JSON import, and also provides a second-level synchronization tool for MySQL data based on Flink-cdc.
• [Experimental Function] Support write function for external tables. Support writing data to another StarRocks cluster table by external tables to solve the read/write separation requirement and provide better resource isolation.

Improvement

• Performance optimization.
  • count distinct int statement
  • group by int statement
  • or statement
• Optimize disk balance algorithm. Data can be automatically balanced after adding disks to a single machine.
• Support partial column export.
• Optimize show processlist to show specific SQL.
• Support multiple variable settings in SET_VAR .
• Improve the error reporting information, including table_sink, routine load, creation of materialized view, etc.

Bugfix

• Fix the issue that the dynamic partition table cannot be created automatically after the data recovery operation is completed. # 337
• Fix the problem of error reported by row_number function after CBO is opened.
• Fix the problem of FE stuck due to statistical information collection
• Fix the problem that set_var takes effect for session but not for statements.
• Fix the problem that select count(*) returns abnormality on the Hive partition external table.