tidb-server v5.1.2

Improvements

• Trigger auto-analyze by histogram row count, increase the accuracy of this trigger action #24237
• Push down mod() to TiFlash, increase the query performance. #2318

Bug Fixes

• Fix the potential wrong results of index hash join when the hash column is in the ENUM type #27893
• Fix a batch client bug that recycle idle connection may block sending requests in some rare cases #27678
• Fix the compatibility issue of the overflow check by keeping the same logic as MySQL #23897
• Fix the issue that TiDB returns an unknow error while it should return the pd is timeout error #26147
• Fix the error of the case when function caused by the wrong charset and collation #26662
• Fix the issue that greatest(datetime) union null returns an empty string #26532
• Fix the potential can not found column in Schema column error for MPP queries #28148
• Fix a bug that TiDB may panic when TiFlash is shutting down #28096
• Fix the issue of wrong range caused by using enum like 'x%' #27130
• Fix the issue that the between expression infers wrong collation. #27146
• Fix the Common Table Expression (CTE) dead lock issue when used with IndexLookupJoin #27410
• Fix a bug that retryable deadlocks are incorrectly recorded into the INFORMATION_SCHEMA.DEADLOCKS table #27400
• Fix a bug that GROUP_CONCAT function does not consider the collation #27429
• Fix a bug that the COUNT(DISTINCT) function on multiple columns returns wrong results when New Collation is on #27091
• Fix an issue that the TABLESAMPLE query result from partitioned tables is not sorted as expected #27349
• Fix a bug that the EXTRACT function returns wrong results when the argument is a negative duration #27236
• Remove the unused /debug/sub-optimal-plan HTTP API #27265
• Fix the issue that a wrong selection is pushed down when the HAVING condition is used in the aggregate function #26496
• Fix a bug that the query may return wrong results when the hash partition table deals with unsigned data #26569
• Fix the unexpected behavior when casting an invalid string to DATE #26762
• Fix a bug that creating partition fails if NO_UNSIGNED_SUBTRACTION is set #26765
• Fix the issue that the distinct flag is missing when Apply is converted to Join #26958
• Fix the issue that NO_ZERO_IN_DATE does not work on the default values #26766
• Set a block duration for the newly recovered TiFlash node to avoid blocking queries during this time #26897
• Fix a bug that might occur when the CTE is referenced more than once #26212
• Fix a CTE bug when MergeJoin is used #25474
• Fix a bug that the 'SELECT FOR UPDATE' statement does not correctly lock the data when a normal table joins a partition table #26251
• Fix the issue that the 'SELECT FOR UPDATE' statement returns an error when a normal table joins a partition table #26250
• Fix the issue that PointGet does not use the lite version of resolve lock #26562

tidb-server v4.0.15

Compatibility changes

• Fix the issue that executing SHOW VARIABLES in a new session is slow. This fix reverts some changes made in #21045 and might cause compatibility issues. #24326

Improvements

• Trigger auto-analyze based on the histogram row count #26706

Bug fixes

• Fix a bug that collation is incorrectly set for binary literals when building range 26455
• Fix the issue of wrong character set and collation for the case when expression #26671
• Fix the "index out of range" error that occurs when a query includes both GROUP BY and UNION 26553
• Fix the issue that TiDB might fail to send requests if TiKV has tombstone stores #25849
• Fix the issue of unexpected behavior when casting the invalid string to DATE #27935
• Fix the issue that column information is missed when converting the Apply operator to Join #27282
• Fix a bug that the count distinct result on multiple columns is wrong when the new collation is enabled #27830
• Fix the result wrong that occurs when the argument of the extract function is a negative duration #27236
• Fix the wrong execution results that occur when the column in the group_concat function has a non-bin collation #27429
• Fix the wrong execution results that occur when the collations around the between expression are different #27146
• Fix the issue that greatest(datetime) union null returns empty string #26564
• Fix the issue that the having clause might not work correctly #26496
• Remove the undocumented /debug/sub-optimal-plan HTTP API #27264

tidb-server v5.2.1

Bug Fixes

• Fix an error that occurs during execution caused by the wrong execution plan. The wrong execution plan is caused by the shallow copy of schema columns when pushing down the aggregation operators on partitioned tables. #27797 #26554

tidb-server v5.0.5

There is no release note for this version.

tidb-server v5.0.4

Compatibility Changes

• Fix the issue that executing SHOW VARIABLES in a new session is slow. This fix reverts some changes made in #19341 and might cause compatibility issues. #24326
• Change the default value of the tidb_stmt_summary_max_stmt_count variable from 200 to 3000 #25873
• The following bug fixes change execution results, which might cause upgrade incompatibilities:
  • Fix the issue that TiDB returns wrong result when the children of UNION contain the NULL value #26559
  • Fix the issue that greatest(datetime) union null returns empty string #26532
  • Fix the issue that the behavior of the last_day function is incompatible in SQL mode #26000
  • Fix the issue that the having clause might not work correctly #26496
  • Fix the wrong execution results that occur when the collations around the between expression are different #27146
  • Fix the wrong execution results that occur when the column in the group_concat function has a non-bin collation #27429
  • Fix an issue that using a count(distinct) expression on multiple columns returns wrong result when the new collation is enabled #27091
  • Fix the result wrong that occurs when the argument of the extract function is a negative duration #27236
  • Fix the issue that inserting an invalid date does not report an error when the SQL_MODE is 'STRICT_TRANS_TABLES' #26762
  • Fix the issue that using an invalid default date does not report an error when the SQL_MODE is 'NO_ZERO_IN_DATE' #26766
  • Fix a bug on the query range of prefix index #26029
  • Fix the issue that the LOAD DATA statement might abnormally import non-utf8 data #25979
  • Fix the issue that insert ignore on duplicate update might insert wrong data when the secondary index has the same column with the primary key #25809
  • Fix the issue that insert ignore duplicate update might insert wrong data when a partitioned table has a clustered index #25846
  • Fix the issue that the query result might be wrong when the key is the ENUM type in point get or batch point get #24562
  • Fix the wrong result that occurs when dividing a BIT-type value #23479
  • Fix the issue that the results of prepared statements and direct queries might be inconsistent #22949
  • Fix the issue that the query result might be wrong when a YEAR type is compared with a string or an integer type #23262

Feature enhancements

• Support setting tidb_enforce_mpp=1 to ignore the optimizer estimation and forcibly use the MPP mode #26382

Improvements

• Trigger auto-analyze based on the histogram row count #24237
  • Stop sending requests to a TiFlash node for a period if the node has failed and restarted before #26757
  • Increase the split region upper limit to make split table and presplit more stable #26657
  • Support retry for MPP queries #26483
  • Check the availability of TiFlash before launching MPP queries #1807
  • Support the stable result mode to make the query result more stable #26084
  • Support the MySQL system variable init_connect and its associated features #18894
  • Thoroughly push down the COUNT(DISTINCT) aggregation function in the MPP mode #25861
  • Print log warnings when the aggregation function cannot be pushed down in EXPLAIN statements #25736
  • Add error labels for TiFlashQueryTotalCounter in Grafana dashboard #25327
  • Support getting the MVCC data of a clustered index table through a secondary index by HTTP API #24209
  • Optimize the memory allocation of prepared statement in parser #24371

Bug Fixes

• Fix the issue that TiDB might panic when querying a partitioned table and the partition key has the IS NULL condition #23802
  • Fix the issue that the overflow check of the FLOAT64 type is different with that of MySQL #23897
  • Fix the wrong character set and collation for the case when expression #26662
  • Fix the issue that committing pessimistic transactions might cause write conflicts #25964
  • Fix a bug that the index keys in a pessimistic transaction might be repeatedly committed #26359 #10600
  • Fix the issue that TiDB might panic when resolving the async commit locks #25778
  • Fix a bug that a column might not be found when using INDEX MERGE #25045
  • Fix a bug that ALTER USER REQUIRE SSL clears users' authentication_string #25225
  • Fix a bug that the value of the tidb_gc_scan_lock_mode global variable on a new cluster shows "PHYSICAL" instead of the actual default mode "LEGACY" #25100
  • Fix the bug that the TIKV_REGION_PEERS system table does not show the correct DOWN status #24879
  • Fix the issue of memory leaks that occurs when HTTP API is used #24649
  • Fix the issue that views do not support DEFINER #24414
  • Fix the issue that tidb-server --help exits with the code 2 #24046
  • Fix the issue that setting the global variable dml_batch_size does not take effect #24709
  • Fix the issue that using read_from_storage and partitioned table at the same time causes an error #20372
  • Fix the issue that TiDB panics when executing the projection operator #24264
  • Fix the issue that statistics might cause queries to panic #24061
  • Fix the issue that using the approx_percentile function on a BIT column might panic #23662
  • Fix the issue that the metrics on the Coprocessor Cache panel in Grafana are wrong #26338
  • Fix the issue that concurrently truncating the same partition causes DDL statements to stuck #26229
  • Fix the issue of wrong query results that occurs when the session variable is used as the GROUP BY item #27106
  • Fix the wrong implicit conversion between VARCHAR and timestamp when joining tables #25902
  • Fix the wrong results in associated subquery statements #27233

tidb-server v5.2.0

Improvements

• Support pushing down the built-in function json_unquote() to TiKV #24415
• Support removing the union branch from the dual table #25614
• Optimize the aggregate operator's cost factor #25241
• Allow the MPP outer join to choose the build table based on the table row count #25142
• Support balancing the MPP query workload among different TiFlash nodes based on Regions #24724
• Support invalidating stale Regions in the cache after the MPP query is executed #24432
• Improve the MySQL compatibility of the built-in functionstr_to_date for the format specifiers %b/%M/%r/%T #25767
• Fix the issue that inconsistent binding caches might be created in multiple TiDB after recreating different bindings for the same query #26015
• Fix the issue that the existing bindings cannot be loaded into cache after upgrade #23295
• Support ordering the result of SHOW BINDINGS by (original_sql, update_time) #26139
• Improve the logic of query optimization when bindings exist, and reduce optimization times of a query #26141
• Support completing the garbage collection automatically for the bindings in the "deleted" status #26206
• Support showing whether a binding is used for query optimization in the result of EXPLAIN VERBOSE #26930
• Add a new status variation last_plan_binding_update_time to view the timestamp corresponding to the binding cache in the current TiDB instance #26340
• Support reporting an error when starting binding evolution or running admin evolve bindings to ban the baseline evolution (currently disabled in the on-premises TiDB version because it is an experimental feature) affecting other features #26333

Bug Fixes

• Fix the issue that an incorrect result is returned when using merge join on the SET type column #25669
• Fix the data corruption issue in the IN expression's arguments #25591
• Avoid the sessions of GC being affected by global variables #24976
• Fix the panic issue that occurs when using limit in the window function queries #25344
• Fix the wrong value returned when querying a partitioned table using Limit #24636
• Fix the issue that IFNULL does not correctly take effect on the ENUM or SET type column #24944
• Fix the wrong results caused by changing the count in the join subqueries to first_row #24865
• Fix the query hang issue that occurs when ParallelApply is used under the TopN operator #24930
• Fix the issue that more results than expected are returned when executing SQL statements using multi-column prefix indexes #24356
• Fix the issue that the <=> operator cannot correctly take effect #24477
• Fix the data race issue of the parallel Apply operator #23280
• Fix the issue that the index out of range error is reported when sorting the IndexMerge results of the PartitionUnion operator #23919
• Fix the issue that setting the tidb_snapshot variable to an unexpectedly large value might damage the transaction isolation #25680
• Fix the issue that the ODBC-styled constant (for example, {d '2020-01-01'}) cannot be used as the expression #25531
• Fix the issue that SELECT DISTINCT converted to Batch Get causes incorrect results #25320
• Fix the issue that backing off queries from TiFlash to TiKV cannot be triggered #23665 #24421
• Fix the index-out-of-range error that occurs when checking only_full_group_by #23839)
• Fix the issue that the result of index join in correlated subqueries is wrong #25799

tidb-server v5.1.1

Compatibility changes

• For TiDB clusters upgrade from v4.0 to v5.1, the default value of tidb_multi_statement_mode is OFF. It is recommended to use the multi-statement feature of your client library instead. See the documentation on tidb_multi_statement_mode for details. #25751
• Change the default value of the tidb_stmt_summary_max_stmt_count variable from 200 to 3000 #25874
• Require the SUPER privilege to access the table_storage_stats table #26352
• Require the SELECT privilege on mysql.user to access the information_schema.user_privileges table to show other user's privileges #26311
• Require the CONFIG privilege to access the information_schema.cluster_hardware table #26297
• Require the PROCESS privilege to access the information_schema.cluster_info table #26297
• Require the PROCESS privilege to access the information_schema.cluster_load table #26297
• Require the PROCESS privilege to access the information_schema.cluster_systeminfo table #26297
• Require the PROCESS privilege to access the information_schema.cluster_log table #26297
• Require the CONFIG privilege to access the information_schema.cluster_config table #26150

Improvements

• Announce the general availability (GA) of the Stale Read feature
• Avoid allocation for paramMarker to speed up data insertion #26076
• Support the stable result mode to make the query results more stable #25995
• Support pushing down the built-in function json_unquote() to TiKV #26265
• Support retrying MPP queries #26480
• Change the LOCK record into the PUT record for the index keys using point get or batch point get for UPDATE reads #26225
• Forbid creating views from stale queries #26200
• Thoroughly push down the COUNT(DISTINCT) aggregation function in the MPP mode #26194
• Check the availability of TiFlash before launching MPP queries #26192
• Do not allow setting the read timestamp to a future time #25763
• Print log warnings when aggregation functions cannot be pushed down in EXPLAIN statements #25737
• Add the statements_summary_evicted table to record the evicted count information of a cluster #25587
• Improve the MySQL compatibility of the built-in function str_to_date for the format specifiers %b/%M/%r/%T #25768

Bug fixes

• Fix the data loss issue that might occur when changing the column type with tidb_enable_amend_pessimistic_txn=on #26203
• Fix the issue that the behavior of the last_day function is incompatible in the SQL mode #26001
• Fix the panic issue that might occur when LIMIT is on top of window functions #25344
• Fix the issue that committing pessimistic transactions might cause write conflict #25964
• Fix the issue that the result of index join in correlated subqueries is wrong #25799
• Fix a bug that the successfully committed optimistic transactions might report commit errors #10468
• Fix the issue that an incorrect result is returned when using merge join on the SET type column #25669
• Fix a bug that the index keys in a pessimistic transaction might be repeatedly committed #26359
• Fix the risk of integer overflow when the optimizer is locating partitions #26227
• Fix the issue that invalid values might be written when casting DATE to timestamp #26292
• Fix the issue that the Coprocessor Cache metrics are not displayed on Grafana #26338
• Fix the issue of annoying logs caused by telemetry #25760 #25785
• Fix a bug on the query range of prefix index #26029
• Fix the issue that concurrently truncating the same partition hangs DDL executions #26229
• Fix the issue of duplicate ENUM items #25955
• Fix a bug that the CTE iterator is not correctly closed #26112
• Fix the issue that the LOAD DATA statement might abnormally import non-utf8 data #25979
• Fix the panic issue that might occur when using the window function on the unsigned integer columns #25956
• Fix the issue that TiDB might panic when resolving async commit locks #25778
• Fix the issue that Stale Read is not fully compatible with the PREPARE statements #25800
• Fix the issue that the ODBC-styled constant (for example, {d '2020-01-01'}) cannot be used as the expression #25531
• Fix an error that occurs when running TiDB alone #25555

tidb-server v4.0.14

Compatibility changes

• Change the default value of tidb_multi_statement_mode from WARN to OFF in v4.0. It is recommended to use the multi-statement feature of your client library instead. See the documentation on tidb_multi_statement_mode for details. #25749
• Upgrade Grafana dashboard from v6.1.16 to v7.5.7 to solve two security vulnerabilities. See the Grafana blog post for details.
• Change the default value of the tidb_stmt_summary_max_stmt_count variable from 200 to 3000 #25872

Improvements

• Change the LOCK record into the PUT record for the index keys using point get or batch point get for UPDATE reads #26223
• Support the MySQL system variable init_connect and its associated features #26031
• Support the stable result mode to make the query results more stable #26003
• Support pushing down the built-in function json_unquote() to TiKV #25721
• Make the SQL Plan Management (SPM) not affected by the character set #23295

Bug fixes

• Fix the issue that the SELECT result is incompatible with MySQL when joining a subquery with a WHERE clause evaluated to false #24865
• Fix the calculation error of the ifnull function that occurs when the argument is the ENUM or SET type #24944
• Fix the wrong aggregate pruning in some cases #25202
• Fix the incorrect result of the merge join operation that might occur when the column is the SET type #25669
• Fix the issue that TiDB returns wrong results for cartesian join #25591
• Fix the panic issue that occurs when SELECT ... FOR UPDATE works on a join operation and the join uses a partitioned table #20028
• Fix the issue that the cached prepared plan is incorrectly used for point get #24741
• Fix the issue that the LOAD DATA statement can abnormally import non-utf8 data #25979
• Fix a potential memory leak issue that occurs when accessing the statistics via an HTTP API #24650
• Fix a security issue that occurs when executing the ALTER USER statement #25225
• Fix a bug that the TIKV_REGION_PEERS table cannot correctly handle the DOWN status #24879
• Fix the issue that invalid strings are not truncated when parsing DateTime #22231
• Fix the issue that the select into outfile statement might have no result when the column type is YEAR #22159
• Fix the issue that the query result might be wrong when NULL is in the UNION subquery #26532
• Fix the issue that the projection operator in execution might cause panic in some cases #26534

tidb-server v5.0.3

Compatibility Changes

• After a v4.0 cluster is upgraded to v5.0 or a later version (dev or v5.1), the default value of the tidb_multi_statement_mode variable changes from WARN to OFF
• TiDB is now compatible with MySQL 5.7's noop variable innodb_default_row_format #23541

Improvements

• Support pushing down the TopN operator to TiFlash #25162
• Support pushing down the built-in function json_unquote() to TiKV #24415
• Support removing the union branch from the dual table #25614
• Support pushing down the built-in function replace() to TiFlash #25565
• Support pushing down the built-in functions unix_timestamp(), concat(), year(), day(), datediff(), datesub(), and concat_ws() to TiFlash #25564
• Optimize the aggregate operator's cost factor #25241
• Support pushing down the Limit operator to TiFlash #25159
• Support pushing down the built-in function str_to_date to TiFlash #25148
• Allow the MPP outer join to choose the build table based on the table row count #25142
• Support pushing down the built-in functions left(), right(), and abs() to TiFlash #25133
• Support pushing down the Broadcast Cartesian join to TiFlash #25106
• Support pushing down the Union All operator to TiFlash #25051
• Support balancing the MPP query workload among different TiFlash nodes based on Regions #24724
• Support invalidating stale Regions in the cache after the MPP query is executed #24432
• Improve the MySQL compatibility of the built-in function str_to_date for the format specifiers %b/%M/%r/%T #25767

Bug Fixes

• Fix the issue that an incorrect result is returned when using merge join on the SET type column #25669
• Fix the data corruption issue in the IN expression's arguments #25591
• Avoid the sessions of GC being affected by global variables #24976
• Fix the panic issue that occurs when using limit in the window function queries #25344
• Fix the wrong value returned when querying a partitioned table using Limit #24636
• Fix the issue that IFNULL does not correctly take effect on the ENUM or SET type column #24944
• Fix the wrong results caused by changing the count in the join subqueries to first_row #24865
• Fix the query hang issue that occurs when ParallelApply is used under the TopN operator #24930
• Fix the issue that more results than expected are returned when executing SQL statements using multi-column prefix indexes #24356
• Fix the issue that the <=> operator cannot correctly take effect #24477
• Fix the data race issue of the parallel Apply operator #23280
• Fix the issue that the index out of range error is reported when sorting the IndexMerge results of the PartitionUnion operator #23919
• Fix the issue that setting the tidb_snapshot variable to an unexpectedly large value might damage the transaction isolation #25680
• Fix the issue that the ODBC-styled constant (for example, {d '2020-01-01'}) cannot be used as the expression #25531
• Fix the issue that SELECT DISTINCT converted to Batch Get causes incorrect results #25320
• Fix the issue that backing off queries from TiFlash to TiKV cannot be triggered #23665 #24421
• Fix the index-out-of-range error that occurs when checking only_full_group_by #23839)
• Fix the issue that queries with TABLESAMPLE on an empty table returns unexpected rows #25257
• Fix the issue that the result of index join in correlated subqueries is wrong #25799

tidb-server v5.1.0

Improvements

• Support the built-in function VITESS_HASH() #23915
• Support pushing down data of the enumerated type to TiKV to improve performance when using enumerated types in WHERE clauses #23619
• Optimize the calculation of Window Function to solve TiDB OOM problems when paging data with ROW_NUMBER() #23807
• Optimize the calculation of UNION ALL to solve the TiDB OOM problems when using UNION ALL to join a large number of SELECT statements #21441
• Optimize the dynamic mode of partitioned tables to improve performance and stability #24150
• Fix the Region is Unavailable issue that occurs in multiple scenarios project#62
• Fix multiple Region is Unavailable issues that might occur in frequent scheduling situations
• Fix Region is Unavailable issue that might occur in some high stress write situations
• Avoid frequently reading the mysql.stats_histograms table if the cached statistics is up-to-date to avoid high CPU usage #24317

Bug Fixes

• Fix the issue that the execution result of project elimination might be wrong when the projection result is empty #23887
• Fix the issue of wrong query results when a column contains NULL values in some cases #23891
• Forbid generating MPP plans when the scan contains virtual columns #23886
• Fix the wrong reuse of PointGet and TableDual in Plan Cache #23187 #23144 #23304 #23290
• Fix the error that occurs when the optimizer builds the IndexMerge plan for clustered indexes #23906
• Fix the type inference of the BIT-type errors #23832
• Fix the issue that some optimizer hints do not take effect when the PointGet operator exists #23570
• Fix the issue that DDL operations might fail when rolling back due to an error #23893
• Fix the issue that the index range of the binary literal constant is incorrectly built #23672
• Fix the potential wrong results of the IN clause in some cases #23889
• Fix the wrong results of some string functions #23759
• Users now need both INSERT and DELETE privileges on a table to perform REPLACE operations #23909
• Users now need both INSERT and DELETE privileges on a table to perform REPLACE operations #24070
• Fix the wrong TableDual plans caused by incorrectly comparing binaries and bytes#23846
• Fix the panic issue caused by using the prefix index and index join in some cases #24547 #24716 #24717
• Fix the issue that the prepared plan cache of point get is incorrectly used by the point get statement in the transaction #24741
• Fix the issue of writing the wrong prefix index value when the collation is ascii_bin or latin1_bin #24569
• Fix the issue that the ongoing transaction might be interrupted by the GC worker #24591
• Fix a bug that the point query might get wrong on the clustered index when new-collation is enabled but new-row-format is disabled #24541
• Refactor the conversion of partition keys for shuffle hash join #24490
• Fix the panic issue that occurs when building the plan for queries that contain the HAVING clause #24045
• Fix the issue that the column pruning improvement causes the Apply and Join operators' results to go wrong #23887
• Fix a bug that the primary lock fallen back from async commit cannot be resolved #24384
• Fix a GC issue of statistics that might cause duplicated fm-sketch records #24357
• Avoid unnecessary pessimistic rollback when the pessimistic locking receives the ErrKeyExists error #23799
• Fix the issue that numeric literals cannot be recognized when the sql_mode contains ANSI_QUOTES #24429
• Forbid statements such as INSERT INTO table PARTITION (<partitions>) ... ON DUPLICATE KEY UPDATE to read data from non-listed partitions #24746
• Fix the potential index out of range error when a SQL statement contains both GROUP BY and UNION #24281
• Fix the issue that the CONCAT function incorrectly handles the collation #24296
• Fix the issue that the collation_server global variable does not take effect in new sessions #24156