Pre-release

@jackysp jackysp released this Jan 18, 2019 · 2 commits to master since this release

Assets 2

New Features

  • Support View
  • Support Window Function
  • Support Range Partition
  • Support Hash Partition

SQL Optimizer

  • Re-support the optimization rule of AggregationElimination #7676
  • Optimize the NOT EXISTS subquery and convert it to Anti Semi Join #7842
  • Add the tidb_enable_cascades_planner variable to support the new Cascades optimizer. Currently, the Cascades optimizer is not yet fully implemented and is turned off by default #7879
  • Support using Index Join in transactions #7877
  • Optimize the constant propagation on the Outer Join, so that the filtering conditions related to the Outer table in the Join result can be pushed down through the Outer Join to the Outer table, reducing the useless calculation of the Outer Join and improving the execution performance #7794
  • Adjust the optimization rule of Projection Elimination to the position after the Aggregation Elimination, to avoid redundant Project operators #7909
  • Optimize the IFNULL function and eliminate this function when the input parameter has a non-NULL attribute #7924
  • Support building range for _tidb_rowid, to avoid full table scan and reduce cluster stress #8047
  • Optimize the IN subquery to do the Inner Join after the aggregation, and add the tidb_opt_insubq_to_join_and_agg variable to control whether to enable this optimization rule and open it by default #7531
  • Support using subqueries in the DO statement #8343
  • Add the optimization rule of Outer Join elimination to reduce unnecessary table scan and Join operations and improve execution performance #8021
  • Modify the Hint behavior of the TIDB_INLJ optimizer, and the optimizer will use the table specified in Hint as the Inner table of Index Join #8243
  • Use PointGet in a wide range so that it can be used when the execution plan cache of the Prepare statement takes effect #8108
  • Introduce the greedy Join Reorder algorithm to optimize the join order selection when joining multiple tables #8394
  • Support View #8757
  • Support Window Function #8630
  • Return warning to the client when TIDB_INLJ is not in effect, to enhance usability #9037
  • Support deducing the statistics for filtered data based on filtering conditions and table statistics #7921
  • Improve the Partition Pruning optimization rule of Range Partition #8885

SQL Executor

  • Optimize the Merge Join operator to support the empty ON condition #9037
  • Optimize the log and print the user variables used when executing the EXECUTE statement #7684
  • Optimize the log to print slow query information for the COMMIT statement #7951
  • Support the EXPLAIN ANALYZE feature to make the SQL tuning process easier #7827
  • Optimize the write performance of wide tables with many columns #7935
  • Support admin show next_row_id #8242
  • Add the tidb_init_chunk_size variable to control the size of the initial Chunk used by the execution engine #8480
  • Improve shard_row_id_bits and cross-check the auto-increment ID #8936

Prepare Statement

  • Prohibit adding the Prepare statement containing subqueries to the query plan cache to guarantee the query plan is correct when different user variables are input #8064
  • Optimize the query plan cache to guarantee the plan can be cached when the statement contains non-deterministic functions #8105
  • Optimize the query plan cache to guarantee the query plan of DELETE/UPDATE/INSERT can be cached #8107
  • Optimize the query plan cache to remove the corresponding plan when executing the DEALLOCATE statement #8332
  • Optimize the query plan cache to avoid the TiDB OOM issue caused by caching too many plans by limiting the memory usage #8339
  • Optimize the Prepare statement to support using the ? placeholder in the ORDER BY/GROUP BY/LIMIT clause #8206

Privilege Management

  • Add the privilege check for the ANALYZE statement #8486
  • Add the privilege check for the USE statement #8414
  • Add the privilege check for the SET GLOBAL statement #8837
  • Add the privilege check for the SHOW PROCESSLIST statement #7858

Server

  • Support the Trace feature #9029
  • Support the plugin framework #8788
  • Support using unix_socket and TCP simultaneously to connect to the database #8836
  • Support the interactive_timeout system variable #8573
  • Support the wait_timeout system variable #8346
  • Support splitting a transaction into multiple transactions based on the number of statements using the tidb_batch_commit variable #8293
  • Support using the ADMIN SHOW SLOW statement to check slow logs #7785

Compatibility

  • Support the ALLOW_INVALID_DATES SQL mode #9027
  • Improve LoadData fault-tolerance for the CSV file #9005
  • Support the MySQL 320 handshake protocol #8812
  • Support using the unsigned bigint column as the auto-increment column #8181
  • Support the SHOW CREATE DATABASE IF NOT EXISTS syntax #8926
  • Abandon the predicate pushdown operation when the filtering condition contains a user variable to improve the compatibility with MySQL’s behavior of using user variables to mock the Window Function behavior #8412

DDL

  • Support fast recovery of mistakenly deleted tables #7937
  • Support adjusting the number of concurrencies of ADD INDEX dynamically #8295
  • Support changing the character set of tables or columns to utf8/utf8mb4 #8037
  • Change the default character set from utf8 to utf8mb4 #7965
  • Support Range Partition #8011
Jan 4, 2019
rebase release-2.0 to release-2.0-binlog (#8928)

@winkyao winkyao released this Jan 3, 2019 · 10 commits to release-2.0 since this release

Assets 2
  • Fix the issue that the error is not handled properly when PD is in an abnormal condition #8764 #8923
  • Fix the issue that the Rename operation on a table in TiDB is not compatible with that in MySQL #8809
  • Fix the issue that the error message is wrongly reported when the ADMIN CHECK TABLE operation is performed in the process of executing the ADD INDEX statement #8750
  • Fix the issue that the prefix index range is incorrect in some cases #8877
  • Fix the panic issue of the UPDATE statement when columns are added in some cases #8904
Dec 26, 2018
binlog: use pumps client to write binlog (#7659) (#8078) (#8822)

@winkyao winkyao released this Dec 21, 2018 · 38 commits to release-2.1 since this release

Assets 2
  • Make TiDB compatible with TiDB-Binlog of the Kafka version #8747
  • Improve the exit mechanism of TiDB in a rolling update #8707
  • Fix the panic issue caused by adding the index for the generated column in some cases #8676
  • Fix the issue that the optimizer cannot find the optimal query plan when TIDB_SMJ Hint exists in the SQL statement in some cases #8729
  • Fix the issue that AntiSemiJoin returns an incorrect result in some cases #8730
  • Improve the valid character check of the utf8 character set #8754
  • Fix the issue that the field of the time type might return an incorrect result when the write operation is performed before the read operation in a transaction #8746

@winkyao winkyao released this Dec 18, 2018 · 22 commits to release-2.0 since this release

Assets 2
  • Fix the possible issue caused by canceling a DDL job #8513
  • Fix the issue that the ORDER BY and UNION clauses cannot quote the column including a table name #8514
  • Fix the issue that the UNCOMPRESS function does not judge the incorrect input length #8607
  • Fix the issue encountered by ANSI_QUOTES SQL_MODE when upgrading TiDB #8575
  • Fix the issue that SELECT returns the wrong result in some cases #8570
  • Fix the possible issue that TiDB cannot exit when it receives the exit signal #8501
  • Fix the issue that IndexLookUpJoin returns the wrong result in some cases #8508
  • Avoid pushing down the filter containing GetVar or SetVar #8454
  • Fix the issue that the result length of the UNION clauses is incorrect in some cases #8491
  • Fix the issue of PREPARE FROM @var_name #8488
  • Fix the panic issue when dumping statistics information in some cases #8464
  • Fix the statistics estimation issue of point queries in some cases #8493
  • Fix the panic issue when the returned default enum value is a string #8476
  • Fix the issue that too much memory is consumed in the scenario of wide tables #8467
  • Fix the issue encountered when Parser incorrectly formats the mod opcode #8431
  • Fix the panic issue caused by adding foreign key constraints in some cases #8421 #8410
  • Fix the issue that the YEAR column type incorrectly converts the zero value #8396
  • Fix the panic issue occurred when the argument of the VALUES function is not a column #8404
  • Disable Plan Cache for statements containing subqueries #8395

@jackysp jackysp released this Dec 12, 2018 · 50 commits to release-2.1 since this release

Assets 2

SQL Optimizer/Executor

  • Fix the round error of the negative date #8574
  • Fix the issue that the uncompress function does not check the data length #8606
  • Reset bind arguments of the prepare statement after the execute command is executed #8652
  • Support automatically collecting the statistics information of a partition table #8649
  • Fix the wrongly configured integer type when pushing down the abs function #8628
  • Fix the data race on the JSON column #8660

Server

  • Fix the issue that the transaction obtained TSO is incorrect when PD breaks down #8567
  • Fix the bootstrap failure caused by the statement that does not conform to ANSI standards #8576
  • Fix the issue that incorrect parameters are used in transaction retries #8638

DDL

  • Change the default character set and collation of tables into utf8mb4 #8590
  • Add the ddl_reorg_batch_size variable to control the speed of adding indexes #8614
  • Make the character set and collation options content in DDL case-insensitive #8611
  • Fix the issue of adding indexes for generated columns #8655

@winkyao winkyao released this Nov 30, 2018 · 64 commits to release-2.1 since this release

Assets 2

Upgrade caveat

  • TiDB 2.1 does not support downgrading to v2.0.x or earlier due to the adoption of the new storage engine
  • Parallel DDL is enabled in TiDB 2.1, so the clusters with TiDB version earlier than 2.0.1 cannot upgrade to 2.1 using rolling update. You can choose either of the following two options:
    • Stop the cluster and upgrade to 2.1 directly
    • Roll update to 2.0.1 or later 2.0.x versions, and then roll update to the 2.1 version
  • If you upgrade from TiDB 2.0.6 or earlier to TiDB 2.1, make sure if there is any ongoing DDL operation, especially the time consuming Add Index operation, because the DDL operations slow down the upgrading process.

SQL Optimizer

  • Optimize the selection range of Index Join to improve the execution performance
  • Optimize the selection of outer table for Index Join and use the table with smaller estimated value of Row Count as the outer table
  • Optimize Join Hint TIDB_SMJ so that Merge Join can be used even without proper index available
  • Optimize Join Hint TIDB_INLJ to specify the Inner table to Join
  • Optimize correlated subquery, push down Filter, and extend the index selection range, to improve the efficiency of some queries by orders of magnitude
  • Support using Index Hint and Join Hint in the UPDATE and DELETE statement
  • Support pushing down more functions:ABS / CEIL / FLOOR / IS TRUE / IS FALSE
  • Optimize the constant folding algorithm for the IF and IFNULL built-in functions
  • Optimize the output of the EXPLAIN statement and use hierarchy structure to show the relationship between operators

SQL executor

  • Refactor all the aggregation functions and improve execution efficiency of the Stream and Hash aggregation operators
  • Implement the parallel Hash Aggregate operators and improve the computing performance by 350% in some scenarios
  • Implement the parallel Project operators and improve the performance by 74% in some scenarios
  • Read the data of the Inner table and Outer table of Hash Join concurrently to improve the execution performance
  • Optimize the execution speed of the REPLACE INTO statement and increase the performance nearly by 10 times
  • Optimize the memory usage of the time data type and decrease the memory usage of the time data type by fifty percent
  • Optimize the point select performance and improve the point select efficiency result of Sysbench by 60%
  • Improve the performance of TiDB on inserting or updating wide tables by 20 times
  • Support configuring the memory upper limit of a single statement in the configuration file
  • Optimize the execution of Hash Join, if the Join type is Inner Join or Semi Join and the inner table is empty, return the result without reading data from the outer table
  • Support using the EXPLAIN ANALYZE statement to check the runtime statistics including the execution time and the number of returned rows of each operator

Statistics

  • Support enabling auto ANALYZE statistics only during certain period of the day
  • Support updating the table statistics automatically according to the feedback of the queries
  • Support configuring the number of buckets in the histogram using the ANALYZE TABLE WITH BUCKETS statement
  • Optimize the Row Count estimation algorithm using histogram for mixed queries of equality query and range queries

Expressions

  • Support following built-in function:
    • json_contains
    • json_contains_path
    • encode/decode

Server

DDL

  • Support the parallel execution of the add index statement and other statements to avoid the time consuming add index operation blocking other operations
  • Optimize the execution speed of ADD INDEX and improve it greatly in some scenarios
  • Support the select tidb_is_ddl_owner() statement to facilitate deciding whether TiDB is DDL Owner
  • Support the ALTER TABLE FORCE syntax
  • Support the ALTER TABLE RENAME KEY TO syntax
  • Add the table name and database name in the output information of admin show ddl jobs
  • Support using the ddl/owner/resign HTTP interface to release the DDL owner and start electing a new DDL owner

Compatibility

  • Support more MySQL syntaxes
  • Make the BIT aggregate function support the ALL parameter
  • Support the SHOW PRIVILEGES statement
  • Support the CHARACTER SET syntax in the LOAD DATA statement
  • Support the IDENTIFIED WITH syntax in the CREATE USER statement
  • Support the LOAD DATA IGNORE LINES statement
  • The Show ProcessList statement returns more accurate information

@winkyao winkyao released this Nov 20, 2018 · 55 commits to release-2.0 since this release

Assets 2
  • Fix the issue caused by the empty statistics histogram #7927
  • Fix the panic issue of the UNION ALL statement in some cases #7942
  • Fix the stack overflow issue caused by wrong DDL Jobs #7959
  • Add the slow log for the Commit operation #7983
  • Fix the panic issue caused by the too large Limit value #8004
  • Support specifying the utf8mb4 character set in the USING clause #8048
  • Make the TRUNCATE built-in function support parameters of unsigned integer type #8069
  • Fix the selectivity estimation issue of the primary key for the statistics module in some cases #8150
  • Add the Session variable to control whether _tidb_rowid is allowed to be written in #8126
  • Fix the panic issue of PhysicalProjection in some cases #8154
  • Fix the unstable results of the Union statement in some cases #8168
  • Fix the issue that NULL is not returned by values in the non-Insert statement #8179
  • Fix the issue that the statistics module cannot clear the outdated data in some cases #8184
  • Make the maximum allowed running time for a transaction a configurable option 8209
  • Fix the wrong comparison algorithm of expression rewriter in some cases #8288
  • Eliminate the extra columns generated by the UNION ORDER BY statement #8307
  • Support the admin show next_row_id statement #8274
  • Fix the escape issue of special characters in the Show Create Table statement #8321
  • Fix the unexpected errors in the UNION statement in some cases #8318
  • Fix the issue that canceling a DDL job causes no rollback of a schema in some cases #8312
  • Change tidb_max_chunk_size to a global variable #8333
  • Add an upper bound to the Scan command of ticlient, to avoid overbound scan #8309, #8310

@winkyao winkyao released this Nov 12, 2018 · 115 commits to release-2.1 since this release

Assets 2

SQL Optimizer

  • Fix the issue that IndexReader reads the wrong handle in some cases #8132
  • Fix the issue occurred while the IndexScan Prepared statement uses Plan Cache #8055
  • Fix the issue that the result of the Union statement is unstable #8165

SQL Execution Engine

  • Improve the performance of TiDB on inserting or updating wide tables #8024
  • Support the unsigned int flag in the Truncate built-in function #8068
  • Fix the error occurred while converting JSON data to the decimal type #8109
  • Fix the error occurred when you Update the float type #8170

Statistics

  • Fix the incorrect statistics issue during point queries in some cases #8035
  • Fix the selectivity estimation of statistics for primary key in some cases #8149
  • Fix the issue that the statistics of deleted tables are not cleared up for a long period of time #8182

Server

  • Improve the readability of logs and make logs better
  • Fix the error occurred when obtaining the table data of infoschema.profiling #8096
  • Replace the unix socket with the pumps client to write binlogs #8098
  • Add the threshold value for the tidb_slow_log_threshold environment variable, which dynamically sets the slow log #8094
  • Add the original length of a SQL statement truncated while the tidb_query_log_max_len environment variable dynamically sets logs 8200
  • Add the tidb_opt_write_row_id environment variable to control whether to allow writing _tidb_rowid #8218
  • Add an upper bound to the Scan command of ticlient, to avoid overbound scan #8081, #8247

DDL

  • Fix the issue that executing DDL statements in transactions encounters an error in some cases #8056
  • Fix the issue that executing truncate table in partition tables does not take effect #8103
  • Fix the issue that the DDL operation does not roll back correctly after being cancelled in some cases #8057
  • Add the admin show next_row_id command to return the next available row ID #8268