Skip to content

Commit

Permalink
Merge Initial TiDB Support (PR #724)
Browse files Browse the repository at this point in the history
  • Loading branch information
nene committed Mar 16, 2024
2 parents e2bb28b + cd2db33 commit 585bb8a
Show file tree
Hide file tree
Showing 14 changed files with 905 additions and 3 deletions.
1 change: 1 addition & 0 deletions AUTHORS
Expand Up @@ -13,6 +13,7 @@ Boris Verkhovskiy <boris.verk@gmail.com>
Christian Jorgensen <chr.jorgensen1@gmail.com>
Christopher Manouvrier <chris@dovetailapp.com>
Damon Davison <ddavison@avalere.com>
Daniël van Eeden <daniel.van.eeden@pingcap.com>
Davut Can Abacigil <can@teamsql.io>
eeeXun <sdes96303@gmail.com>
Erik Hirmo <erik.hirmo@roguewave.com>
Expand Down
6 changes: 3 additions & 3 deletions README.md
Expand Up @@ -7,7 +7,7 @@
It started as a port of a [PHP Library][], but has since considerably diverged.

It supports various SQL dialects:
GCP BigQuery, IBM DB2, Apache Hive, MariaDB, MySQL, Couchbase N1QL, Oracle PL/SQL, PostgreSQL, Amazon Redshift, SingleStoreDB, Snowflake, Spark, SQL Server Transact-SQL, Trino (and Presto).
GCP BigQuery, IBM DB2, Apache Hive, MariaDB, MySQL, TiDB, Couchbase N1QL, Oracle PL/SQL, PostgreSQL, Amazon Redshift, SingleStoreDB, Snowflake, Spark, SQL Server Transact-SQL, Trino (and Presto).
See [language option docs](docs/language.md) for more details.

It does not support:
Expand Down Expand Up @@ -123,7 +123,7 @@ sql-formatter -h

```
usage: sql-formatter [-h] [-o OUTPUT] \
[-l {bigquery,db2,db2i,hive,mariadb,mysql,n1ql,plsql,postgresql,redshift,singlestoredb,snowflake,spark,sql,sqlite,transactsql,trino,tsql}] [-c CONFIG] [--version] [FILE]
[-l {bigquery,db2,db2i,hive,mariadb,mysql,n1ql,plsql,postgresql,redshift,singlestoredb,snowflake,spark,sql,sqlite,tidb,transactsql,trino,tsql}] [-c CONFIG] [--version] [FILE]
SQL Formatter
Expand All @@ -135,7 +135,7 @@ optional arguments:
-o, --output OUTPUT
File to write SQL output (defaults to stdout)
--fix Update the file in-place
-l, --language {bigquery,db2,db2i,hive,mariadb,mysql,n1ql,plsql,postgresql,redshift,singlestoredb,snowflake,spark,sql,sqlite,trino,tsql}
-l, --language {bigquery,db2,db2i,hive,mariadb,mysql,n1ql,plsql,postgresql,redshift,singlestoredb,snowflake,spark,sql,sqlite,tidb,trino,tsql}
SQL dialect (defaults to basic sql)
-c, --config CONFIG
Path to config JSON file or json string (will use default configs if unspecified)
Expand Down
2 changes: 2 additions & 0 deletions docs/dialect.md
Expand Up @@ -27,6 +27,7 @@ The following dialects can be imported from `"sql-formatter"` module:
- `hive` - [Apache Hive][]
- `mariadb` - [MariaDB][]
- `mysql` - [MySQL][]
- `tidb` - [TiDB][]
- `n1ql` - [Couchbase N1QL][]
- `plsql` - [Oracle PL/SQL][]
- `postgresql` - [PostgreSQL][]
Expand Down Expand Up @@ -75,6 +76,7 @@ You likely only want to use this if your other alternative is to fork SQL Format
[apache hive]: https://hive.apache.org/
[mariadb]: https://mariadb.com/
[mysql]: https://www.mysql.com/
[tidb]: https://github.com/pingcap/tidb/
[couchbase n1ql]: http://www.couchbase.com/n1ql
[oracle pl/sql]: http://www.oracle.com/technetwork/database/features/plsql/index.html
[postgresql]: https://www.postgresql.org/
Expand Down
2 changes: 2 additions & 0 deletions docs/language.md
Expand Up @@ -19,6 +19,7 @@ const result = format('SELECT * FROM tbl', { dialect: 'sqlite' });
- `"hive"` - [Apache Hive][]
- `"mariadb"` - [MariaDB][]
- `"mysql"` - [MySQL][]
- `"tidb"` - [TiDB][]
- `"n1ql"` - [Couchbase N1QL][]
- `"plsql"` - [Oracle PL/SQL][]
- `"postgresql"` - [PostgreSQL][]
Expand Down Expand Up @@ -52,6 +53,7 @@ See docs for [dialect][] option.
[apache hive]: https://hive.apache.org/
[mariadb]: https://mariadb.com/
[mysql]: https://www.mysql.com/
[tidb]: https://github.com/pingcap/tidb/
[couchbase n1ql]: http://www.couchbase.com/n1ql
[oracle pl/sql]: http://www.oracle.com/technetwork/database/features/plsql/index.html
[postgresql]: https://www.postgresql.org/
Expand Down
1 change: 1 addition & 0 deletions docs/params.md
Expand Up @@ -121,6 +121,7 @@ The placeholder types available by default depend on SQL dialect used:
- snowflake - _no support_
- sqlite - `?`, `?1`, `:name`, `@name`, `$name`
- spark - _no support_
- tidb - `?`
- tsql - `@name`, `@"name"`, `@[name]`
- trino - _no support_

Expand Down
1 change: 1 addition & 0 deletions package.json
Expand Up @@ -35,6 +35,7 @@
"db2",
"db2i",
"sqlite",
"tidb",
"trino",
"presto",
"prestosql",
Expand Down
1 change: 1 addition & 0 deletions src/allDialects.ts
Expand Up @@ -4,6 +4,7 @@ export { db2i } from './languages/db2i/db2i.formatter.js';
export { hive } from './languages/hive/hive.formatter.js';
export { mariadb } from './languages/mariadb/mariadb.formatter.js';
export { mysql } from './languages/mysql/mysql.formatter.js';
export { tidb } from './languages/tidb/tidb.formatter.js';
export { n1ql } from './languages/n1ql/n1ql.formatter.js';
export { plsql } from './languages/plsql/plsql.formatter.js';
export { postgresql } from './languages/postgresql/postgresql.formatter.js';
Expand Down
1 change: 1 addition & 0 deletions src/index.ts
Expand Up @@ -9,6 +9,7 @@ export { db2i } from './languages/db2i/db2i.formatter.js';
export { hive } from './languages/hive/hive.formatter.js';
export { mariadb } from './languages/mariadb/mariadb.formatter.js';
export { mysql } from './languages/mysql/mysql.formatter.js';
export { tidb } from './languages/tidb/tidb.formatter.js';
export { n1ql } from './languages/n1ql/n1ql.formatter.js';
export { plsql } from './languages/plsql/plsql.formatter.js';
export { postgresql } from './languages/postgresql/postgresql.formatter.js';
Expand Down
221 changes: 221 additions & 0 deletions src/languages/tidb/tidb.formatter.ts
@@ -0,0 +1,221 @@
import { DialectOptions } from '../../dialect.js';
import { expandPhrases } from '../../expandPhrases.js';
import { postProcess } from '../mariadb/likeMariaDb.js';
import { dataTypes, keywords } from './tidb.keywords.js';
import { functions } from './tidb.functions.js';

const reservedSelect = expandPhrases(['SELECT [ALL | DISTINCT | DISTINCTROW]']);

const reservedClauses = expandPhrases([
// queries
'WITH [RECURSIVE]',
'FROM',
'WHERE',
'GROUP BY',
'HAVING',
'WINDOW',
'PARTITION BY',
'ORDER BY',
'LIMIT',
'OFFSET',
// Data manipulation
// - insert:
'INSERT [LOW_PRIORITY | DELAYED | HIGH_PRIORITY] [IGNORE] [INTO]',
'REPLACE [LOW_PRIORITY | DELAYED] [INTO]',
'VALUES',
'ON DUPLICATE KEY UPDATE',
// - update:
'SET',
]);

const standardOnelineClauses = expandPhrases(['CREATE [TEMPORARY] TABLE [IF NOT EXISTS]']);

const tabularOnelineClauses = expandPhrases([
// https://docs.pingcap.com/tidb/stable/sql-statement-create-view
'CREATE [OR REPLACE] [SQL SECURITY DEFINER | SQL SECURITY INVOKER] VIEW [IF NOT EXISTS]',
// https://docs.pingcap.com/tidb/stable/sql-statement-update
'UPDATE [LOW_PRIORITY] [IGNORE]',
// https://docs.pingcap.com/tidb/stable/sql-statement-delete
'DELETE [LOW_PRIORITY] [QUICK] [IGNORE] FROM',
// https://docs.pingcap.com/tidb/stable/sql-statement-drop-table
'DROP [TEMPORARY] TABLE [IF EXISTS]',
// https://docs.pingcap.com/tidb/stable/sql-statement-alter-table
'ALTER TABLE',
'ADD [COLUMN]',
'{CHANGE | MODIFY} [COLUMN]',
'DROP [COLUMN]',
'RENAME [TO | AS]',
'RENAME COLUMN',
'ALTER [COLUMN]',
'{SET | DROP} DEFAULT', // for alter column
// https://docs.pingcap.com/tidb/stable/sql-statement-truncate
'TRUNCATE [TABLE]',
// https://docs.pingcap.com/tidb/stable/sql-statement-alter-database
'ALTER DATABASE',
// https://docs.pingcap.com/tidb/stable/sql-statement-alter-instance
'ALTER INSTANCE',
'ALTER RESOURCE GROUP',
'ALTER SEQUENCE',
// https://docs.pingcap.com/tidb/stable/sql-statement-alter-user
'ALTER USER',
'ALTER VIEW',
'ANALYZE TABLE',
'CHECK TABLE',
'CHECKSUM TABLE',
'COMMIT',
'CREATE DATABASE',
'CREATE INDEX',
'CREATE RESOURCE GROUP',
'CREATE ROLE',
'CREATE SEQUENCE',
'CREATE USER',
'DEALLOCATE PREPARE',
'DESCRIBE',
'DROP DATABASE',
'DROP INDEX',
'DROP RESOURCE GROUP',
'DROP ROLE',
'DROP TABLESPACE',
'DROP USER',
'DROP VIEW',
'EXPLAIN',
'FLUSH',
// https://docs.pingcap.com/tidb/stable/sql-statement-grant-privileges
'GRANT',
'IMPORT TABLE',
'INSTALL COMPONENT',
'INSTALL PLUGIN',
'KILL',
'LOAD DATA',
'LOCK INSTANCE FOR BACKUP',
'LOCK TABLES',
'OPTIMIZE TABLE',
'PREPARE',
'RELEASE SAVEPOINT',
'RENAME TABLE',
'RENAME USER',
'REPAIR TABLE',
'RESET',
'REVOKE',
'ROLLBACK',
'ROLLBACK TO SAVEPOINT',
'SAVEPOINT',
'SET CHARACTER SET',
'SET DEFAULT ROLE',
'SET NAMES',
'SET PASSWORD',
'SET RESOURCE GROUP',
'SET ROLE',
'SET TRANSACTION',
'SHOW',
'SHOW BINARY LOGS',
'SHOW BINLOG EVENTS',
'SHOW CHARACTER SET',
'SHOW COLLATION',
'SHOW COLUMNS',
'SHOW CREATE DATABASE',
'SHOW CREATE TABLE',
'SHOW CREATE USER',
'SHOW CREATE VIEW',
'SHOW DATABASES',
'SHOW ENGINE',
'SHOW ENGINES',
'SHOW ERRORS',
'SHOW EVENTS',
'SHOW GRANTS',
'SHOW INDEX',
'SHOW MASTER STATUS',
'SHOW OPEN TABLES',
'SHOW PLUGINS',
'SHOW PRIVILEGES',
'SHOW PROCESSLIST',
'SHOW PROFILE',
'SHOW PROFILES',
'SHOW STATUS',
'SHOW TABLE STATUS',
'SHOW TABLES',
'SHOW TRIGGERS',
'SHOW VARIABLES',
'SHOW WARNINGS',
// https://docs.pingcap.com/tidb/stable/sql-statement-table
'TABLE',
'UNINSTALL COMPONENT',
'UNINSTALL PLUGIN',
'UNLOCK INSTANCE',
'UNLOCK TABLES',
// https://docs.pingcap.com/tidb/stable/sql-statement-use
'USE',
]);

const reservedSetOperations = expandPhrases(['UNION [ALL | DISTINCT]']);

const reservedJoins = expandPhrases([
'JOIN',
'{LEFT | RIGHT} [OUTER] JOIN',
'{INNER | CROSS} JOIN',
'NATURAL [INNER] JOIN',
'NATURAL {LEFT | RIGHT} [OUTER] JOIN',
// non-standard joins
'STRAIGHT_JOIN',
]);

const reservedPhrases = expandPhrases([
'ON {UPDATE | DELETE} [SET NULL]',
'CHARACTER SET',
'{ROWS | RANGE} BETWEEN',
'IDENTIFIED BY',
]);

// https://docs.pingcap.com/tidb/stable/basic-features
export const tidb: DialectOptions = {
name: 'tidb',
tokenizerOptions: {
reservedSelect,
reservedClauses: [...reservedClauses, ...standardOnelineClauses, ...tabularOnelineClauses],
reservedSetOperations,
reservedJoins,
reservedPhrases,
supportsXor: true,
reservedKeywords: keywords,
reservedDataTypes: dataTypes,
reservedFunctionNames: functions,
// TODO: support _ char set prefixes such as _utf8, _latin1, _binary, _utf8mb4, etc.
stringTypes: [
'""-qq-bs',
{ quote: "''-qq-bs", prefixes: ['N'] },
{ quote: "''-raw", prefixes: ['B', 'X'], requirePrefix: true },
],
identTypes: ['``'],
identChars: { first: '$', rest: '$', allowFirstCharNumber: true },
variableTypes: [
{ regex: '@@?[A-Za-z0-9_.$]+' },
{ quote: '""-qq-bs', prefixes: ['@'], requirePrefix: true },
{ quote: "''-qq-bs", prefixes: ['@'], requirePrefix: true },
{ quote: '``', prefixes: ['@'], requirePrefix: true },
],
paramTypes: { positional: true },
lineCommentTypes: ['--', '#'],
operators: [
'%',
':=',
'&',
'|',
'^',
'~',
'<<',
'>>',
'<=>',
'->',
'->>',
'&&',
'||',
'!',
'*.*', // Not actually an operator
],
postProcess,
},
formatOptions: {
onelineClauses: [...standardOnelineClauses, ...tabularOnelineClauses],
tabularOnelineClauses,
},
};

0 comments on commit 585bb8a

Please sign in to comment.