[sqlparser] better sql parser?

[BohuTANG]

Summary

sqlparser-rs is not easy to extend and we are looking for a good alternative(if have), comments are welcome

Reference:
https://github.com/datafuselabs/datafuse/blob/36dce2064742a1d64bfcf9a7cd308d0f5a3e4eb5/fusequery/query/src/sql/sql_parser.rs

[BohuTANG]

nom-sql
but no update for a long time

[PsiACE]

It seems that sqlparser lacks good alternatives. If necessary, we may need to maintain one by ourselves.

quaint is not a parser, but

• An AST for building dynamic SQL queries.
• Visitors for different databases to generate SQL strings.

toydb-parser is the parser for toydb

[sundy-li]

How about parsers from yacc rules?

[dantengsky]

FYI :

lalrpop: an lr(1) parser generator (lexer could be customized)

repo: https://github.com/lalrpop/lalrpop
doc: http://lalrpop.github.io/lalrpop/

• never exercise it before
• seems it lacks precedence/association rules declaration (again I am not sure.)

After googling around, I found a SQL parser using it:

https://github.com/hmwill/vervolg/blob/main/src/sql.lalrpop

[leiysky]

nom-sql
but no update for a long time

nom is a nice parser combinator, with which we can extend SQL syntax easily.

I think we can fork nom-sql or make one from scratch(not a big deal).

Here's a nom Cypher parser I wrote at a Hackathon, which only cost me about few hours 🤣 . FYI.

[BohuTANG]

How about parsers from yacc rules?

https://github.com/lalrpop/lalrpop is rust yacc

[leiysky]

Based on my experience, working with bison(yacc) is suffering.

Here's the MySQL's yacc file, which has about 17,000 LOC.

https://github.com/mysql/mysql-server/blob/8.0/sql/sql_yacc.yy

Imagine that coding in a extremely huge plain text file without lint tool(no jump, no high light, no auto format), what a pain.

[PsiACE]

Based on my experience, working with bison(yacc) is suffering.

You are right, I also prefer a handwritten parser

[tlightsky]

Based on my experience, working with bison(yacc) is suffering.

Here's the MySQL's yacc file, which has about 17,000 LOC.

https://github.com/mysql/mysql-server/blob/8.0/sql/sql_yacc.yy

Imagine that coding in a extremely huge plain text file without lint tool(no jump, no high light, no auto format), what a pain.

yes, hard to make changes to it

[abhikjain360]

If writing your own, maybe something like pest can be used as well. Tensorbase uses it (see here), and it seems pretty maintainable?

[BohuTANG]

@abhikjain360 Thanks for the information, cool

[zhang2014]

Maybe our sql parser is support multi dialect

• When users use the ClickHouse protocol to access DataFuse, we may be ClickHouse style (syntax, function name)
• When users use the MySQL protocol to access DataFuse, we should use MySQL style(syntax, function name)