sqlparser integration and code generation

[Samyak2]

Tracking issue for parser integration

Links

• GitHub repo: https://github.com/sqlparser-rs/sqlparser-rs
• Documentation: https://docs.rs/sqlparser/0.18.0/sqlparser/index.html

[Samyak2]

This includes:

• Use sqlparser to generate AST from SQL code. This part is simple.
• Code generation: use the AST to generate intermediate code using instruction set #12. This will be tricky and will require many tests to ensure correctness.
  • This will be done in phases:
    1. CREATE DATABASE and CREATE SCHEMA
    2. Convert sqlparser's Value to our Value.
    3. Convert sqlparser's Expr to our Expr.
    4. CREATE TABLE: Columns with name and types only and with IF NOT EXISTS.
    5. INSERT: into table, simple values.
    6. SELECT: from a table, filter using WHERE.
    7. UPDATE: a table with simple expr.
    8. DELETE: a table with simple expr.
    9. DROP TABLE, DROP SCHEMA, DROP DATABASE
  • All other SQL features will be on top of these basic statements. They will be implemented once the execution for these basic statements is done.

[tyt2y3]

Actually, can we reuse sqlparser's Expr (we simply re-export them) ? May be we should also use sqlparser's Value internally, or at least make them inter-convertable?

Would that save us some time & effort?

[Samyak2]

Actually, can we reuse sqlparser's Expr (we simply re-export them) ? May be we should also use sqlparser's Value internally, or at least make them inter-convertable?

Would that save us some time & effort?

sqlparser::ast::Value may be good for representation of values in a query, but I don't think it's appropriate for the executor. As an example, the only number variant is stored as a string. So everytime we need to read or do operations on them, we'll need to convert it to an appropriate numeric type. While converting to our own Value, I have handled this, so that we do not have to bother about this in the executor.

As for Expr, some of its variants store a Value. To make it consistent, we should make our own Expr that stores our Value. I believe this won't add too much maintenance overhead because extending it will be simply adding a new variant and changing impl TryFrom<sqlparser::ast::Expr> to use it. It also very clearly describes all the operations that we will support.

[tyt2y3]

Okay cool. Thank you for the explanation. I am looking forward to some unit tests!