Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor parse #728

Merged
merged 13 commits into from
May 22, 2024
Merged

Refactor parse #728

merged 13 commits into from
May 22, 2024

Conversation

brennanjl
Copy link
Contributor

@brennanjl brennanjl commented May 13, 2024

This commit unifies all of the parsers into a single grammar. This is so that we can get accurate error messages in the IDE.

This commit also introduces uint256 and decimal types.

Still a WIP, as I still need to:

  • build conversion code to the old asts. I think we will only need to do this for the SQL grammar. It is probably quite a bit of work to redo the old visitors (however many of them can be removed at this point). The procedure syntax is mostly 1-1.
  • add tests
  • delete the old parse code

;

sql_expr:
literal type_cast? # literal_sql_expr
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be really clear and easier to read if the 'alternate label' are aligned on the right. I kind of have a strong feeling about it, especially when I'm reviewing it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's fair. Will do that

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool, can do do it on all rules? That'll be super helpful

parse/parse.go Outdated Show resolved Hide resolved
Copy link
Contributor

@Yaiba Yaiba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've reviewed the grammar rules till sql_expr, and corresponded parser logic, I kind go over the parser line by line since we've change the grammer so much. Will continue to review.

BTW, I like the new grammer/visitor/tree structure, a lot cleaner, and it helps a lot since we have matching structures.

It's much easier than the version before for reviewing, @jchappelow.

LEGACY_NO_ACTION: 'no_action';

IDENTIFIER:
[a-z] [a-z_0-9]*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For SQL this might be a issue, we used to support table/column name surrounded by ` and ". Unless we want to remove this support.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just re-added


constraint:
MIN LPAREN literal RPAREN # MIN
| MAX LPAREN literal RPAREN # MAX
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems nbd. You probably did this on purpose.
I don't think you mean to use keywords from Lexer(I don't think you can? because ANTLR spits out just the captialized label) for 'alternate label', but here it's bit confusing in IDE

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh whoops yeah that's really dumb of me

LBRACE action_block RBRACE
;

procedure_declaration:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just came up a edge case: If a view procedure calls another view(authn=true) procedure, it bypasses the called procedure's authn constraint, but this combination seems won't happen.
I guess the author should be aware that the caller procedure also needs to be view(authn=true)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah Truflation ran into this. Not really something we can handle in the parser. But yeah, it makes me think we need some better access control.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure how we can ensure this, since we cannot enforce the caller procedure has same annotation as called procedure(or only enforce authn annotation), it's not kwild's concern


func (s *schemaVisitor) VisitStmt_variable_declaration(ctx *gen.Stmt_variable_declarationContext) any {
stmt := &ProcedureStmtDeclaration{
Variable: varFromTerminalNode(ctx.VARIABLE()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing Type?

| RETURN NEXT procedure_expr_list SCOL # stmt_return_next
;

// unfortunately necessary to preserve order in generated code
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean 'order'?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to not have to declared a rule this basic that is only used once, but it is necessary to know the order that these are used in. Will delete the comment

parse/antlr.go Outdated
act.Public = public
act.Modifiers = mods

params := ctx.Variable_list().Accept(s).([]*ExpressionVariable)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Variable_list can be nil

paramStrs[i] = p.String()
}
act.Parameters = paramStrs

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

call ctx.Action_block().Accept(s) here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yeah we dont' need this, as we'll parse this from ParseAction

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do actually, to preserve order. Fixed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What order? will we compare the result later?

parse/antlr.go Show resolved Hide resolved
parse/antlr.go Show resolved Hide resolved
parse/grammar/KuneiformParser.g4 Outdated Show resolved Hide resolved
Copy link
Contributor

@Yaiba Yaiba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed grammar and parser, and partial analyzer

parse/antlr.go Show resolved Hide resolved
parse/analyze.go Outdated Show resolved Hide resolved
parse/analyze.go Show resolved Hide resolved
parse/analyze.go Outdated Show resolved Hide resolved
(ELSE else_clause=sql_expr)? END #case_expr
| (NOT? EXISTS)? LPAREN select_statement RPAREN type_cast? # subquery_sql_expr
// setting precedence for arithmetic operations:
| left=sql_expr CONCAT right=sql_expr # arithmetic_sql_expr
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels strange to treat concat operator as a arithmetic operator, I know it's for the convenience of parser/analyzer, but this will cause confusion IMO.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I just didn't know where else to include

parse/grammar/KuneiformParser.g4 Outdated Show resolved Hide resolved
parse/analyze.go Show resolved Hide resolved
parse/analyze.go Show resolved Hide resolved
parse/grammar/KuneiformParser.g4 Show resolved Hide resolved
parse/ast.go Outdated Show resolved Hide resolved
parse/grammar/KuneiformParser.g4 Outdated Show resolved Hide resolved
parse/analyze.go Outdated Show resolved Hide resolved
parse/types.go Outdated Show resolved Hide resolved
@brennanjl brennanjl marked this pull request as ready for review May 21, 2024 23:20
@brennanjl brennanjl merged commit ed2c974 into kwilteam:main May 22, 2024
2 checks passed
@brennanjl brennanjl deleted the refactor-parse branch May 22, 2024 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants