Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go/vt/sqlparser: statement serialized incorrectly #797

Closed
dvyukov opened this issue Jun 16, 2015 · 0 comments
Closed

go/vt/sqlparser: statement serialized incorrectly #797

dvyukov opened this issue Jun 16, 2015 · 0 comments

Comments

@dvyukov
Copy link
Contributor

dvyukov commented Jun 16, 2015

The following program fails:

package main

import (
    "fmt"
    "github.com/youtube/vitess/go/vt/sqlparser"
)

func main() {
    data := []byte("select a as `BY` from f")
    stmt, err := sqlparser.Parse(string(data))
    if err != nil {
        return
    }
    data1 := sqlparser.String(stmt)
    _, err = sqlparser.Parse(data1)
    if err != nil {
        fmt.Printf("data0: %q\n", data)
        fmt.Printf("data1: %q\n", data1)
        panic(err)
    }
}
data0: "select a as `BY` from f"
data1: "select a as by from f"
panic: syntax error at position 15 near by

on commit 27bdc06

sougou added a commit that referenced this issue Aug 4, 2015
Fix for issue #797.
Names that used keywords were not always getting back-quoted
correctly during codegen. This is a comprehensive fix that
covers all such possible cases.
@sougou sougou closed this as completed Aug 4, 2015
systay pushed a commit to planetscale/vitess that referenced this issue Aug 19, 2022
…essio#949)

* Linter: `go fmt` a comment

Signed-off-by: Patrick Reynolds <patrick@piki.org>

* Stop marking unnormalized stmts as `<error>`

In the first draft of Insights, we unconditionally reported `<error>` as
the normalized SQL statement for any reported statement that had an error.
But that was too limiting, and we wanted to report SQL statements for
errors any time we were reasonably confident they didn't have PII in them.
So in 0b35306, we introduced `LogStats.IsNormalized` to indicate that
a statement had been normalized successfully.

We successfully applied `IsNormalized` to pass along real SQL statements
with errors, if those statements had been normalized.  Unfortunately, we
overdid it and replaced all un-normalized statements with `<error>`
whether or not the statement's execution had caused an error!

It turns out lots of statements that aren't errors, also aren't
normalized.  @rafer found the two functions that control that, in vitessio#797.

This commit fixes all that.  Now we pass a SQL statement in all cases that
we know to be normalized, as well as those that we know did not cause an
error.  We only send `<error>` for statements that are both un-normalized
_and_ an error, which was the intent all along.

Fixes vitessio#797.

Signed-off-by: Patrick Reynolds <patrick@piki.org>

* Add some tests

Signed-off-by: Patrick Reynolds <patrick@piki.org>

* Send raw SQL even if the query wasn't normalized

If the user asks for raw queries, we should send them whether or not the
queries got normalized.  There's no leak or privacy violation once they
opt into raw queries.

There are two kinds of queries that aren't normalized, and we want raw SQL
for both:
 - statements that don't need to be normalized, like `BEGIN`, or `ROLLBACK`
 - syntax errors, for which having raw SQL is useful for debugging

Signed-off-by: Patrick Reynolds <patrick@piki.org>

Signed-off-by: Patrick Reynolds <patrick@piki.org>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com>
dbussink added a commit that referenced this issue Jan 30, 2023
* Linter: `go fmt` a comment

Signed-off-by: Patrick Reynolds <patrick@piki.org>

* Stop marking unnormalized stmts as `<error>`

In the first draft of Insights, we unconditionally reported `<error>` as
the normalized SQL statement for any reported statement that had an error.
But that was too limiting, and we wanted to report SQL statements for
errors any time we were reasonably confident they didn't have PII in them.
So in 0b35306, we introduced `LogStats.IsNormalized` to indicate that
a statement had been normalized successfully.

We successfully applied `IsNormalized` to pass along real SQL statements
with errors, if those statements had been normalized.  Unfortunately, we
overdid it and replaced all un-normalized statements with `<error>`
whether or not the statement's execution had caused an error!

It turns out lots of statements that aren't errors, also aren't
normalized.  @rafer found the two functions that control that, in #797.

This commit fixes all that.  Now we pass a SQL statement in all cases that
we know to be normalized, as well as those that we know did not cause an
error.  We only send `<error>` for statements that are both un-normalized
_and_ an error, which was the intent all along.

Fixes #797.

Signed-off-by: Patrick Reynolds <patrick@piki.org>

* Add some tests

Signed-off-by: Patrick Reynolds <patrick@piki.org>

* Send raw SQL even if the query wasn't normalized

If the user asks for raw queries, we should send them whether or not the
queries got normalized.  There's no leak or privacy violation once they
opt into raw queries.

There are two kinds of queries that aren't normalized, and we want raw SQL
for both:
 - statements that don't need to be normalized, like `BEGIN`, or `ROLLBACK`
 - syntax errors, for which having raw SQL is useful for debugging

Signed-off-by: Patrick Reynolds <patrick@piki.org>

Signed-off-by: Patrick Reynolds <patrick@piki.org>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants