Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Name resolution correctness tests #1850

Merged
merged 22 commits into from Jul 5, 2023
Merged

Conversation

max-hoffman
Copy link
Contributor

@max-hoffman max-hoffman commented Jun 23, 2023

This fixes many of the remaining correctness tests for TestSimpleQueries, TestsJoinOps, TestJoinPlanning, TestColumnAliases, TestDerivedTableOuterScopeVisibility, TestAmbiguousColumnResolution, TestReadOnlyVersionedQueries with the new name resolution strategy.

Many of the query plans are slightly different but mostly equivalent. Join rearrangements and un-nesting in particular are better after this change, because I needed the transform logic work for both. There are a variety of other bugs the slight plan differences exposed that are fixed now.

This does not fix every set of enginetests, there is still a lot to do. But I'm locking in compatibility for most of the core tests to prevent backsliding.

The next follow-up is probably replacing the old name resolution. I will need to figure out if triggers, procs, prepared statements need any sort of special treatment.

@max-hoffman max-hoffman changed the title Name res, window funcs and named windows Name resolution correctness tests Jun 29, 2023
@max-hoffman max-hoffman marked this pull request as ready for review June 29, 2023 21:45
@max-hoffman max-hoffman requested a review from zachmu June 29, 2023 21:46
Copy link
Member

@zachmu zachmu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

I have suspicions about the duplication of project nodes, otherwise this looks great

func TestDerivedTableOuterScopeVisibility(t *testing.T) {
enginetest.TestDerivedTableOuterScopeVisibility(t, enginetest.NewDefaultMemoryHarness())
}

func TestDerivedTableOuterScopeVisibility_Experimental(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Dolt run these tests yet? Should probably start

@@ -37,18 +37,15 @@ func (g *FrameFactoryGen) genImports() {
}

func (g *FrameFactoryGen) genNewFrameFactory() {
fmt.Fprintf(g.w, "func NewFrame(ctx *sql.Context, f *ast.Frame) (sql.WindowFrame, error) {\n")
fmt.Fprintf(g.w, "func (b *PlanBuilder) NewFrame(inScope *scope, f *ast.Frame) sql.WindowFrame {\n")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As these become more solidified, it might make sense to invest a bit in some kind of templating language to make them easier to read / modify

If we're committing to code gen to this extent, it should be as easy to manage as possible, i.e. not just a sequence of printf calls

sql/analyzer/hoist_select_exists.go Outdated Show resolved Hide resolved
@@ -868,7 +868,7 @@ func checkForNonAggregatedColumnReferences(w *plan.Window) error {
if agg.Window() == nil {
index, gf := findFirstWindowAggregationColumnReference(w)

if index > 0 {
if index >= 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be a test that was previously passing in error that is now passing with this bug fix

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the new tests broke this

sql/analyzer/validation_rules.go Outdated Show resolved Hide resolved
└─ Table
├─ name: xy
└─ columns: [x y z]
└─ Project
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a redundant projection, is the idea this will get collapsed during analysis so it doesn't matter?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And can we not do that? Not clear what's causing this behavior form this change without much closer inspection

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried a couple deadends, this only works if we pull projection pruning forward. Most direct way of doing that is after we start moving rules onto a new format. If someone hits a perf issue because of that I can do sooner.

sql/planbuilder/parse_test.go Outdated Show resolved Hide resolved
sql/planbuilder/scalar.go Show resolved Hide resolved
sql/planbuilder/scope.go Outdated Show resolved Hide resolved
sql/planbuilder/select.go Show resolved Hide resolved
@max-hoffman max-hoffman merged commit 8a14d2e into main Jul 5, 2023
6 checks passed
@max-hoffman max-hoffman deleted the max/name-res-simple-queries-2 branch July 5, 2023 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants