Skip to content
This repository has been archived by the owner on May 31, 2022. It is now read-only.

Add support for views in diffing #15

Merged
merged 48 commits into from
Jan 27, 2022
Merged

Conversation

iheanyi
Copy link
Member

@iheanyi iheanyi commented Nov 1, 2021

This pull request adds support for view diffing to Tengo. I pretty much followed how we are doing view diffing for things such as tables and such and tried to do the same thing for views where possible. If anything looks off, let me know, but I couldn't think of any better ways of doing this.

TODO:

  • Introspection for Views
  • Supporting create options, security types, and algorithms (maybe punt on this)
  • Tests

@coveralls
Copy link

coveralls commented Nov 1, 2021

Pull Request Test Coverage Report for Build 1753758663

  • 231 of 295 (78.31%) changed or added relevant lines in 6 files are covered.
  • 27 unchanged lines in 3 files lost coverage.
  • Overall coverage decreased (-1.2%) to 92.487%

Changes Missing Coverage Covered Lines Changed/Added Lines %
schema.go 19 20 95.0%
instance.go 17 25 68.0%
diff.go 73 83 87.95%
introspect.go 66 81 81.48%
view.go 56 71 78.87%
view_alterclause.go 0 15 0.0%
Files with Coverage Reduction New Missed Lines %
table.go 2 99.23%
instance.go 4 88.16%
introspect.go 21 91.02%
Totals Coverage Status
Change from base Build 1397987024: -1.2%
Covered Lines: 3767
Relevant Lines: 4073

💛 - Coveralls

@iheanyi iheanyi marked this pull request as ready for review January 26, 2022 23:29
@shlomi-noach shlomi-noach self-requested a review January 27, 2022 06:35
Copy link

@shlomi-noach shlomi-noach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very good to me! I left a lot of 👍 along the way because it was a long change and I wanted to keep track of the parts I reviewed -- you can safely mark those as resolved.

The one thing I'm unsure of: there's excellent test, but I'm not sure I saw a test that has a non-programmatic nature to comparing views in two schemas. To explain better, and I'm not even sure this is generally supported in tengo: is there a way to just have some SQL, as in testdata/integration.go, but have two different versions of this, then have the integration test show a CREATE OR REPLACE VIEW statement? What I see in existence is programmatic comparison of views; but maybe I misunderstand how this works and am missing the point; maybe what I'm asking about already exists.

@@ -95,6 +95,8 @@ type SchemaDiff struct {
ToSchema *Schema
TableDiffs []*TableDiff // a set of statements that, if run, would turn tables in FromSchema into ToSchema
RoutineDiffs []*RoutineDiff // " but for funcs and procs

ViewDiffs []*ViewDiff // a set of statements that, if run, would turn views in FromSchema into ToSchema

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

}

return viewDiffs
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍


return vd.Type
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

if stmt, err := vd.Statement(StatementModifiers{}); err != nil || !strings.HasPrefix(stmt, "CREATE OR REPLACE") {
t.Errorf("Unexpected return value from Statement(): %s / %s", stmt, err)
}
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

}

return row.CreateStatement, nil
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

return nil, err
}

g, subCtx := errgroup.WithContext(ctx)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL errgroup and I'm embarrassed.

}

func extractAlgorithm(statement string) string {
r := regexp.MustCompile(`(?i)\bALGORITHM=(?P<Algorithm>MERGE|UNDEFINED|TEMPTABLE)\b`)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

}

return views, nil
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

for name, view := range s.ViewsByName() {
key := ObjectKey{Type: ObjectTypeView, Name: name}
dict[key] = view.CreateStatement
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

if va, ok := viewAlters[0].(ChangeViewDefinition); !ok {
t.Errorf("Expected alters[0] to be a ChangeViewDefinition, instead found %T", va)
}
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@iheanyi
Copy link
Member Author

iheanyi commented Jan 27, 2022

Thanks for the review @shlomi-noach! Yeah, I can try and do that for sure. Lemme see what I can whip up today.

@shlomi-noach
Copy link

@iheanyi cool! If this does not play well with how the current tests work, no worries -- we have Vitess tests on top of that, so even though it's one repo apart, we can still have some form of validation.

@iheanyi
Copy link
Member Author

iheanyi commented Jan 27, 2022

Perfect, I'll merge it in then if I fall short :)

@iheanyi
Copy link
Member Author

iheanyi commented Jan 27, 2022

@shlomi-noach So I feel like given the fact that we can introspect a schema and then test our diffing logic in diff_test.go, it covers the case of schema diffing in this case. The introspection ensures that the maps are populated properly with the schema results, while also ensuring that we're diffing properly with those view objects in that other test. Gonna go ahead and merge

@iheanyi iheanyi merged commit 74f7658 into main Jan 27, 2022
@shlomi-noach shlomi-noach deleted the iheanyi/add-support-for-views branch January 27, 2022 15:37
@shlomi-noach
Copy link

@iheanyi sounds good!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
3 participants