Improve view support for pscale database dump / restore-dump#932
Merged
Improve view support for pscale database dump / restore-dump#932
Conversation
Currently, `dumper.go` is able to export view definitions without issue, but it unfortunately also runs the view and includes, or attempts to include, the data associated with the view too which is unnecessary. The changes here add support for collecting the list of views so that processing can be slightly adjusted if a "table" is actually a view. In that situation, the same `-schema-view.sql` file suffix is utilized that the MyDumper project uses for views and the step that unnecessarily runs the view query is skipped to avoid that issue.
The changes made to `loader.go` add support for collecting any files ending in the view suffix, `-schema-view.sql`, and processing them slightly differently than in `restoreTableSchema()`. The main change within the new `restoreViews()` method, aside from some variable name adjustments and the use of the `viewSuffix`, is the change to using `DROP VIEW` during the overwrite step to properly allow for deleting the existing views. Additionally, the creation of the views is set to occur after the creation of the tables to help avoid that dependency issue.
Some additional cases for the new `information_schema` query needed to be accounted for in the tests.
iheanyi
approved these changes
Nov 4, 2024
orware
added a commit
that referenced
this pull request
Nov 4, 2024
Now that #932 was merged in a little earlier I was able to incorporate some extra show details output specific for views into this PR now. The current `canRestoreSchema()` has also been wrapped around the portion of code that restores the view definitions. In the future we may want to differentiate between restoring tables vs. views only but for now they are both considered to be part of the schema so I'm using the same check for both at the moment which should work ok for now.
This was referenced Nov 25, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This should address the problem described in the following two issues:
#921
#701
And adds support for exporting views with the
-schema-view.sqlsuffix, avoids running the view queries and exporting data from them unnecessarily, and allows for restoring / overwriting the views when the--overwrite-tablesflag is passed in.The goal was to implement the desired functionality with minimal changes to the existing files but the new
information_schemaquery that was added indumper.godid lead to some required additions in thedumper_test.gofile.As I also have PR#910 open at the moment that includes other modifications to
dumper.goandloader.gothis one may be simpler to review / approve first and then I can update the other pull request afterward.