Invoke-DbaDbDataMasking - Improve performance #9671
+253
−286
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Switch from column-based masking to row-based masking.
| $uniqueData = Invoke-DbaQuery -SqlInstance $server -SqlCredential $SqlCredential -Database tempdb -Query $query | ||
| } catch { | ||
| Stop-Function -Message "Something went wrong getting the unique data" -Target $query -ErrorRecord $_ | ||
| continue |
There was a problem hiding this comment.
Why using "continue" here (you could use the -Continue parameter of Stop-Function) and "return" a few rows later? We need a general way to handle errors.
There was a problem hiding this comment.
I will take a look at that and get it worked out the other way.
|
This is a big change and not easy to review. Best would be to let @sanderstad have a look at this and approve it. Sander, do you have the time? |
10 tasks
|
As #9664 is merged, you should update your branch to resolve the conflict. Then we can continue to review this PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Switch from column-based masking to row-based masking.
Type of Change
Invoke-ManualPester)Purpose
To improve the speed at which Invoke-DbaDbDataMasking masks data in large tables
Approach
This change switches from Column by column masking to row by row masking, allowing for a batch of rows to be done at a single time, instead of having to iterate over each column to be masked in the table for each row.