Skip to content

Issue4106 #4231

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
potatoqualitee merged 10 commits into from
Oct 17, 2018
Merged

Issue4106 #4231

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
30f14d3
Merge remote-tracking branch 'upstream/development' into development
sqllensman Oct 10, 2018
3f53001
Merge remote-tracking branch 'upstream/development' into development
sqllensman Oct 11, 2018
7f6f367
Merge remote-tracking branch 'upstream/development' into development
sqllensman Oct 12, 2018
496b0b8
Merge remote-tracking branch 'upstream/development' into development
sqllensman Oct 15, 2018
5355741
Adding options for Schema and stopping copy onto self
sqllensman Oct 15, 2018
5b9f7c3
Handle multiple objects returned for Table
sqllensman Oct 15, 2018
a8367e9
Change to Warning Message
sqllensman Oct 15, 2018
dac937a
Updated examples
sqllensman Oct 16, 2018
1bf51b5
Fixing Code and Tests to Detect and Test copying into same location.
sqllensman Oct 16, 2018
fae1a05
Merge branch 'development' into Issue4106
potatoqualitee Oct 16, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 80 additions & 49 deletions functions/Copy-DbaDbTableData.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ function Copy-DbaDbTableData {
The database to copy the table to. If not specified, it is assumed to be the same of Database

.PARAMETER Table
Define a specific table you would like to use as source. You can specify up to three-part name like db.sch.tbl.
Define a specific table you would like to use as source. You can specify a two-part name like sch.tbl.
If the object has special characters please wrap them in square brackets [ ].
This dbo.First.Table will try to find table named 'Table' on schema 'First' and database 'dbo'.
The correct way to find table named 'First.Table' on schema 'dbo' is passing dbo.[First.Table]
Expand All @@ -40,7 +40,9 @@ function Copy-DbaDbTableData {
The table you want to use as destination. If not specified, it is assumed to be the same of Table

.PARAMETER Query
If you want to copy only a portion, specify the query (but please, select all the columns, or nasty things will happen)
If you want to copy only a portion of a table or selected tables, specify the query.
Ensure to select all required columns. Calculated Columns or columns with default values may be excluded.
The tablename should be a full three-part name in form [Database].[Schema].[Table]

.PARAMETER BatchSize
The BatchSize for the import defaults to 5000.
Expand Down Expand Up @@ -103,14 +105,14 @@ function Copy-DbaDbTableData {
https://dbatools.io/Copy-DbaDbTableData

.EXAMPLE
PS C:\> Copy-DbaDbTableData -SqlInstance sql1 -Destination sql2 -Database dbatools_from -Table test_table
PS C:\> Copy-DbaDbTableData -SqlInstance sql1 -Destination sql2 -Database dbatools_from -Table dbo.test_table

Copies all the data from sql1 to sql2, using the database dbatools_from.
Copies all the data from table dbo.test_table in database dbatools_from on sql1 to table test_table in database dbatools_from on sql2.

.EXAMPLE
PS C:\> Copy-DbaDbTableData -SqlInstance sql1 -Destination sql2 -Database dbatools_from -DestinationDatabase dbatools_dest -Table test_table
PS C:\> Copy-DbaDbTableData -SqlInstance sql1 -Destination sql2 -Database dbatools_from -DestinationDatabase dbatools_dest -Table [Schema].[test table]

Copies all the data from sql1 to sql2, using the database dbatools_from as source and dbatools_dest as destination
Copies all the data from table [Schema].[test table] in database dbatools_from on sql1 to table [Schema].[test table] in database dbatools_dest on sql2

.EXAMPLE
PS C:\> Get-DbaDbTable -SqlInstance sql1 -Database tempdb -Table tb1, tb2 | Copy-DbaDbTableData -DestinationTable tb3
Expand All @@ -124,20 +126,29 @@ function Copy-DbaDbTableData {
then
Copies data from tbl2 in tempdb on sql1 to tbl2 in tempdb on sql2

.EXAMPLE
PS C:\> Copy-DbaDbTableData -SqlInstance sql1 -Destination sql2 -Database dbatools_from -Table test_table

Copies all the data from sql1 to sql2, using the database dbatools_from.

.EXAMPLE
PS C:\> Copy-DbaDbTableData -SqlInstance sql1 -Destination sql2 -Database dbatools_from -Table test_table -KeepIdentity -Truncate

Copies all the data from sql1 to sql2, using the database dbatools_from, keeping identity columns and truncating the destination
Copies all the data in table test_table from sql1 to sql2, using the database dbatools_from, keeping identity columns and truncating the destination

.EXAMPLE
PS C:\> Copy-DbaDbTableData -SqlInstance sql1 -Destination sql2 -Database dbatools_from -Table test_table -KeepIdentity -Truncate

Copies all the data from sql1 to sql2, using the database dbatools_from, keeping identity columns and truncating the destination
PS C:\> $params = @{
>> SourceSqlInstance = 'sql1'
>> DestinationSqlInstance = 'sql2'
>> Database = 'dbatools_from'
>> DestinationDatabase = 'dbatools_dest'
>> Table = '[Schema].[Table]'
>> DestinationTable = '[dbo].[Table.Copy]'
>> KeepIdentity = $true
>> KeepNulls = $true
>> Truncate = $true
>> BatchSize = 10000
>> }
>>
PS C:\> Copy-DbaDbTableData @params

Copies all the data from table [Schema].[Table] in database dbatools_from on sql1 to table [dbo].[Table.Copy] in database dbatools_dest on sql2
Keeps identity columns and Nulls, truncates the destination and processes in BatchSize of 10000.

#>
[CmdletBinding(DefaultParameterSetName = "Default", SupportsShouldProcess = $true)]
Expand Down Expand Up @@ -213,6 +224,11 @@ function Copy-DbaDbTableData {
return
}

if ((Test-Bound -Not -ParameterName Destination, DestinationDatabase, DestinationTable)) {
Stop-Function -Message "Cannot copy $Table into itself. One of destination Server, Database or Table must be specified " -Target $Table
return
}

try {
$server = Connect-SqlInstance -SqlInstance $SqlInstance -SqlCredential $SqlCredential
}
Expand All @@ -227,7 +243,14 @@ function Copy-DbaDbTableData {
}

try {
$InputObject += Get-DbaDbTable -SqlInstance $server -Table $Table -Database $Database -EnableException -Verbose:$false
$dbTable = Get-DbaDbTable -SqlInstance $server -Table $Table -Database $Database -EnableException -Verbose:$false
if ($dbTable.Count -eq 1) {
$InputObject += $dbTable
}
else {
Write-Message -Level Warning -Message "The table $Table matches $($dbTable.Count) objects. Unable to determine which object to copy"
continue
}
}
catch {
Stop-Function -Message "Unable to determine source table : $Table"
Expand All @@ -244,7 +267,7 @@ function Copy-DbaDbTableData {
}

if ((Test-Bound -Not -ParameterName DestinationTable)) {
$DestinationTable = $sqltable.Name
$DestinationTable = '[' + $sqltable.Schema + '].[' + $sqltable.Name + ']'
}

if ((Test-Bound -Not -ParameterName Destination)) {
Expand Down Expand Up @@ -283,6 +306,12 @@ function Copy-DbaDbTableData {
$fqtnfrom = "$($server.Databases[$Database]).$sqltable"
$fqtndest = "$($destServer.Databases[$DestinationDatabase]).$desttable"

if ($fqtndest -eq $fqtnfrom -and $server.Name -eq $destServer.Name) {
Stop-Function -Message "Cannot copy $fqtnfrom on $($server.Name) into $fqtndest on ($destServer.Name). Source and Destination must be different " -Target $Table
return
}


if (Test-Bound -ParameterName Query -Not) {
$Query = "SELECT * FROM $fqtnfrom"
}
Expand All @@ -292,24 +321,26 @@ function Copy-DbaDbTableData {
$null = $destServer.Databases[$DestinationDatabase].ExecuteNonQuery("TRUNCATE TABLE $fqtndest")
}
}
$cmd = $server.ConnectionContext.SqlConnectionObject.CreateCommand()
$cmd.CommandText = $Query
if ($server.ConnectionContext.IsOpen -eq $false) {
$server.ConnectionContext.SqlConnectionObject.Open()
}
$bulkCopy = New-Object Data.SqlClient.SqlBulkCopy("$connstring;Database=$DestinationDatabase", $bulkCopyOptions)
$bulkCopy.DestinationTableName = $fqtndest
$bulkCopy.EnableStreaming = $true
$bulkCopy.BatchSize = $BatchSize
$bulkCopy.NotifyAfter = $NotifyAfter
$bulkCopy.BulkCopyTimeOut = $BulkCopyTimeOut

$elapsed = [System.Diagnostics.Stopwatch]::StartNew()
# Add RowCount output
$bulkCopy.Add_SqlRowsCopied({
$RowsPerSec = [math]::Round($args[1].RowsCopied / $elapsed.ElapsedMilliseconds * 1000.0, 1)
Write-Progress -id 1 -activity "Inserting rows" -Status ([System.String]::Format("{0} rows ({1} rows/sec)", $args[1].RowsCopied, $RowsPerSec))
})
if ($Pscmdlet.ShouldProcess($server, "Copy data from $fqtnfrom")) {
$cmd = $server.ConnectionContext.SqlConnectionObject.CreateCommand()
$cmd.CommandText = $Query
if ($server.ConnectionContext.IsOpen -eq $false) {
$server.ConnectionContext.SqlConnectionObject.Open()
}
$bulkCopy = New-Object Data.SqlClient.SqlBulkCopy("$connstring;Database=$DestinationDatabase", $bulkCopyOptions)
$bulkCopy.DestinationTableName = $fqtndest
$bulkCopy.EnableStreaming = $true
$bulkCopy.BatchSize = $BatchSize
$bulkCopy.NotifyAfter = $NotifyAfter
$bulkCopy.BulkCopyTimeOut = $BulkCopyTimeOut

$elapsed = [System.Diagnostics.Stopwatch]::StartNew()
# Add RowCount output
$bulkCopy.Add_SqlRowsCopied({
$RowsPerSec = [math]::Round($args[1].RowsCopied / $elapsed.ElapsedMilliseconds * 1000.0, 1)
Write-Progress -id 1 -activity "Inserting rows" -Status ([System.String]::Format("{0} rows ({1} rows/sec)", $args[1].RowsCopied, $RowsPerSec))
})
}

if ($Pscmdlet.ShouldProcess($destServer, "Writing rows to $fqtndest")) {
$reader = $cmd.ExecuteReader()
Expand All @@ -320,21 +351,21 @@ function Copy-DbaDbTableData {
if ($rowCount -is [int]) {
Write-Progress -id 1 -activity "Inserting rows" -status "Complete" -Completed
}
}

$bulkCopy.Close()
$bulkCopy.Dispose()
$reader.Close()

[pscustomobject]@{
SourceInstance = $server.Name
SourceDatabase = $Database
SourceTable = $sqltable.Name
DestinationInstance = $destServer.name
DestinationDatabase = $DestinationDatabase
DestinationTable = $desttable.Name
RowsCopied = $rowstotal
Elapsed = [prettytimespan]$elapsed.Elapsed
$bulkCopy.Close()
$bulkCopy.Dispose()
$reader.Close()

[pscustomobject]@{
SourceInstance = $server.Name
SourceDatabase = $Database
SourceTable = $sqltable.Name
DestinationInstance = $destServer.name
DestinationDatabase = $DestinationDatabase
DestinationTable = $desttable.Name
RowsCopied = $rowstotal
Elapsed = [prettytimespan]$elapsed.Elapsed
}
}
}
catch {
Expand Down
15 changes: 11 additions & 4 deletions tests/Copy-DbaDbTableData.Tests.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,13 @@ Describe "$commandname Integration Tests" -Tags "IntegrationTests" {
SELECT top 10 1
FROM sys.objects")
$null = $db.Query("CREATE TABLE dbo.dbatoolsci_example2 (id int)")
$null = $db2.Query("CREATE TABLE dbo.dbatoolsci_example3 (id int)")
$null = $db.Query("CREATE TABLE dbo.dbatoolsci_example3 (id int)")
$null = $db.Query("CREATE TABLE dbo.dbatoolsci_example4 (id int);
INSERT dbo.dbatoolsci_example4
SELECT top 13 1
FROM sys.objects")
$null = $db2.Query("CREATE TABLE dbo.dbatoolsci_example (id int)")
$null = $db2.Query("CREATE TABLE dbo.dbatoolsci_example3 (id int)")
$null = $db2.Query("CREATE TABLE dbo.dbatoolsci_example4 (id int);
INSERT dbo.dbatoolsci_example4
SELECT top 13 2
Expand All @@ -40,8 +41,9 @@ Describe "$commandname Integration Tests" -Tags "IntegrationTests" {
AfterAll {
$null = $db.Query("DROP TABLE dbo.dbatoolsci_example")
$null = $db.Query("DROP TABLE dbo.dbatoolsci_example2")
$null = $db2.Query("DROP TABLE dbo.dbatoolsci_example3")
$null = $db.Query("DROP TABLE dbo.dbatoolsci_example3")
$null = $db.Query("DROP TABLE dbo.dbatoolsci_example4")
$null = $db2.Query("DROP TABLE dbo.dbatoolsci_example3")
$null = $db2.Query("DROP TABLE dbo.dbatoolsci_example4")
$null = $db2.Query("DROP TABLE dbo.dbatoolsci_example")
}
Expand All @@ -68,7 +70,7 @@ Describe "$commandname Integration Tests" -Tags "IntegrationTests" {
}

It "supports piping more than one table" {
$results = Get-DbaDbTable -SqlInstance $script:instance1 -Database tempdb -Table dbatoolsci_example2, dbatoolsci_example | Copy-DbaDbTableData -DestinationTable dbatoolsci_example2
$results = Get-DbaDbTable -SqlInstance $script:instance1 -Database tempdb -Table dbatoolsci_example2, dbatoolsci_example | Copy-DbaDbTableData -DestinationTable dbatoolsci_example3
$results.Count | Should -Be 2
}

Expand All @@ -87,4 +89,9 @@ Describe "$commandname Integration Tests" -Tags "IntegrationTests" {
$table4db2check = $db2.Query("select id from dbo.dbatoolsci_example4 where id = 1")
$table4db2check.Count | Should -Be 13
}
}

It "Should return nothing if Source and Destination are same" {
$result = Copy-DbaDbTableData -SqlInstance $script:instance1 -Database tempdb -Table dbatoolsci_example -Truncate
$result | Should Be $null
}
}