Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove readonly properties within replication_settings for aws_dms_replication_config resource #34356

Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
982a720
allows move to new instance arn
misterjacko Apr 13, 2023
c6b05aa
fixes acc tests
misterjacko Apr 14, 2023
ef79b67
adds changelog
Apr 14, 2023
fb8b829
add acc test
misterjacko Apr 14, 2023
34f450c
increases some timeouts
misterjacko Apr 20, 2023
d2bcc3f
feat: use RFC3339 date for cdc_start_time
Jun 12, 2023
006d554
feat: use RFC3339 date for cdc_start_time
jeremychauvet Jun 12, 2023
8ec3d39
Merge branch 'b-19801-dms-replication-task-cdc-start-time-allow-non-u…
Jun 12, 2023
2472e54
feat: use RFC3339 date for cdc_start_time
jeremychauvet Jun 12, 2023
d26698b
feat: allow RFC3339 date for cdc_start_time
jeremychauvet Jun 13, 2023
6ab260e
feat: allow RFC3339 date for cdc_start_time
jeremychauvet Jun 13, 2023
b8b643b
feat: remove readonly properties and updated tests
DanielRieske Nov 11, 2023
942efea
chore: added changelog
DanielRieske Nov 11, 2023
b5b160a
chore: fmt changelog
DanielRieske Nov 11, 2023
5f6d052
Merge branch 'main' into HEAD
ewbankkit Dec 14, 2023
87ba32e
Add 'internal/json' package.
ewbankkit Dec 14, 2023
9613585
'internal/ujson' -> 'internal/json/ujson'.
ewbankkit Dec 14, 2023
71251ea
Add 'json.RemoveReadOnlyFields'.
ewbankkit Dec 14, 2023
6ce0b89
Add 'verify.SuppressEquivalentJSONRemovingReadOnlyFieldsDiffs'.
ewbankkit Dec 14, 2023
44c774e
r/aws_dms_replication_config: Use 'verify.SuppressEquivalentJSONRemov…
ewbankkit Dec 14, 2023
144aeb8
startReplication: replication NotFound is OK.
ewbankkit Dec 14, 2023
99d7818
dms: Add 'TaskSettings'.
ewbankkit Dec 14, 2023
e455fcf
'RemoveReadOnlyFields' -> 'RemoveFields'.
ewbankkit Dec 14, 2023
5c37118
Move 'internal/experimental/depgraph/stack' -> 'internal/types/stack'.
ewbankkit Dec 15, 2023
106f2c1
Add 'json.RemoveEmptyFields'.
ewbankkit Dec 15, 2023
825407b
'flattenSettings' -> 'flattenTaskSettings'.
ewbankkit Dec 15, 2023
4f8ddd6
Fix 'TestStack'.
ewbankkit Dec 15, 2023
370ec89
Move 'internal/option' -> 'internal/types/option'.
ewbankkit Dec 15, 2023
d1e9a80
generate/tags: 'internal/option' -> 'internal/types/option'.
ewbankkit Dec 15, 2023
bb6806c
Run 'make gen'.
ewbankkit Dec 15, 2023
5060bd2
provider: 'internal/option' -> 'internal/types/option'.
ewbankkit Dec 15, 2023
2f620b8
stack.Peek() and Pop() return Option.
ewbankkit Dec 15, 2023
1e47a79
json.RemoveEmptyFields: Use a stack.
ewbankkit Dec 18, 2023
9c30cb0
json.RemoveEmptyFields: 'string' -> '[]byte'.
ewbankkit Dec 18, 2023
87b4841
dms: Add 'suppressEquivalentTaskSettings'.
ewbankkit Dec 18, 2023
80ff3ae
dms: Add 'setLastReplicationError'.
ewbankkit Dec 18, 2023
2a313df
r/aws_dms_replication_task: Use 'dms.MigrationTypeValue_Values()'.
ewbankkit Dec 18, 2023
bc2419c
Merge branch 'main' into HEAD
ewbankkit Dec 18, 2023
fa60946
Merge commit '6ab260e1e5ace4a802f78da379b2c3ab61094e2a' into HEAD
ewbankkit Dec 18, 2023
7d8b609
Tweak CHANGELOG entry.
ewbankkit Dec 18, 2023
2584012
Tweak CHANGELOG entry.
ewbankkit Dec 18, 2023
be6996b
Tweak documentation.
ewbankkit Dec 18, 2023
c6c270f
Add 'timestamp.IsRFC3339()'.
ewbankkit Dec 18, 2023
60da92a
Add 'flex.StringValueToInt64Value'.
ewbankkit Dec 18, 2023
2873e18
r/aws_dms_replication_task: Correct update of 'cdc_start_time'.
ewbankkit Dec 18, 2023
c6f7689
Merge commit '34f450c6f2d8bffc04455cec506289513332b2d1' into HEAD
ewbankkit Dec 18, 2023
5da0ce1
Tweak CHANGELOG entry.
ewbankkit Dec 18, 2023
ac5377b
r/aws_dms_replication_task: Tidy up Create.
ewbankkit Dec 18, 2023
abe4bc1
r/aws_dms_replication_task: Tidy up Read.
ewbankkit Dec 18, 2023
c99231d
r/aws_dms_replication_task: Prevent erroneous diffs on `replication_t…
ewbankkit Dec 18, 2023
e75ba7f
dms: Tidy up 'startReplicationTask'.
ewbankkit Dec 18, 2023
940841a
dms: Tidy up 'stopReplicationTask'.
ewbankkit Dec 18, 2023
956aff9
r/aws_dms_replication_task: Tidy up Delete.
ewbankkit Dec 18, 2023
e3b0689
r/aws_dms_replication_task: Tidy up Update.
ewbankkit Dec 18, 2023
6020ccf
r/aws_dms_replication_task: Tidy up waiters.
ewbankkit Dec 18, 2023
2254145
r/aws_dms_replication_task: Tidy up acceptance tests.
ewbankkit Dec 18, 2023
cca7ce7
Fix semgrep 'dgryski.semgrep-go.oddifsequence.odd-sequence-ifs'.
ewbankkit Dec 18, 2023
3527c44
Fix terrafmt error.
ewbankkit Dec 18, 2023
d0ded6b
dms: Tweak acceptance test configuration names.
ewbankkit Dec 19, 2023
b025e27
Simplify 'TestAccDMSEndpoint_pauseReplicationTasks'.
ewbankkit Dec 19, 2023
a50d08e
Simplify 'TestAccDMSEndpoint_pauseReplicationTasks' again.
ewbankkit Dec 19, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .changelog/34356.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:bug
resource/aws_dms_replication_config: Remove read-only properties from `replication_settings` to suppress diffs
```
44 changes: 40 additions & 4 deletions internal/service/dms/replication_config.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ package dms

import (
"context"
"encoding/json"
"fmt"
"log"
"time"
Expand Down Expand Up @@ -112,9 +113,11 @@ func ResourceReplicationConfig() *schema.Resource {
ForceNew: true,
},
"replication_settings": {
Type: schema.TypeString,
Optional: true,
Computed: true,
Type: schema.TypeString,
Optional: true,
Computed: true,
ValidateFunc: validation.StringIsJSON,
DiffSuppressFunc: verify.SuppressEquivalentJSONDiffs,
},
"replication_type": {
Type: schema.TypeString,
Expand Down Expand Up @@ -230,13 +233,19 @@ func resourceReplicationConfigRead(ctx context.Context, d *schema.ResourceData,
return sdkdiag.AppendErrorf(diags, "setting compute_config: %s", err)
}
d.Set("replication_config_identifier", replicationConfig.ReplicationConfigIdentifier)
d.Set("replication_settings", replicationConfig.ReplicationSettings)
d.Set("replication_type", replicationConfig.ReplicationType)
d.Set("source_endpoint_arn", replicationConfig.SourceEndpointArn)
d.Set("supplemental_settings", replicationConfig.SupplementalSettings)
d.Set("table_mappings", replicationConfig.TableMappings)
d.Set("target_endpoint_arn", replicationConfig.TargetEndpointArn)

settings, err := replicationConfigRemoveReadOnlySettings(aws.StringValue(replicationConfig.ReplicationSettings))
if err != nil {
return sdkdiag.AppendErrorf(diags, "reading DMS Replication Config (%s): %s", d.Id(), err)
}

d.Set("replication_settings", settings)

return diags
}

Expand Down Expand Up @@ -655,3 +664,30 @@ func expandComputeConfigInput(tfMap map[string]interface{}) *dms.ComputeConfig {

return apiObject
}

func replicationConfigRemoveReadOnlySettings(settings string) (*string, error) {
var settingsData map[string]interface{}
if err := json.Unmarshal([]byte(settings), &settingsData); err != nil {
return nil, err
}

controlTablesSettings, ok := settingsData["ControlTablesSettings"].(map[string]interface{})
if ok {
delete(controlTablesSettings, "historyTimeslotInMinutes")
}

logging, ok := settingsData["Logging"].(map[string]interface{})
if ok {
delete(logging, "EnableLogContext")
delete(logging, "CloudWatchLogGroup")
delete(logging, "CloudWatchLogStream")
}

cleanedSettings, err := json.Marshal(settingsData)
if err != nil {
return nil, err
}

cleanedSettingsString := string(cleanedSettings)
return &cleanedSettingsString, nil
}
1 change: 1 addition & 0 deletions internal/service/dms/replication_config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -386,6 +386,7 @@ resource "aws_dms_replication_config" "test" {
source_endpoint_arn = aws_dms_endpoint.source.endpoint_arn
target_endpoint_arn = aws_dms_endpoint.target.endpoint_arn
table_mappings = "{\"rules\":[{\"rule-type\":\"selection\",\"rule-id\":\"1\",\"rule-name\":\"1\",\"object-locator\":{\"schema-name\":\"%%\",\"table-name\":\"%%\"},\"rule-action\":\"include\"}]}"
replication_settings = "{\"BeforeImageSettings\":null,\"ChangeProcessingDdlHandlingPolicy\":{\"HandleSourceTableAltered\":true,\"HandleSourceTableDropped\":true,\"HandleSourceTableTruncated\":true},\"ChangeProcessingTuning\":{\"BatchApplyMemoryLimit\":500,\"BatchApplyPreserveTransaction\":true,\"BatchApplyTimeoutMax\":30,\"BatchApplyTimeoutMin\":1,\"BatchSplitSize\":0,\"CommitTimeout\":1,\"MemoryKeepTime\":60,\"MemoryLimitTotal\":1024,\"MinTransactionSize\":1000,\"StatementCacheSize\":50},\"CharacterSetSettings\":null,\"ControlTablesSettings\":{\"CommitPositionTableEnabled\":false,\"ControlSchema\":\"\",\"FullLoadExceptionTableEnabled\":false,\"HistoryTableEnabled\":false,\"HistoryTimeslotInMinutes\":5,\"StatusTableEnabled\":false,\"SuspendedTablesTableEnabled\":false},\"ErrorBehavior\":{\"ApplyErrorDeletePolicy\":\"IGNORE_RECORD\",\"ApplyErrorEscalationCount\":0,\"ApplyErrorEscalationPolicy\":\"LOG_ERROR\",\"ApplyErrorFailOnTruncationDdl\":false,\"ApplyErrorInsertPolicy\":\"LOG_ERROR\",\"ApplyErrorUpdatePolicy\":\"LOG_ERROR\",\"DataErrorEscalationCount\":0,\"DataErrorEscalationPolicy\":\"SUSPEND_TABLE\",\"DataErrorPolicy\":\"LOG_ERROR\",\"DataTruncationErrorPolicy\":\"LOG_ERROR\",\"EventErrorPolicy\":\"IGNORE\",\"FailOnNoTablesCaptured\":false,\"FailOnTransactionConsistencyBreached\":false,\"FullLoadIgnoreConflicts\":true,\"RecoverableErrorCount\":-1,\"RecoverableErrorInterval\":5,\"RecoverableErrorStopRetryAfterThrottlingMax\":false,\"RecoverableErrorThrottling\":true,\"RecoverableErrorThrottlingMax\":1800,\"TableErrorEscalationCount\":0,\"TableErrorEscalationPolicy\":\"STOP_TASK\",\"TableErrorPolicy\":\"SUSPEND_TABLE\"},\"FailTaskWhenCleanTaskResourceFailed\":false,\"FullLoadSettings\":{\"CommitRate\":10000,\"CreatePkAfterFullLoad\":false,\"MaxFullLoadSubTasks\":8,\"StopTaskCachedChangesApplied\":false,\"StopTaskCachedChangesNotApplied\":false,\"TargetTablePrepMode\":\"DROP_AND_CREATE\",\"TransactionConsistencyTimeout\":600},\"Logging\":{\"EnableLogging\":false,\"LogComponents\":[{\"Id\":\"TRANSFORMATION\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"SOURCE_UNLOAD\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"IO\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"TARGET_LOAD\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"PERFORMANCE\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"SOURCE_CAPTURE\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"SORTER\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"REST_SERVER\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"VALIDATOR_EXT\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"TARGET_APPLY\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"TASK_MANAGER\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"TABLES_MANAGER\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"METADATA_MANAGER\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"FILE_FACTORY\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"COMMON\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"ADDONS\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"DATA_STRUCTURE\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"COMMUNICATION\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"},{\"Id\":\"FILE_TRANSFER\",\"Severity\":\"LOGGER_SEVERITY_DEFAULT\"}]},\"LoopbackPreventionSettings\":null,\"PostProcessingRules\":null,\"StreamBufferSettings\":{\"CtrlStreamBufferSizeInMB\":5,\"StreamBufferCount\":3,\"StreamBufferSizeInMB\":8},\"TTSettings\":{\"EnableTT\":false,\"FailTaskOnTTFailure\":false,\"TTRecordSettings\":null,\"TTS3Settings\":null},\"TargetMetadata\":{\"BatchApplyEnabled\":false,\"FullLobMode\":false,\"InlineLobMaxSize\":0,\"LimitedSizeLobMode\":true,\"LoadMaxFileSize\":0,\"LobChunkSize\":0,\"LobMaxSize\":32,\"ParallelApplyBufferSize\":0,\"ParallelApplyQueuesPerThread\":0,\"ParallelApplyThreads\":0,\"ParallelLoadBufferSize\":0,\"ParallelLoadQueuesPerThread\":0,\"ParallelLoadThreads\":0,\"SupportLobs\":true,\"TargetSchema\":\"\",\"TaskRecoveryTableEnabled\":false}}"

compute_config {
replication_subnet_group_id = aws_dms_replication_subnet_group.test.replication_subnet_group_id
Expand Down
Loading