Changing flag for enhanced VPC routing from false to true on redshift serverless results in a validation exception

[meetvasu15]

Describe the bug

When changing flag for enhanced VPC routing from false to true on redshift serverless workgroup using UpdateWorkgroup command, the request returns a validation exception "Can't update multiple configurations at the same time for workgroup [redacted]" even when the request was made with no changes in the existing config parameter values.

Expected Behavior

The CLI should update the enhancedVpcRouting flag.

Current Behavior

The CLI returns {"__type":"ValidationException","message":"Can't update multiple configurations at the same time for workgroup eng-datamart-compute."}

Reproduction Steps

1. Create a redshift serverless work group and namespace.
2. Update enhancedVpcRouting from false to true the redshift serveless workgroup using UpdateWorkgroup command. (Include existing config parameters value with no change in them) .

Possible Solution

No response

Additional Information/Context

Existing config

{
"workgroup": {
"baseCapacity": 32,
"configParameters": [
{
"parameterKey": "auto_mv",
"parameterValue": "true"
},
{
"parameterKey": "datestyle",
"parameterValue": "ISO, MDY"
},
{
"parameterKey": "enable_case_sensitive_identifier",
"parameterValue": "false"
},
{
"parameterKey": "enable_user_activity_logging",
"parameterValue": "true"
},
{
"parameterKey": "query_group",
"parameterValue": "default"
},
{
"parameterKey": "search_path",
"parameterValue": "$user, public"
},
{
"parameterKey": "max_query_execution_time",
"parameterValue": "14400"
}
],
"creationDate": "2022-12-07T22:02:16.411Z",
"endpoint": {
"address": "[redacted]",
"port": 5439,
"vpcEndpoints": [
{
"networkInterfaces": [
{
"availabilityZone": "[redacted]",
"networkInterfaceId": "[redacted]",
"privateIpAddress": "[redacted]",
"subnetId": "[redacted]"
}
],
"vpcEndpointId": "[redacted]",
"vpcId": "[redacted]"
}
]
},
"enhancedVpcRouting": false,
"namespaceName": "[redacted]",
"publiclyAccessible": false,
"securityGroupIds": [
"[redacted]"
],
"subnetIds": [
"subnet-[redacted]",
"subnet-[redacted]",
],
"workgroupArn": "[redacted]",
"workgroupId": "[redacted]",
"workgroupName": "[redacted]"
}
}

UpdateWorkgroup request

{
"baseCapacity": 32,
"configParameters": [
{
"parameterKey": "auto_mv",
"parameterValue": "true"
},
{
"parameterKey": "datestyle",
"parameterValue": "ISO, MDY"
},
{
"parameterKey": "enable_case_sensitive_identifier",
"parameterValue": "false"
},
{
"parameterKey": "enable_user_activity_logging",
"parameterValue": "true"
},
{
"parameterKey": "query_group",
"parameterValue": "default"
},
{
"parameterKey": "search_path",
"parameterValue": "$user, public"
},
{
"parameterKey": "max_query_execution_time",
"parameterValue": "14400"
}
],
"enhancedVpcRouting": true,
"securityGroupIds": [
"sg-0b8b813099b6eafe0"
],
"subnetIds": [
"subnet-[redacted]",
"subnet-[redacted]"
],
"workgroupName": ""[redacted]","
}

CLI version used

2.9.3

Environment details (OS name and version, etc.)

Windows 10

[tim-finnigan]

Hi @meetvasu15 thanks for reaching out. I saw a similar issue here: hashicorp/terraform-provider-aws#26624. That error message you mentioned was also referenced in this comment where the user mentioned:

If an update is attempted that solely affects workgroup, apply will fail, because TF believe multiple configurations are being updated simultaneously. This is now blocking our ability to make any changes to our serverless workgroup.

The comment below that one suggested a workaround, although I can't confirm if that works. But based on this information it seems like this issue could be on the Terraform side.

If you believe the issue is with the CLI directly then please provide debug logs (with any sensitive info redacted) by adding --debug to your command in addition to the full reproduction steps.

[meetvasu15]

Hi @tim-finnigan this is a different issue not really related to iam roles attribute hashicorp/terraform-provider-aws#26624 and I dont think its a terraform issue.

The issue happens when I try to update a base_capacity or enhanced vpc routing flag along with an update to config-parameter using update-workgroup sub command.
Steps to reproduce.

1. Create a redshift serverless workgroup
2. update workgroup base capacity or enahanced vpc routing flag along with one or multiple config parameters.
   The following is a sample of step 2.
   aws redshift-serverless update-workgroup --workgroup-name "[redacted]" --base-capacity 64 --config-parameters "parameterKey=enable_user_activity_logging,parameterValue=true"

This results in a validation exception, the expectation is that the update-workgroup subcommand should update the workgroup per arguments to the command.

Attached debug logs. debug-workgroup-aws-cli.txt

[tim-finnigan]

Thanks for following up - it seems like the issue may involve the underlying UpdateWorkgroup API then rather than directly the AWS CLI. I found an internal discussion where someone mentioned that Can’t update multiple configurations at the same time for workgroup... error and it was found there that the best approach is to delete and recreate the workgroup. They said that deleting the workgroup doesn’t affect the data in the database because the workgroup is the compute layer based on this documentation. Have you tried doing that?

[meetvasu15]

As a work around deleting the workgroup and recreating it resolves the issue of updating the workgroup config - it is not the most convenient way to make updates. It forces us to have a downtime since multiple workgroups cannot be associated with a namespace, we will need to first delete the existing workgroup, VPCE and 53 alias record which takes a couple of minutes and then recreate a new one which also takes a couple of minutes.

Is this issue being tracked anywhere else with the API team so that we can get updates?

[tim-finnigan]

I'm not aware of an internal tracking issue for this but can try looking into it. Do you have an AWS Support plan? If so then we recommend opening a support case as that is a more direct way to escalate API issues to the appropriate team.

[meetvasu15]

Hi @tim-finnigan thanks for the response and the suggestion, I opened a support case with AWS.

[tim-finnigan]

Thanks for following up. Please let us know here if you get any updates that may help others, and if any issue is found directly relating to the CLI then we can revisit this. I'll set this issue to auto-close after a few days if we don't hear back.

[meetvasu15]

Follow up from AWS

The team, in conjunction with development, confirmed that the behavior that you are experiencing is intended and that multiple configuration changes are not allowed in tandem as the error message suggests.
I pointed out to the team that the CLI documentation does not indicate this as a constraint, therefore the team will be updating public documentation to clarify this constraint.