Updating manifest fails when addons stack present - empty addons stack changeset results in failure

[ghost]

If an application has an addons stack defined, after initial successful deployment, making a change to the manifest.yml and redeploying fails - Cloudformation gives an error that there are no changes to make to the addons stack and the deployment fails - which is technically true in that there are no changes to the addons stack yml, but to the main manifest yml. Any subsequent changes which require redeployment with the addons stack in place fails due to the changeset status failing.

[huanjani]

Hello, @mptaws!

Thanks for your patience as we dug into this. I was able to recreate this failure with your addons template, though when I did the same steps with a different addons template, the redeployment succeeded.

We believe the issue lies with the double-nested stack that is created by the macros in Transform:. You don't appear to be the only one having this CloudFormation problem (https://forums.aws.amazon.com/thread.jspa?threadID=336657&tstart=0)! What happens if you inline those two resources rather than using the macros?

[ghost]

The transform happens as part of the addition of the secret rotation template and creation of a rotation lambda. There may be an alternate way to include that logic and/or just copy the lambda creation into the main template. That being said - this was functional prior to 1.4. I’ll see if I can experiment with other method of adding a rotation.

…

--- -mpt On March 24, 2021 at 11:58:55 AM PDT, Janice Huang ***@***.***> wrote: Hello, @mptaws<https://github.com/mptaws>! Thanks for your patience as we dug into this. I was able to recreate this failure with your addons template, though when I did the same steps with a different addons template, the redeployment succeeded. We believe the issue lies with the double-nested stack that is created by the macros in Transform:. You don't appear to be the only one having this CloudFormation problem (https://forums.aws.amazon.com/thread.jspa?threadID=336657&tstart=0)! What happens if you inline those two resources rather than using the macros? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#2089 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKD3N2C7JL6NP4OT2VKBJFLTFIY3PANCNFSM4ZT7RKGA>.

[huanjani]

I tried this with 1.3 and had the same issue. It worked with that same template? 🤔

[ghost]

I did extensive testing on the template prior to the 1.4 release and didn’t encounter this issue. I have a workaround for the workshop I’m building - I’ll try your suggestion of inline-ing the templates to get rid of the extra nest - it still seems to me that deploying a template that has no changes no matter how deeply nested shouldn’t fail deployment.

…

--- -mpt On March 24, 2021 at 2:08:39 PM PDT, Janice Huang ***@***.***> wrote: I tried this with 1.3 and had the same issue. It worked with that same template? 🤔 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#2089 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AKD3N2CRQSFYLSREJX6UW7DTFJIBJANCNFSM4ZT7RKGA>.

[huanjani]

Hi @mptaws!

Just curious if this issue persists...?

[srbartlett]

Hi, I'm seeing a similar issue. My scenario is:
Single LB service with Aurora RDS addon

Creating a new environment and running deploy fails with

a9e7f692: digest: sha256:d1d8e52c0ce40f81077774b22f8252fabb6e87c535b7bfbb3c067c269f01e549 size: 3261
✘ Proposing infrastructure changes for stack xxxx
✘ execute svc deploy: deploy service: change set with name copilot-xxx for stack xxx has no changes

It leaves the stack in REVIEW_IN_PROGRESS

Deleting the addon and re-running deploy works as expected.

[Lou1415926]

Hi @srbartlett. Thank you for reaching out for the issue!!

Would you mind giving me more information on what happened? This will help me reproduce the error!

Meanwhile, I've seen this error caused by a malformed addon template. If you have customized your template, you can run aws cloudformation validate-template --template-body file://path/to/your/template.yml (note that file:// is necessary) to see if there is anything wrong with the template?

Related: #2257

[efekarakus]

This is now fixed in v1.7.0!! 🥳 https://github.com/aws/copilot-cli/releases/tag/v1.7.0

[huanjani]

Reopening this, as the underlying cause of this problem was a CFN issue.