Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: <Error during ADF deployment cfn_custom_resource> #696

Closed
1 of 2 tasks
giorgioGTelian opened this issue Mar 26, 2024 · 4 comments
Closed
1 of 2 tasks

[Bug]: <Error during ADF deployment cfn_custom_resource> #696

giorgioGTelian opened this issue Mar 26, 2024 · 4 comments
Labels
bug Something isn't working

Comments

@giorgioGTelian
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Describe the bug

the errors arose during the initial phase of the deployment, initiated through the command make deployment. Despite ensuring all information was accurately filled in, including specifying the Organizational Unit (OU), we encountered an unexpected error.
The attached images indicate that this issue stems from a component called cfn_custom_resource, which seems to be the source of the complication.
you can vision the errors in the properties Organization and DetermineFefaultBranchname

Expected Behavior

I expected the cloudFormation service to be successful without any error.

Current Behavior

this is the current behaviour as of it now it gaves an error in the properties Organization and DetermineFefaultBranchname

image (1)
image

Steps To Reproduce

  1. git clone https://github.com/awslabs/aws-deployment-framework
  2. cd aws-deployment-framework
  3. git fetch --verbose --tags
  4. git checkout ${version_tag_goes_here}
  5. make update_makefile
  6. make and make clean
  7. make deploy putting the informations and creating the mandatory deployment OU in the AWS account

Possible Solution

No response

Additional Information/Context

No response

ADF Version

the latest version

Contributing a fix?

  • Yes, I am working on a fix to resolve this issue
@giorgioGTelian giorgioGTelian added the bug Something isn't working label Mar 26, 2024
@sbkok
Copy link
Collaborator

sbkok commented Mar 26, 2024

Thanks for reporting this issue.
Unfortunately I'm unable to replicate this issue.

It sounds like the custom resources did not build correctly. During the make or make build phase it will run sam build under the hood to build all the Lambda functions. This command will fetch the required dependencies and package them so they can be deployed to the Lambda functions.

Could you try running this:

make clean build deploy

If that does not work, could you report back the versions it was using? You can do so by running the following command and copying the output here:

make report_versions

@giorgioGTelian
Copy link
Author

hello,
this is the report:

*** Beginning of ADF Version Report ***
ADF Source version: v3.2.0
ADF SemanticVersion: 3.2.0

x86_64
Hardware platform: x86_64
Kernel name: Linux
Kernel release: 5.4.0-174-generic

OS Release:
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

Disk:
Filesystem                         Size  Used Avail Use% Mounted on
/dev/mapper/ubuntu--vg-ubuntu--lv   15G   13G  1.1G  93% /

Dependencies:
docker: Docker version 26.0.0, build 2ae903e
git: git version 2.25.1
sed: sed (GNU sed) 4.7
Packaged by Debian
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Jay Fenlason, Tom Lord, Ken Pizzini,
Paolo Bonzini, Jim Meyering, and Assaf Gordon.
GNU sed home page: <https://www.gnu.org/software/sed/>.
General help using GNU software: <https://www.gnu.org/gethelp/>.
E-mail bug reports to: <bug-sed@gnu.org>.
make: GNU Make 4.2.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
pip 20.0.2 from /home/eetech/aws-deployment-framework/.venv/lib/python3.8/site-packages/pip (python 3.8)
Package                   Version
------------------------- --------------
annotated-types           0.6.0
argcomplete               3.2.3
arrow                     1.3.0
attrs                     23.2.0
aws-lambda-builders       1.47.0
aws-sam-cli               1.113.0
aws-sam-translator        1.86.0
backports.zoneinfo        0.2.1
binaryornot               0.4.4
blinker                   1.7.0
boto3                     1.34.71
boto3-stubs               1.34.61
botocore                  1.34.71
botocore-stubs            1.34.69
certifi                   2024.2.2
cffi                      1.16.0
cfn-lint                  0.86.1
chardet                   5.2.0
charset-normalizer        3.3.2
chevron                   0.14.0
click                     8.1.7
cookiecutter              2.6.0
cryptography              42.0.5
dateparser                1.2.0
docker                    7.0.0
flask                     3.0.2
idna                      3.6
importlib-metadata        7.1.0
importlib-resources       6.4.0
itsdangerous              2.1.2
Jinja2                    3.1.3
jmespath                  1.0.1
jschema-to-python         1.2.3
jsonpatch                 1.33
jsonpickle                3.0.3
jsonpointer               2.4
jsonschema                4.21.1
jsonschema-specifications 2023.12.1
junit-xml                 1.9
markdown-it-py            3.0.0
MarkupSafe                2.1.5
mdurl                     0.1.2
mpmath                    1.3.0
mypy-boto3-apigateway     1.34.56
mypy-boto3-cloudformation 1.34.66
mypy-boto3-ecr            1.34.0
mypy-boto3-iam            1.34.8
mypy-boto3-kinesis        1.34.0
mypy-boto3-lambda         1.34.58
mypy-boto3-s3             1.34.65
mypy-boto3-schemas        1.34.0
mypy-boto3-secretsmanager 1.34.63
mypy-boto3-signer         1.34.0
mypy-boto3-sqs            1.34.0
mypy-boto3-stepfunctions  1.34.0
mypy-boto3-sts            1.34.0
mypy-boto3-xray           1.34.0
networkx                  3.1
packaging                 24.0
pbr                       6.0.0
pip                       20.0.2
pkg-resources             0.0.0
pkgutil-resolve-name      1.3.10
pycparser                 2.21
pydantic                  2.6.4
pydantic-core             2.16.3
pygments                  2.17.2
pyOpenSSL                 24.1.0
python-dateutil           2.9.0.post0
python-slugify            8.0.4
pytz                      2024.1
PyYAML                    6.0.1
referencing               0.34.0
regex                     2023.12.25
requests                  2.31.0
rich                      13.7.1
rpds-py                   0.18.0
ruamel.yaml               0.18.6
ruamel.yaml.clib          0.2.8
s3transfer                0.10.1
sarif-om                  1.0.4
setuptools                44.0.0
six                       1.16.0
sympy                     1.12
text-unidecode            1.3
tomlkit                   0.12.4
types-awscrt              0.20.5
types-python-dateutil     2.9.0.20240316
types-s3transfer          0.10.0
typing-extensions         4.10.0
tzlocal                   5.2
urllib3                   2.2.1
watchdog                  4.0.0
werkzeug                  3.0.1
wheel                     0.43.0
xmltodict                 0.13.0
yq                        3.2.3
zipp                      3.18.1

git status
HEAD detached at v3.2.0
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   Makefile

no changes added to commit (use "git add" and/or "git commit -a")
*** End of ADF Version Report ***

we have also tried to run make from the root directory and then sam deploy from the resultant builded application located in the sam installation folder this is the result:

Initiating deployment
=====================
	Uploading to serverlessrepo-aws-deployment-framework/2df7a105ced85640ab41b6c9949a21f4.template  76669 / 76669  (100.00%)
Error: Failed to create changeset for the stack: serverlessrepo-aws-deployment-framework, An error occurred (ValidationError) when calling the CreateChangeSet operation: Template format error: v3.2.0"2010-09-09" is not a supported value forAWSTemplateFormatVersion.

@giorgioGTelian
Copy link
Author

we got the bug fixed the problem was the python version that must be 3.12 and we missed jq the command-line JSON processor

sbkok added a commit to sbkok/aws-deployment-framework that referenced this issue Mar 28, 2024
**Why?**

When running the make / install process of ADF, we need to ensure it will not
continue if some of the tools are not installed yet.

For example, in the past, if you did not install jq, it would still continue
with the build process. Resulting in a broken template at deployment time.

As reported in awslabs#696, the Python version in the environment was set to v3.8,
which resulted in a broken installation process too.

**What?**

* Updated the documentation to indicate that `jq` needs to be available too.
* Added checks in the build process to exit and warn the user about missing
  dependencies.
sbkok added a commit to sbkok/aws-deployment-framework that referenced this issue Mar 28, 2024
**Why?**

When running the make / install process of ADF, we need to ensure it will not
continue if some of the tools are not installed yet.

For example, in the past, if you did not install jq, it would still continue
with the build process. Resulting in a broken template at deployment time.

As reported in awslabs#696, the Python version in the environment was set to v3.8,
which resulted in a broken installation process too.

**What?**

* Updated the documentation to indicate that `jq` needs to be available too.
* Added checks in the build process to exit and warn the user about missing
  dependencies.
* Updated the version number of the Makefile.
sbkok added a commit to sbkok/aws-deployment-framework that referenced this issue Apr 3, 2024
**Why?**

When running the make / install process of ADF, we need to ensure it will not
continue if some of the tools are not installed yet.

For example, in the past, if you did not install jq, it would still continue
with the build process. Resulting in a broken template at deployment time.

As reported in awslabs#696, the Python version in the environment was set to v3.8,
which resulted in a broken installation process too.

**What?**

* Updated the documentation to indicate that `jq` needs to be available too.
* Added checks in the build process to exit and warn the user about missing
  dependencies.
* Updated the version number of the Makefile.
sbkok added a commit to sbkok/aws-deployment-framework that referenced this issue Apr 3, 2024
**Why?**

When running the make / install process of ADF, we need to ensure it will not
continue if some of the tools are not installed yet.

For example, in the past, if you did not install jq, it would still continue
with the build process. Resulting in a broken template at deployment time.

As reported in awslabs#696, the Python version in the environment was set to v3.8,
which resulted in a broken installation process too.

**What?**

* Updated the documentation to indicate that `jq` needs to be available too.
* Added checks in the build process to exit and warn the user about missing
  dependencies.
* Updated the version number of the Makefile.
sbkok added a commit that referenced this issue Apr 5, 2024
)

**Why?**

When running the make / install process of ADF, we need to ensure it will not
continue if some of the tools are not installed yet.

For example, in the past, if you did not install jq, it would still continue
with the build process. Resulting in a broken template at deployment time.

As reported in #696, the Python version in the environment was set to v3.8,
which resulted in a broken installation process too.

**What?**

* Updated the documentation to indicate that `jq` needs to be available too.
* Added checks in the build process to exit and warn the user about missing
  dependencies.
* Updated the version number of the Makefile.
sbkok added a commit to sbkok/aws-deployment-framework that referenced this issue May 16, 2024
…wslabs#702)

**Why?**

When running the make / install process of ADF, we need to ensure it will not
continue if some of the tools are not installed yet.

For example, in the past, if you did not install jq, it would still continue
with the build process. Resulting in a broken template at deployment time.

As reported in awslabs#696, the Python version in the environment was set to v3.8,
which resulted in a broken installation process too.

**What?**

* Updated the documentation to indicate that `jq` needs to be available too.
* Added checks in the build process to exit and warn the user about missing
  dependencies.
* Updated the version number of the Makefile.
sbkok added a commit to sbkok/aws-deployment-framework that referenced this issue May 17, 2024
The upcoming v4 release will introduce breaking changes. As always, it is
recommended to thoroughly review and test the upgrade procedure in a
non-production environment.

Unfortunately, a few issues popped up in ADF v3.2.0 that block a clean
installation in a new environment. Making it harder to test the upgrade
process. This release resolves those issues and includes an updated installer
for ADF to simplify installation.

We hope this shortens the time required to prepare for the v4 upgrade.

---

**Fixes 🐞**

* Fix management account config alias through ADF account management (awslabs#596) by @sbkok.
* Fix CodeBuild stage naming bug (awslabs#628) by @pozeus.
* Fix Jinja2 template rendering with autoescape enabled (awslabs#690) by @sujay0412.
* Fix missing deployment_account_id and initial deployment global IAM bootstrap
  (awslabs#686) by @sbkok, resolves awslabs#594 and awslabs#659.
* Fix permissions to enable delete default VPC in management account (awslabs#699) by @sbkok.
* Fix tagging of Cross Account Access role in the management account (awslabs#700) by @sbkok.
* Fix CloudFormation cross-region changeset approval (awslabs#701) by @sbkok.
* Fix clean bootstrap of the deployment account (awslabs#703) by @sbkok, resolves awslabs#696.
* Bump Jinja2 from 3.1.3 to 3.1.4 (awslabs#720 and awslabs#721) by @dependabot.
* Fix account management lambdas in v3.2 (awslabs#729) by @sbkok.

---

**Installation enhancements 🚀**

This release is the first release with the new installation process baked in.
Please read the [Installation Guide](https://github.com/awslabs/aws-deployment-framework/blob/make/latest/docs/installation-guide.md)
how to install ADF. In case you are upgrading, please follow [the admin guide
on updating ADF](https://github.com/awslabs/aws-deployment-framework/blob/make/latest/docs/admin-guide.md#updating-between-versions)
instead.

Changes baked into this release to support the new installation process:

* New install process (awslabs#677) by @sbkok.
* Install: Add checks to ensure installer dependencies are available (awslabs#702) by
  @sbkok.
* Install: Add version checks and pre-deploy warnings (awslabs#726) by @sbkok.
* Ensure tox fails at first pytest failure (awslabs#686) by @sbkok.
@sbkok sbkok mentioned this issue May 17, 2024
sbkok added a commit to sbkok/aws-deployment-framework that referenced this issue May 23, 2024
The upcoming v4 release will introduce breaking changes. As always, it is
recommended to thoroughly review and test the upgrade procedure in a
non-production environment.

Unfortunately, a few issues popped up in ADF v3.2.0 that block a clean
installation in a new environment. Making it harder to test the upgrade
process. This release resolves those issues and includes an updated installer
for ADF to simplify installation.

We hope this shortens the time required to prepare for the v4 upgrade.

---

**Fixes 🐞**

* Fix management account config alias through ADF account management (awslabs#596) by @sbkok.
* Fix CodeBuild stage naming bug (awslabs#628) by @pozeus.
* Fix Jinja2 template rendering with autoescape enabled (awslabs#690) by @sujay0412.
* Fix missing deployment_account_id and initial deployment global IAM bootstrap
  (awslabs#686) by @sbkok, resolves awslabs#594 and awslabs#659.
* Fix permissions to enable delete default VPC in management account (awslabs#699) by @sbkok.
* Fix tagging of Cross Account Access role in the management account (awslabs#700) by @sbkok.
* Fix CloudFormation cross-region changeset approval (awslabs#701) by @sbkok.
* Fix clean bootstrap of the deployment account (awslabs#703) by @sbkok, resolves awslabs#696.
* Bump Jinja2 from 3.1.3 to 3.1.4 (awslabs#720 and awslabs#721) by @dependabot.
* Fix account management lambdas in v3.2 (awslabs#729) by @sbkok.

---

**Installation enhancements 🚀**

This release is the first release with the new installation process baked in.
Please read the [Installation Guide](https://github.com/awslabs/aws-deployment-framework/blob/make/latest/docs/installation-guide.md)
how to install ADF. In case you are upgrading, please follow [the admin guide
on updating ADF](https://github.com/awslabs/aws-deployment-framework/blob/make/latest/docs/admin-guide.md#updating-between-versions)
instead.

Changes baked into this release to support the new installation process:

* New install process (awslabs#677) by @sbkok.
* Install: Add checks to ensure installer dependencies are available (awslabs#702) by
  @sbkok.
* Install: Add version checks and pre-deploy warnings (awslabs#726) by @sbkok.
* Ensure tox fails at first pytest failure (awslabs#686) by @sbkok.
sbkok added a commit to sbkok/aws-deployment-framework that referenced this issue May 24, 2024
The upcoming v4 release will introduce breaking changes. As always, it is
recommended to thoroughly review and test the upgrade procedure in a
non-production environment.

Unfortunately, a few issues popped up in ADF v3.2.0 that block a clean
installation in a new environment. Making it harder to test the upgrade
process. This release resolves those issues and includes an updated installer
for ADF to simplify installation.

We hope this shortens the time required to prepare for the v4 upgrade.

---

**Fixes 🐞**

* Fix management account config alias through ADF account management (awslabs#596) by @sbkok.
* Fix CodeBuild stage naming bug (awslabs#628) by @pozeus.
* Fix Jinja2 template rendering with autoescape enabled (awslabs#690) by @sujay0412.
* Fix missing deployment_account_id and initial deployment global IAM bootstrap
  (awslabs#686) by @sbkok, resolves awslabs#594 and awslabs#659.
* Fix permissions to enable delete default VPC in management account (awslabs#699) by @sbkok.
* Fix tagging of Cross Account Access role in the management account (awslabs#700) by @sbkok.
* Fix CloudFormation cross-region changeset approval (awslabs#701) by @sbkok.
* Fix clean bootstrap of the deployment account (awslabs#703) by @sbkok, resolves awslabs#696.
* Bump Jinja2 from 3.1.3 to 3.1.4 (awslabs#720 and awslabs#721) by @dependabot.
* Fix account management lambdas in v3.2 (awslabs#729) by @sbkok.
* Fix management account missing required IAM Tag Role permission in v3.2
  (awslabs#729) by @sbkok.

---

**Installation enhancements 🚀**

This release is the first release with the new installation process baked in.
Please read the [Installation Guide](https://github.com/awslabs/aws-deployment-framework/blob/make/latest/docs/installation-guide.md)
how to install ADF. In case you are upgrading, please follow [the admin guide
on updating ADF](https://github.com/awslabs/aws-deployment-framework/blob/make/latest/docs/admin-guide.md#updating-between-versions)
instead.

Changes baked into this release to support the new installation process:

* New install process (awslabs#677) by @sbkok.
* Ensure tox fails at first pytest failure (awslabs#686) by @sbkok.
* Install: Add checks to ensure installer dependencies are available (awslabs#702) by
  @sbkok.
* Install: Add version checks and pre-deploy warnings (awslabs#726) by @sbkok.
* Install: Add uncommitted changes check (awslabs#733)
sbkok added a commit to sbkok/aws-deployment-framework that referenced this issue May 24, 2024
The upcoming v4 release will introduce breaking changes. As always, it is
recommended to thoroughly review and test the upgrade procedure in a
non-production environment.

Unfortunately, a few issues popped up in ADF v3.2.0 that block a clean
installation in a new environment. Making it harder to test the upgrade
process. This release resolves those issues and includes an updated installer
for ADF to simplify installation.

We hope this shortens the time required to prepare for the v4 upgrade.

---

**Fixes 🐞**

* Fix management account config alias through ADF account management (awslabs#596) by @sbkok.
* Fix CodeBuild stage naming bug (awslabs#628) by @pozeus.
* Fix Jinja2 template rendering with autoescape enabled (awslabs#690) by @sujay0412.
* Fix missing deployment_account_id and initial deployment global IAM bootstrap
  (awslabs#686) by @sbkok, resolves awslabs#594 and awslabs#659.
* Fix permissions to enable delete default VPC in management account (awslabs#699) by @sbkok.
* Fix tagging of Cross Account Access role in the management account (awslabs#700) by @sbkok.
* Fix CloudFormation cross-region changeset approval (awslabs#701) by @sbkok.
* Fix clean bootstrap of the deployment account (awslabs#703) by @sbkok, resolves awslabs#696.
* Bump Jinja2 from 3.1.3 to 3.1.4 (awslabs#720 and awslabs#721) by @dependabot.
* Fix account management lambdas in v3.2 (awslabs#729) by @sbkok.
* Fix management account missing required IAM Tag Role permission in v3.2
  (awslabs#729) by @sbkok.

---

**Installation enhancements 🚀**

This release is the first release with the new installation process baked in.
Please read the [Installation Guide](https://github.com/awslabs/aws-deployment-framework/blob/make/latest/docs/installation-guide.md)
how to install ADF. In case you are upgrading, please follow [the admin guide
on updating ADF](https://github.com/awslabs/aws-deployment-framework/blob/make/latest/docs/admin-guide.md#updating-between-versions)
instead.

Changes baked into this release to support the new installation process:

* New install process (awslabs#677) by @sbkok.
* Ensure tox fails at first pytest failure (awslabs#686) by @sbkok.
* Install: Add checks to ensure installer dependencies are available (awslabs#702) by
  @sbkok.
* Install: Add version checks and pre-deploy warnings (awslabs#726) by @sbkok.
* Install: Add uncommitted changes check (awslabs#733)
sbkok added a commit that referenced this issue May 27, 2024
)

**Why?**

When running the make / install process of ADF, we need to ensure it will not
continue if some of the tools are not installed yet.

For example, in the past, if you did not install jq, it would still continue
with the build process. Resulting in a broken template at deployment time.

As reported in #696, the Python version in the environment was set to v3.8,
which resulted in a broken installation process too.

**What?**

* Updated the documentation to indicate that `jq` needs to be available too.
* Added checks in the build process to exit and warn the user about missing
  dependencies.
* Updated the version number of the Makefile.
sbkok added a commit that referenced this issue May 27, 2024
The upcoming v4 release will introduce breaking changes. As always, it is
recommended to thoroughly review and test the upgrade procedure in a
non-production environment.

Unfortunately, a few issues popped up in ADF v3.2.0 that block a clean
installation in a new environment. Making it harder to test the upgrade
process. This release resolves those issues and includes an updated installer
for ADF to simplify installation.

We hope this shortens the time required to prepare for the v4 upgrade.

---

**Fixes 🐞**

* Fix management account config alias through ADF account management (#596) by @sbkok.
* Fix CodeBuild stage naming bug (#628) by @pozeus.
* Fix Jinja2 template rendering with autoescape enabled (#690) by @sujay0412.
* Fix missing deployment_account_id and initial deployment global IAM bootstrap
  (#686) by @sbkok, resolves #594 and #659.
* Fix permissions to enable delete default VPC in management account (#699) by @sbkok.
* Fix tagging of Cross Account Access role in the management account (#700) by @sbkok.
* Fix CloudFormation cross-region changeset approval (#701) by @sbkok.
* Fix clean bootstrap of the deployment account (#703) by @sbkok, resolves #696.
* Bump Jinja2 from 3.1.3 to 3.1.4 (#720 and #721) by @dependabot.
* Fix account management lambdas in v3.2 (#729) by @sbkok.
* Fix management account missing required IAM Tag Role permission in v3.2
  (#729) by @sbkok.

---

**Installation enhancements 🚀**

This release is the first release with the new installation process baked in.
Please read the [Installation Guide](https://github.com/awslabs/aws-deployment-framework/blob/make/latest/docs/installation-guide.md)
how to install ADF. In case you are upgrading, please follow [the admin guide
on updating ADF](https://github.com/awslabs/aws-deployment-framework/blob/make/latest/docs/admin-guide.md#updating-between-versions)
instead.

Changes baked into this release to support the new installation process:

* New install process (#677) by @sbkok.
* Ensure tox fails at first pytest failure (#686) by @sbkok.
* Install: Add checks to ensure installer dependencies are available (#702) by
  @sbkok.
* Install: Add version checks and pre-deploy warnings (#726) by @sbkok.
* Install: Add uncommitted changes check (#733)
@sbkok
Copy link
Collaborator

sbkok commented May 27, 2024

Thank you for your patience. I am happy to inform you that this issue has been resolved in our latest release v3.2.1 just now.
Please open a new issue if you are experiencing any issues with the latest release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants