Pipeline Fails Due to a Previous Configuration Reference

[joshbfei]

Describe the bug
A previous accounts configuration contained a reference to an OU that was then removed from the configuration. The pipeline fails to build at the prepare stack step with the resulting CloudWatch message:

Provisioning failure error message: InvalidParametersException The parent organizational unit 'OUName (ou-afqi-xxx5xxx9)' is not enrolled in AWS Control Tower.

where 'OUName (ou-afqi-xxx5xxx9)' does not exist in any LZ Accelerator configuration file. All accounts to be created are under different existing OUs registered successfully in control tower.

To Reproduce
Add an OU to the organization config without creating the OU prior.
Add an account to the accounts config that references the OU that does not yet exist.
run the pipeline using this configuration
Add the OU manually using the console and register it in control tower
rerun the pipeline. It will fail with a log message that the parent OU is not registered in Control Tower (even though the OUID number is correct, and control tower shows no issues with the OU. The account is not created.
delete the configuration from the account and organization configs
rerun the pipeline. The same message occurs the the parent OU is not registered in Control Tower

Expected behavior
Expected behavior is that removing the references in the accounts and organization config should remove any artifacts from the pipeline. The pipeline should now run successfully and not try to deploy anything to the previous configuration.

Please complete the following information about the solution:

• Version: [e.g. v1.1.0]
  v1.1.0

• Region: [e.g. us-east-1]

• Was the solution modified from the version published on this repository?

• no

• If the answer to the previous question was yes, are the changes available on GitHub?

• Have you checked your service quotas for the sevices this solution uses?

• Were there any errors in the CloudWatch Logs?
  Full CloudWatch log message:

2022-08-18T22:01:10.162Z 913140eb-f4f7-455f-b024-683254d8af17 INFO {
RequestType: 'Delete',
ServiceToken: 'arn:aws:lambda:us-east-1:1234567891011:function:AWSAccelerator-PrepareSta-CreateCTAccountsCreateCo-jrmzNQRYwAaI',
ResponseURL: 'https://cloudformation-custom-resource-response-useast1.s3.amazonaws.com/arn%3Aaws%3Acloudformation%3Aus-east-1%1234567891011%3Astack/AWSAccelerator-PrepareStack-1234567891011-us-east-1/c4f8e500-1f3f-11ed-8673-0a3a69fb2f09%7CCreateCTAccounts3049A752%7Ca9969771-bdaf-4ef7-9671-1ed7d0b05f66?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20220818T220108Z&X-Amz-SignedHeaders=host&X-Amz-Expires=7200&X-Amz-Credential=AKIA6L7Q4OWTVPX5N4HK%2F20220818%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=7bdc2f2642c72e435def6cc5f00f642150aa2e2ef70550b63b6bfbacd729e718',
StackId: 'arn:aws:cloudformation:us-east-1:1234567891011:stack/AWSAccelerator-PrepareStack-1234567891011-us-east-1/c4f8e500-1f3f-11ed-8673-0a3a69fb2f09',
RequestId: 'a9969771-bdaf-4ef7-9671-1ed7d0b05f66',
LogicalResourceId: 'CreateCTAccounts3049A752',
PhysicalResourceId: '97fc9681-1857-4b29-b43e-dca64893d3b2',
ResourceType: 'Custom::CreateControlTowerAccounts',
ResourceProperties: {
ServiceToken: 'arn:aws:lambda:us-east-1:1234567891011:function:AWSAccelerator-PrepareSta-CreateCTAccountsCreateCo-jrmzNQRYwAaI',
uuid: 'da85c318-59b8-482b-b0fe-53555cad737f'
},
IsComplete: true
}
2022-08-18T22:01:10.524Z 913140eb-f4f7-455f-b024-683254d8af17 INFO getSingleAccount response {"Items":[],"Count":0,"ScannedCount":0}
2022-08-18T22:01:10.649Z 913140eb-f4f7-455f-b024-683254d8af17 INFO Provisioning failure error message: InvalidParametersException The parent organizational unit 'Sandbox (ou-afqi-hatb5wy9)' is not enrolled in AWS Control Tower.
2022-08-18T22:01:10.649Z 913140eb-f4f7-455f-b024-683254d8af17 INFO Control Tower account provisioning failed
2022-08-18T22:01:10.649Z 913140eb-f4f7-455f-b024-683254d8af17 INFO Error: Accounts failed to enroll in Control Tower. Check Service Catalog Console at Runtime.Nr [as handler] (/var/task/index.js:1:17989) at processTicksAndRejections (internal/process/task_queues.js:95:5)
2022-08-18T22:01:10.649Z 913140eb-f4f7-455f-b024-683254d8af17 INFO Create accounts failed. Deleting pending account creation records

[joshbfei]

I have continued to try and troubleshoot this issue by deleting the Prepare Stack and letting it get rebuilt by the pipeline. I have also confirmed that the reference to the OU in question is not listed in the DynamoDB tables that are created in the prepare stack. In addition, adding an account to the accounts-config.yaml resulted in the new account being created, and then the same failure of a reference to a non-existent component.
Any information regarding where the artifact of an account ID could be referenced by the prepare stack, CustomCreateControlTowerAccount resource could be sourced from would be helpful.

[joshbfei]

I was able to track this issue to a service catalog Provisioned Product that was in a error state. Terminating the failed provisioned product allowed the pipeline to continue. What made this difficult was that the error log did not indicate a problem with service catalog. It would be very useful to add the provisioned product ID to the error message in the function provisionSuccess(). The vague terms of "Provisioning failure error message" did not indicate where to look for the problem.