Allow drift detection

[gaffo]

There doesn't see to be a way to enable drift detection in CDK.

[rix0rrr]

Do you mean, start a drift detection action on a stack?

That's right, that feature is not built into the CDK. We could conceivably add a cdk drift command to perform a drift detection cycle, but there would not be a lot of advantage over making the CloudFormation call directly.

[gaffo]

My thought was more to fix the stack or the drift. not sure whta the plans are with aws on drift fixing.

…

On Mon, Mar 4, 2019 at 10:14 AM Rico Huijbers ***@***.***> wrote: Do you mean, start a drift detection action on a stack? That's right, that feature is not built into the CDK. We could conceivably add a cdk drift command to perform a drift detection cycle, but there would not be a lot of advantage over making the CloudFormation call directly. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1723 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAvLM_Kljxcj3bdMbGeKZb5zgvDh0F5ks5vTWKOgaJpZM4ay7fr> .

[rix0rrr]

Please take this up with CloudFormation if you have any requests regarding keeping reality in sync with the template on a continuous basis.

CDK will get your application model to CloudFormation. CloudFormation will get your application model to reality, and do any and all of the work to keep it there.

[bpcrao]

I think this is much needed feature

I wont be able to see the Diff of "AWS INFRA" compared with my "Stack template"
since after CDK deploy some one can modify things from AWS Console

[shivlaks]

@bpcrao As @rix0rrr mentioned earlier, the implementation would be to potentially add a cdk drift command, but it would still ultimately be CloudFormation that will compare templated resources against actual resources.

Although CloudFormation supports drift detection, it's limited to these resources. There is a lot of surface area that is still not covered by CloudFormation drift detection

[Juberstine]

@shivlaks Is correct but I'd still like to see this feature added.

[daisuke-yoshimoto]

@shivlaks @rix0rrr

Are there any updates to this issue? It would be great if it could be completed within the cdk toolchain like cdk diff.

[haimlit]

cdk-drift-monitor, a new repo, tries to solve this issue. Looking forward for feedback and comments!

[awworrell]

Is there an update on this issue? I would say this feature is absolutely vital to have in cdk if it's going to be used as a replacement for products like terraform. In terraform, the command terraform plan shows not only how the environment will be changed based on the changes made in the code but also the drift that exists and what will be changed back to the state of the code. Given cdk doesn't have this feature, it's a massive gap.

In every organization I've worked for, using terraform plan to understand the impacts to a production environment is how one can tell the scope of change and understand any drift. Personally, i think cdk diff should do that naturally. Sure it may need to be coordinated with the cloudformation team and the cloudformation team will need to extend what is supported beyond what already exists at https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resource-import-supported-resources.html.

[awworrell]

cdk-drift-monitor, a new repo, tries to solve this issue. Looking forward for feedback and comments!

It's great to have a separate repo; however, cdk should do this naturally.

[dmschauer]

I agree with others who said that this is a feature in high demand.

Take a look at this thread (one of the top search results for me when googling "terraform or aws cdk). Drift detection is THE main point of critique for people who still prefer Terraform over AWS CDK. Personally I just started out with AWS CDK and like it a lot. I wouldn't like advocating for it at work only for others to shoot down its adoption with the drift detection argument...

In my limited experience AWS CDK saves so much time spent dreadfully on manually specifying permissions that it would be a shame if it wasn't adopted only because this feature needs some alignment with the CloudFormation team.

[maximillianus]

I am quite surprised as well that this drift detection feature does not exist. I kinda expect that cdk diff will do this naturally. Agree with the others that this feature will be quite vital to have in AWS CDK. It's hard to detect changes in our deployed CDK infrastructure.

[github-actions]

This issue has received a significant amount of attention so we are automatically upgrading its priority. A member of the community will see the re-prioritization and provide an update on the issue.

[Naumel]

Hello all,
We have noted the community engagement on this issue and we understand its importance.
Since there are a number of such items, we need to make sure that we’re working on the most important ones.
This means that the issue will be moved into the backlog; it will be checked against the other projects for complexity, risk and effort versus benefit.
We will keep posting updates as they become available!

[pauldraper]

CDK will get your application model to CloudFormation. CloudFormation will get your application model to reality, and do any and all of the work to keep it there.

I've seen CDK interact a lot more than that. You make it seem like a JSON generator.

I'm seen it give progress of CloudFormation applying the template.

[keoma-di]

Glad this is in the backlog now. It's an important feature.

[pauldraper]

Note that Cloudformation (and therefore CDK) has no way of automatically resolving drift.

So this would detect and display the drift, but not be able to anything further. (That simply requires a different tool, e.g. Terraform.)

[misterjacko]

I just had an issue where an ingress rule that was added to a SG by CDK was removed SoMehOw. CDK should be able to let me know that that ingress rule is now missing on the next diff or plan or something.

Suggesting the use of the AWS tool that we are trying to avoid by using CDK is not very customer obsessive.

[tmegow]

Create a lambda function using CDK. Then delete the function. Now cdk diff and cdk deploy will never inform that the function is missing, nor will it be recreated.

[rix0rrr]

The steps to perform drift reconciliation in CloudFormation are as follows:

• Run a drift detection operation
• For all the resources that have drift:
  • Set RemovalPolicy to RETAIN
  • Deploy
  • Remove from template
  • Deploy
  • Correct state in template to match detected drift
  • Run an IMPORT operation on the resources to get them managed by CloudFormation again

The next deployment with the actual desired state (CDK app) will now get rid of the drift.

If there is enough information in the drift report, we could streamline this process by performing these steps on behalf of the user (patching the template and redeploying).

There is a risky bit in between where we dropped the resources and we still need to do the IMPORT: if we fail in between those steps, we leave the user's stack in a state that is probably not very obvious to them how to recover from it.

Complications: changes made by Custom Resources cannot be detected in this way.

[vumdao]

@rix0rrr How is the step Remove from template? Is it done through CFN or CDK code ?

[rix0rrr]

I would recommend you do this by doing surgery on the CFN template. You can also achieve it by editing your CDK app, but you want to restore your CDK app to the original state before doing the final deploy.

[misterjacko]

@rix0rrr are you outlining the drift reconciliation steps as an illustration of just how awkward and user hostile current methods are?

I continue to contend that people use CDK because they don't want to mess about in CloudFormation..

[rix0rrr]

@misterjacko, point well-taken, and I agree that it's awkward.

From our end it's unlikely that we'll have the availability to work on this, so I'm outlining the steps so that a person who is so inclined can think about automating them; perhaps as an experimental CDK CLI feature, perhaps as a standalone tool (because none of these steps are really CDK-specific).

The overall idea seems simple enough, but I'm sure there's a long tail of problems that need to be resolved to make this work well enough for real world uses.

[evgenyka]

We can initiate and display the drift, but the output will be CloudFormation YAML, not CDK code. It's important to note that we cannot automatically generate CDK abstractions from CFN YAML to rectify the stack or the drift, as these are not bidirectional. Therefore, the best-case scenario is that you'll receive the same output as CFN describe-stack-resource-drifts.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[dmschauer]

Question to the community: Since this issue was closed as "completed" I assume that Terraform is the superior tool for AWS users who care about drift detection. Anything wrong with this way of thinking about it? The way I see it the simple use case in #1723 (comment) wasn't resolved or am I missing something?

[misterjacko]

Question to the community: Since this issue was closed as "completed" I assume that Terraform is the superior tool for AWS users who care about drift detection. Anything wrong with this way of thinking about it? The way I see it the simple use case in #1723 (comment) wasn't resolved or am I missing something?

100% agree that if you want declarative IaC CDK is not the right tool.

[sammcj]

It's a real shame to see this closed without any fix.

CDK remains vastly inferior to Terraform when it comes to managing state and state resolution.