Skip to content

RFC 477: Policy Validation#478

Merged
otaviomacedo merged 26 commits intomasterfrom
otaviom/rfc-0477
Mar 27, 2023
Merged

RFC 477: Policy Validation#478
otaviomacedo merged 26 commits intomasterfrom
otaviom/rfc-0477

Conversation

@otaviomacedo
Copy link
Copy Markdown
Contributor

@otaviomacedo otaviomacedo commented Feb 8, 2023

This is a request for comments about validation of policies by the CDK at synth time. See #477 for additional details.

APIs are signed off by @iliapolo.

Rendered version


By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache-2.0 license

@otaviomacedo otaviomacedo requested a review from iliapolo February 8, 2023 14:02
@iliapolo iliapolo added status/review Proposal pending review/revision and removed status/review Proposal pending review/revision labels Feb 8, 2023
otaviomacedo and others added 2 commits February 16, 2023 11:52
Co-authored-by: Eli Polonsky <epolon@amazon.com>
Copy link
Copy Markdown
Contributor

@iliapolo iliapolo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@otaviomacedo @corymhall I think the major decisions in the RFC are solid. That is:

  1. The decision not to use CloudFormation hooks.
  2. The decision to invoke validations in the framework.

Can we move on to API discussions?

Comment on lines +74 to +78
* Developer writes a CDK application without the correct validations config.
* Developer deploys non-compliant stacks.
* Deployment guardrails catch these violations, and instruct the developer to add a validations property to their
application.
* Developer adds the validations property, and avoids these violations going forward.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is covered? Whaddayatrynasay?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you talking about all the bullet points or just the last one?

Comment on lines +129 to +132
How to fix:
> Using override `app.findChild('my-bucket').addPropertyOverride('SSEAlgorithm', 'aws:kms');`
> Add to construct properties for `cdk-app/MyStack/Bucket`
`encryption: BucketEncryption.KMS`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this, but this needs be propped further up front that there will be hints attached for fixing violations, and that's one of the value adds of this feature!


Recommendation: Missing value for key `SSEAlgorithm` - must specify `aws:kms`
How to fix:
> Using override `app.findChild('my-bucket').addPropertyOverride('SSEAlgorithm', 'aws:kms');`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh my God can we generate the correct code here? That'd be neat!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think we can provide the correct code to use, but we may or may not be able to generate it depending on the plugin/tool. I'm not sold on the escape hatch mechanism here, it would be much better if we could provide instructions on how to update the actual construct in question.

1 thing we could do is we could know whether or not the user is using one of our L2 constructs. If they are then we "know" how to fix all of the issues. It would probably require hand written instructions (unless we had a way to trace a CFN property back to the L2 property).

updating section on exemptions/suppressions

// globally for the entire app (an app is a stage)
const app = new App({
validationPlugins: [
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has a potential to extend into lifecycle phase based plugins (App Lifecycle) Something to the effect of

const app = new App({
  plugins: [{
    phase: "postSynthesis",
    plugin: new CfnGuardValidator(...)
  },...
  ]

corymhall and others added 4 commits March 7, 2023 06:06
Co-authored-by: Eli Polonsky <epolon@amazon.com>
Updated the part about human readable format.

```ts
const app = new App({
context: { '@aws-cdk/core:validationReportJson': true },
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit strange. Normally JSON output is needed in order for a machine to parse the output, but running cdk synth also produces additional stuff that will make the output non parsable. That is, it will contains some lines, then a JSON report, and some more lines?

I think its going to be hard to make sure the CLI doesn't print anything when this feature is turned on, so I would suggest allowing to pass a file where the JSON report will be written to. Is this possible via context?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the report returns anything that means it fails and synth will not write anything to stdout. The only thing written to stdout will be the report.


```ts
const app = new App({
context: { '@aws-cdk/core:validationReportJson': true },
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side note: why @aws-cdk/core? I think we should steer clear of that prefix.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just trying to stay consistent and match the feature flags https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.cx_api-readme.html.

The `validate` method returns an instance of `ValidationPluginReport`, which
tells the CDK whether the template is compliant, which violations were found
(if any), and any metadata about the report. These are the report related
interfaces:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this should be in the appendix or design sections, not the working backwards.

otaviomacedo and others added 2 commits March 16, 2023 11:48
Co-authored-by: Eli Polonsky <epolon@amazon.com>
@otaviomacedo otaviomacedo added the status/api-approved API Bar Raiser signed-off the API of this RFC label Mar 17, 2023
@otaviomacedo otaviomacedo added the status/final-comment-period Pending final approval label Mar 17, 2023
@otaviomacedo otaviomacedo merged commit f498547 into master Mar 27, 2023
@otaviomacedo otaviomacedo deleted the otaviom/rfc-0477 branch March 27, 2023 10:56
@otaviomacedo otaviomacedo added status/implementing RFC is being implemented and removed status/final-comment-period Pending final approval labels Mar 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status/api-approved API Bar Raiser signed-off the API of this RFC status/implementing RFC is being implemented

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants