Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(assets): Add deploy-time content hash #2334

Merged
merged 14 commits into from
May 23, 2019

Conversation

RomainMuller
Copy link
Contributor

@RomainMuller RomainMuller commented Apr 19, 2019

Introduces an IAsset interface that centralizes common aspects about
assets, such as the sourceHash and bundleHash properties.

The sourceHash fingerprints the objects that are used as the source
for the asset bundling logic, and is available at synthesis time (it can
for example be injected in construct IDs when it one wants to ensure a
new logical ID is issued for every new version of the asset).

The bundleHash fingerprints the result of the bundling logic, and is
more accurate than sourceHash (in that, if the same source can produce
different artifacts at different points in time, the sourceHash will
remain un-changed, but the bundleHash will change. The bundleHash is
however a deploy-time value and thus cannot be used in construct IDs.

Fixes #1400


Pull Request Checklist

  • Testing
    • Unit test added (prefer not to modify an existing test, otherwise, it's probably a breaking change)
    • CLI change?: coordinate update of integration tests with team
    • cdk-init template change?: coordinated update of integration tests with team
  • Docs
    • jsdocs: All public APIs documented
    • README: README and/or documentation topic updated
  • Title and Description
    • Change type: title prefixed with fix, feat will appear in changelog
    • Title: use lower-case and doesn't end with a period
    • Breaking?: last paragraph: "BREAKING CHANGE: <describe what changed + link for details>"
    • Issues: Indicate issues fixed via: "Fixes #xxx" or "Closes #xxx"
  • Sensitive Modules (requires 2 PR approvers)
    • IAM Policy Document (in @aws-cdk/aws-iam)
    • EC2 Security Groups and ACLs (in @aws-cdk/aws-ec2)
    • Grant APIs (only if not based on official documentation with a reference)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.

Introduces an `IAsset` interface that centralizes common aspects about
assets, such as the `sourceHash` and `bundleHash` properties.

The `sourceHash` fingerprints the objects that are used as the source
for the asset bundling logic, and is available at synthesis time (it can
for example be injected in construct IDs when it one wants to ensure a
new logical ID is issued for every new version of the asset).

The `bundleHash` fingerprints the result of the bundling logic, and is
more accurate than `sourceHash` (in that, if the same source can produce
different artifacts at different points in time, the `sourceHash` will
remain un-changed, but the `bundleHash` will change. The `bundleHash` is
however a deploy-time value and thus cannot be used in construct IDs.

Fixes #1400
@RomainMuller RomainMuller requested review from skinny85, SoManyHs and a team as code owners April 19, 2019 16:45
packages/@aws-cdk/assets/lib/asset.ts Outdated Show resolved Hide resolved
packages/@aws-cdk/assets/lib/asset.ts Outdated Show resolved Hide resolved
packages/@aws-cdk/assets/lib/asset.ts Outdated Show resolved Hide resolved
packages/@aws-cdk/assets/lib/asset.ts Outdated Show resolved Hide resolved
packages/aws-cdk/bin/cdk.ts Outdated Show resolved Hide resolved
packages/aws-cdk/lib/cdk-toolkit.ts Outdated Show resolved Hide resolved
packages/aws-cdk/lib/docker.ts Outdated Show resolved Hide resolved
}
const rootDirectory = fs.statSync(fileOrDirectory).isDirectory()
? fileOrDirectory
: path.dirname(fileOrDirectory);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing tests for all the new options

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I know, I have not introduced any new options 😅 but yes, some of that surface is untested. I'll se if I can get around to fixing that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ability to exclude files has been added (previously these options were ignored)

@@ -692,13 +692,13 @@
"RepositoryName"
]
},
":",
"@sha256:",
Copy link
Contributor

@jogold jogold Apr 23, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mix of Fn::Select and Fn::Join in the Image property looks weird to me (+ it ends up with amazonaws.com in the final string and not AWS::URLSuffix), shouldn't this result in something like:

{
  "Image": {
    "Fn::Join": [
       "",
       [
         { "Ref": "AWS::AccountId" },
         ".dkr.ecr.",
         { "Ref": "AWS::Region" },
         ".",
         { "Ref": "AWS::URLSuffix" },
         "/",
         { "Fn::GetAtt": [ "EventImageAdoptRepositoryDFAAC242", "RepositoryName"] },
         "@sha256:"
         { "Ref": "EventImageImageNameE972A8B1" },
       ] 
    ]
  }
}

or { "Ref": "AdoptEcrRepositorydbc60defc59544bcaa5c28c95d68f62cCodeArtifactHash8BCBAA49" } for the last element

Copy link
Contributor

@jogold jogold Apr 23, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+after pushing, now that the repo digest is used we have the full "resolved URI" (with region, accountid, etc.), is it still needed to use Refs in this case? A single Ref to the parameter should be possible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're passing the repository name, not full URI, in a parameter. This is because the CustomResource that "adopts" the CDK-toolkit-created ECR repository works on the name (for that is the ECR API). I'm torn myself on whether I should cut the chase and just pass the URI around, but it makes parsing the name out of the URI an awkward process (I'd rather recompose the URI than parse it out).

packages/@aws-cdk/assets-docker/lib/image-asset.ts Outdated Show resolved Hide resolved
}
const rootDirectory = fs.statSync(fileOrDirectory).isDirectory()
? fileOrDirectory
: path.dirname(fileOrDirectory);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ability to exclude files has been added (previously these options were ignored)

@@ -53,6 +58,7 @@ export class Staging extends Construct {
super(scope, id);

this.sourcePath = props.sourcePath;
this.sourceHash = fingerprint(this.sourcePath);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should expose the copy options (exclude, symlinks), but this can be in a subsequent PR.

@NGL321 NGL321 added the contribution/core This is a PR that came from AWS. label Sep 27, 2019
This was referenced Dec 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Assets: Add property to retrieve content hash
4 participants