feat: deny list

[eladb]

Allow users to specify deny list rules for the construct hub. Rules can match a package name and optionally a specific version. If a package is matched against a rule, the package is ignored by the discovery function, its relevant objects are deleted from the package data bucket and the catalog is created.

new ConstructHub(this, 'ConstructHub', {
  denyList: [
    { package: '@aws-cdk/cdk', reason: 'This package has been deprecated in favor of @aws-cdk/core' },
    { package: 'cdk-foo-bar', version: '0.0.2', reason: 'Dummy package' },
  ],
});

Design

A deny-list.json file is created in a dedicated S3 bucket and contains a hash map from package (or package/vX.Y.Z) to the deny list rule. A DenyListClient is provided with a simple API for consuming the deny list.

The discovery function will skip any packages that match a deny list rule.

An AWS Lambda function called "Prune Handler" is triggered every time the deny list is updated and periodically every 5 minutes (to compensate for race conditions with the ingestion flow). The prune handler iterates over the deny list and lists all the S3 objects in the package data bucket that match the prefix. It queues the matched objects into an SQS queue.

The SQS queue is consumed by another handler called "Prune Delete Handler" which deletes pruned objects and also triggers the catalog builder function so that the catalog will be rebuilt as needed.

We have also added REMOVE_OBJECT notification to the cache invalidation handler to make sure deleted objects are reflected in the edge cache as soon as possible. This required changing the cache invalidation CallerReference to use the Lambda request ID instead of the object ID.

A section was added to the backend dashboard which monitors these resources.

Integration Tests

This commit also adds support for integration tests, which work similarly to AWS CDK integration tests. They are small CDK apps (with the xxx.integ.ts suffix) which can be deployed using yarn integ:xxx:deploy and if their deployment is successful, a snapshot is stored and compared against a synthesized result during tests. To implement integration test assertions, users can implement a trigger.xxx.lambda.ts handlers which can be added to the test app and executed during deployment (see README for more details).

Misc

• Removed the explicit names for the watchful and high-severity dashboards so now they are automatically generated.
• Remove the requirement to specify alarm actions for the monitoring system (to facilitate tests).
• Added a button to access the S3 bucket and the catalog builder function to the "Catalog Overview" section in the backend dashboard.
• Modified the catalog builder function to allow rebuilding the catalog by just invoking the function without an event.

Resolves https://github.com/cdklabs/construct-hub-internal/issues/34
Resolves #66

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[RomainMuller]

Approved as discussed - outstanding minor comments to be handled in a subsequent PR.