Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(rds): new DatabaseInstance.fromLookup #33258

Open
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

pcheungamz
Copy link

@pcheungamz pcheungamz commented Jan 31, 2025

Issue # (if applicable)

Closes #31720

This replaces my previous PR #32901. I addressed the PR comments in this new PR.

This depends on this PR: cdklabs/cloud-assembly-schema#124.

Also depends on this CDK CLI PR: aws/aws-cdk-cli#138. That PR should be merged first and the CLI released, before this PR can be merged.

Reason for this change

Add DatabaseInstance.fromLookup() feature

Description of changes

Describe any new or updated permissions being added

User will need to have permission to run CloudControl API.

Description of how you validated changes

Tested with this code. I already have an RDS DB in my AWS account. I want to look it up and grant connect to a new user.
Saved to packages/@aws-cdk-testing/framework-integ/test/aws-rds/test/my-test-app.ts

import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
import * as rds from 'aws-cdk-lib/aws-rds';

const awsAccountId = 'XXXXXXXXXX79';
const instanceId = 'XXXXXXXXXX-instance-1';

const appWithDb = new cdk.App();
const stack = new cdk.Stack(appWithDb, 'StackWithVpc', {
  env: {
    region: 'us-east-1',
    account: awsAccountId,
  },
});

const dbFromLookup = rds.DatabaseInstance.fromLookup(stack, 'dbFromLookup', {
  instanceIdentifier: instanceId,
});

/* eslint-disable no-console */
console.log('lookup values', dbFromLookup.dbInstanceEndpointAddress, dbFromLookup.dbInstanceEndpointPort);

const consoleReadOnlyRole = new iam.Role(stack, 'TestRole', {
  assumedBy: new iam.ArnPrincipal('arn_for_trusted_principal'),
});
dbFromLookup.grantConnect(consoleReadOnlyRole, 'dbTestUser');

Ran this command:

../../aws-cdk/bin/cdk -a 'npx ts-node test/aws-rds/test/my-test-app.ts' synth 

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@pcheungamz pcheungamz requested a review from a team as a code owner January 31, 2025 14:28
@github-actions github-actions bot added the beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK label Jan 31, 2025
@aws-cdk-automation aws-cdk-automation requested a review from a team January 31, 2025 14:28
@github-actions github-actions bot added effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2 labels Jan 31, 2025
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This review is outdated)

@pcheungamz pcheungamz changed the title [feat] Add aws-rds.DatabaseInstance.fromLookup feat(rds) Add aws-rds.DatabaseInstance.fromLookup Jan 31, 2025
@pcheungamz pcheungamz changed the title feat(rds) Add aws-rds.DatabaseInstance.fromLookup feat Add aws-rds.DatabaseInstance.fromLookup Jan 31, 2025
@aws-cdk-automation
Copy link
Collaborator

This PR cannot be merged because it has conflicts. Please resolve them. The PR will be considered stale and closed if it remains in an unmergeable state.

1 similar comment
@aws-cdk-automation
Copy link
Collaborator

This PR cannot be merged because it has conflicts. Please resolve them. The PR will be considered stale and closed if it remains in an unmergeable state.

Comment on lines 173 to 175
const securityGroups: ec2.ISecurityGroup[] = [];
const dbsg: [string] = instance.DBSecurityGroups;
dbsg.forEach(securityGroupId => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💅 Instead of using forEach for a list.push() side effect, it's nicer to use a list.map() to produce another list.

const securityGroups = instance.DBSecurityGroups.map(id => ec2.SecurityGroup.fromSecurityGroupid(...))

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks. will do

Comment on lines 9 to 14
Object.assign(dataObj, { ['DBInstanceArn']: 'arn:aws:rds:us-east-1:123456789012:db:instance-1' });
Object.assign(dataObj, { ['Endpoint.Address']: 'instance-1.testserver.us-east-1.rds.amazonaws.com' });
Object.assign(dataObj, { ['Endpoint.Port']: '5432' });
Object.assign(dataObj, { ['DbiResourceId']: 'db-ABCDEFGHI' });
Object.assign(dataObj, { ['DBSecurityGroups']: [] });
Object.assign(dataObj, { ['Identifier']: 'instance-1' });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💅 Object.assign() is a way to copy fields from object A to object B, usually useful if you don't know what fields an object might have. Since all values are constants known in advance here, we could also have done:

const dataObj = {
  'DBInstanceArn': 'arn:aws...',
  'Endpoint.Address': 'instance-1.testserver',
  'Endpoint.Port': '5432',
  // ...
};

Which is a lot simpler.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran into a linter error. I guess it is better to turn off linter for this test files than to use Object.assign.

describe('DatabaseInstanceBase from lookup with DBSG', () => {
test('return correct instance info', () => {
const dataObj = {};
Object.assign(dataObj, { ['DBInstanceArn']: 'arn:aws:rds:us-east-1:123456789012:db:instance-1' });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💅 Same comment here.

Comment on lines 67 to 88
function mockDbInstanceContextProviderWith(response: Object, paramValidator?: (options: cxschema.CcApiContextQuery) => void) {
const previous = ContextProvider.getValue;
ContextProvider.getValue = (_scope: Construct, options: GetContextValueOptions) => {
// do some basic sanity checks
expect(options.provider).toEqual(cxschema.ContextProvider.CC_API_PROVIDER);

if (paramValidator) {
paramValidator(options.props as any);
}

return {
value: {
...response,
} as Map<string, Map<string, any>>,
};
};
return previous;
}

function restoreContextProvider(previous: (scope: any, options: GetContextValueOptions) => GetContextValueResult): void {
ContextProvider.getValue = previous;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you copy this from elsewhere? Because using a Jest mock is simpler than this and it's the currently recommended way to write this.

// GIVEN
const mock = jest.spyOn(ContextProvider.prototype, 'getValue').mockResolvedValue(...);

// WHEN
// ...

// THEN
expect(mock).toHaveBeenCalledWith(...);

mock.restore();

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I borrowed it from aws-kms/test/key.from-lookup.test.ts. Will change that to follow the recommended standard.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't get jest.spyOn to work with ContextProvider. I got:

Property `getValue` does not exist in the provided object

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NVM, this is a static method, so I have to use

const mock = jest.spyOn(ContextProvider, 'getValue').mockReturnValue(value);

Comment on lines 93 to 95
const result = await cc.listResources({
TypeName: args.typeName,
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Slightly annoying but I believe that cc.getResource and cc.listResources will return different properties (at the discretion of the resource handler implementation). getResource will definitely return everything, and listResources wil return some summarized version.

That will probably be an annoying an unexpected behavior for users in the future. The safer behavior would be to call listResources first, and then call getResource for every resource returned that way. I'm debating whether we cast that behavior in stone right now, or wait for problems to turn up...

At least this behavior should be documented somewhere.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested with

aws cloudcontrol list-resources --type-name AWS::RDS::DBInstance

and

aws cloudcontrol get-resource --type-name AWS::RDS::DBInstance --identifier shine-reporting-aurora-instance-1

The properties return are the same. getResource and listResources are consistent at least for DBInstance.

const propsObj = getResultObj(propsObject, args.propertiesToReturn);

// Add the identifier back to the propsObj.
propsObj.Identifier = result.ResourceDescription?.Identifier;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make returning Identifier part of getResultObj as well? Then that function is entirely responsible for producing the returned object, which seems nice and comprehensible, and we don't have to remember to do this in both query functions.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sense. will do

Comment on lines 103 to 109
let misMatch = false;
matchKey.forEach((key) => {
const value = findJsonValue(propsObject, key);
if (value !== args.propertyMatch![key]) {
misMatch = true;
}
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be formulated as:

const filters = Object.values(args.propertyMatch);

const isMatch = filters.every(([key, expected] => {
  const actual = findJsonValue(propsObject, key);
  return propertyMatchesFilter(actual, expected);
});

function propertyMatchesFilter(actual: any, expected: any) {
  // For now we just check for strict equality, but we can implement pattern matching and fuzzy matching here later
  return expected === actual;
}

I did 3 things here:

  • Use Object.values() instead of Object.keys(), so we iterate over key/value pairs (saves a lookup and some visual noise, imo).
  • Use list.every() instead of a forEach combined with mutating a boolean
  • Extract the actual comparison to a helper function, so that we have a clear location to start implementing more advanced filtering later

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. will use this pattern.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


// Add the identifier back to the propsObj.
propsObj.Identifier = result.ResourceDescription?.Identifier;
resultObj[CcApiContextProviderPlugin.RESULTS] = [propsObj];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you decided to return a object with a { results: [...] } key, rather than a list directly.

I'm not against it because it's more easily extensible, but I'm not sure I see the need yet and I'm curious if there are any specific reasons to do it this wat? Just returning a direct list seems simpler to me, but I wonder if you're predicting some future change I'm not seeing yet.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was following the Coral results within the return object that I am used to seeing. I am not tied to it. Happy to change it to a direct list, since this is more idiomatic for CDK.

// THEN
const results = result[CcApiContextProviderPlugin.RESULTS];
let propsObj = results[0];
expect(propsObj.DBInstanceArn).toEqual('arn:aws:rds:us-east-1:123456789012:db:test-instance-1');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💅 Could also do:

expect(propsObj).toEqual({
  DBInstanceArn: 'arn:...',
  StorageEncrypted: 'true',
  // ...
});

Or even

expect(propsObj).toEqual(expect.objectContaining({
  DBInstanceArn: 'arn:...',
  StorageEncrypted: 'true',
  // ...
}));

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks. will use this pattern.

propsObj.Identifier = result.ResourceDescription?.Identifier;
resultObj[CcApiContextProviderPlugin.RESULTS] = [propsObj];
} else {
throw new TypeError(`Could not get resource ${args.exactIdentifier}.`);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add in the error message here.

And I don't agree that this is a TypeError.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will use ContextProviderError instead.

throw new TypeError(`Could not get resource ${args.exactIdentifier}.`);
}
} catch (err) {
throw new TypeError(`Encountered CC API error while getting resource ${args.exactIdentifier}.`);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add in the error message here.

And I don't agree that this is a TypeError.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will use ContextProviderError and include err in the message.

@aws-cdk-automation
Copy link
Collaborator

This PR cannot be merged because it has conflicts. Please resolve them. The PR will be considered stale and closed if it remains in an unmergeable state.

1 similar comment
@aws-cdk-automation
Copy link
Collaborator

This PR cannot be merged because it has conflicts. Please resolve them. The PR will be considered stale and closed if it remains in an unmergeable state.

@mrgrain mrgrain changed the title feat Add aws-rds.DatabaseInstance.fromLookup feat(core): Add aws-rds.DatabaseInstance.fromLookup Feb 20, 2025
@mrgrain mrgrain changed the title feat(core): Add aws-rds.DatabaseInstance.fromLookup feat(rds): new DatabaseInstance.fromLookup Feb 20, 2025
@mrgrain mrgrain added the @aws-cdk/core Related to core CDK functionality label Feb 20, 2025
@aws-cdk-automation
Copy link
Collaborator

This PR cannot be merged because it has conflicts. Please resolve them. The PR will be considered stale and closed if it remains in an unmergeable state.

Copy link
Contributor

@rix0rrr rix0rrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more thing!

`LSG-${securityGroupId}`,
securityGroupId,
);
}) as unknown as ec2.ISecurityGroup[];
Copy link
Contributor

@rix0rrr rix0rrr Feb 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cast shouldn't be necessary. Because of the { ... } you added, securityGroups will always turn out to be an array of voids [void, void, void]. The forms for an anonymous function in JavaScript are:

// Form 1: pure expression
const fn1 = (...args) => expr;

// Form 2: statements
const fn2 = (...args) => {
  // potentially more statements
  return expr;
};

You did:

const fn3 = (...args) => {
  // Missing 'return'
  expr;
}

Which is a function that evaluates <expr>, drops the result on the floor, then returns void.

This tells me there's no test for it 🥸

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without the cast, I got this error:

Type 'void[]' is not assignable to type 'ISecurityGroup[]'.
  Type 'void' is not assignable to type 'ISecurityGroup'.ts(2322)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NVM, I understand your comment now. New Code:

    if (dbsg) {
      securityGroups = dbsg.map(securityGroupId => {
        return ec2.SecurityGroup.fromSecurityGroupId(
          scope,
          `LSG-${securityGroupId}`,
          securityGroupId,
        );
      });
    }

I have a test for that here: https://github.com/pcheungamz/aws-cdk/blob/rds-cc-api-2/packages/aws-cdk-lib/aws-rds/test/instance.from-lookup.test.ts#L94.

 expect(instance.connections.securityGroups.length).toEqual(2);

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah ok. I get it now. Will do a new commit to check securityGroupId for each of the two security groups.

Identifier: exactIdentifier,
});
const id = result.ResourceDescription?.Identifier ?? '';
if (id !== '') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💅 I generally prefer early returns, it makes code easier to read because the straight-down code path is the expected code path.

// ❌ Don't
if (goodCondition) {
  doThing();
} else {
  throw new Error('oh no');
}
// ✅ Do
if (badCondition) {
  throw new Error('oh no');
}

doThing();

(Style, not a hard blocker, I'll leave this to you to decide what to do with)

rix0rrr added a commit to aws/aws-cdk-cli that referenced this pull request Feb 25, 2025
A generic Context Provider for CloudControl API. Extracted from
aws/aws-cdk#33258.
github-merge-queue bot pushed a commit to aws/aws-cdk-cli that referenced this pull request Feb 26, 2025
… provider for CloudControl API (#138)

A generic Context Provider for CloudControl API. See
aws/aws-cdk#33258 for an example how to
implement a `fromLookup()` method using the new context provider.

(Extracted from aws/aws-cdk#33258)

---
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache-2.0 license

---------

Signed-off-by: github-actions <github-actions@github.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Momo Kornher <kornherm@amazon.co.uk>
@rix0rrr rix0rrr added the pr/do-not-merge This PR should not be merged at this time. label Feb 27, 2025
rix0rrr
rix0rrr previously approved these changes Feb 27, 2025
Copy link
Contributor

@rix0rrr rix0rrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, but it shouldn't be merged before the matching CLI PR has been released.

@mrgrain
Copy link
Contributor

mrgrain commented Feb 27, 2025

Approved, but it shouldn't be merged before the matching CLI PR has been released.

The matching CLI PR has been released! 🎉
https://github.com/aws/aws-cdk-cli/releases/tag/aws-cdk%40v2.1001.0

@kaizencc kaizencc added the pr-linter/exempt-integ-test The PR linter will not require integ test changes label Mar 7, 2025
@aws-cdk-automation aws-cdk-automation dismissed their stale review March 7, 2025 16:45

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@mergify mergify bot dismissed rix0rrr’s stale review March 7, 2025 16:46

Pull request has been modified.

Copy link

codecov bot commented Mar 7, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.24%. Comparing base (3ed7c4d) to head (b1bb804).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #33258   +/-   ##
=======================================
  Coverage   82.24%   82.24%           
=======================================
  Files         119      119           
  Lines        6875     6875           
  Branches     1161     1161           
=======================================
  Hits         5654     5654           
  Misses       1118     1118           
  Partials      103      103           
Flag Coverage Δ
suite.unit 82.24% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
packages/aws-cdk ∅ <ø> (∅)
packages/aws-cdk-lib/core 82.24% <ø> (ø)
🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: b1bb804
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/core Related to core CDK functionality beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2 pr/do-not-merge This PR should not be merged at this time. pr-linter/exempt-integ-test The PR linter will not require integ test changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

(aws-rds): Add lookup functonality
5 participants