feat(stepfunctions-tasks): step functions task for cross-region AWS API call #30061

tmokmss · 2024-05-04T14:34:48Z

Issue # (if applicable)

Reason for this change

It would be useful if we could call AWS API across regions from a Step Functions state machine. Currently it is not officially supported even with AWS SDK integration tasks.

Our usecase is to automate a cross-region failover scenario in a multi-region application. This requires you to orchestrate multiple API calls for both active and standby regions (e.g. failover Aurora DB cluster, rewrite AppConfig parameter, etc), and it would be great if we can manage these operations in a single state machine.

Description of changes

This PR adds a new construct CallAwsServiceCrossRegion that deploys 1. a Lambda function to call AWS API in different regions 2. SFn task to call the function.

Because most properties are compatible with the existing CallAwsService construct, you can use the new construct by just adding the region property.

Additionally, it also allows to set endpoint to override AWS API endpoint, because some AWS APIs requires you to override it. (e.g. Route53 ARC)

Description of how you validated changes

Added unit tests and integ tests.

Checklist

My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

…WS API call

aws-cdk-automation

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

...k/custom-resource-handlers/lib/aws-stepfunctions-tasks/cross-region-aws-sdk-handler/index.ts

nmussy

Looks good overall, just some concerns and minor comments. I'm also wondering if it's better for maintainability not to create a new class, and to reuse the existing CallAwsService, and just to add an optional crossRegionTarget prop

...k/custom-resource-handlers/lib/aws-stepfunctions-tasks/cross-region-aws-sdk-handler/index.ts

...mework-integ/test/aws-stepfunctions-tasks/test/lambda/integ.call-aws-service-cross-region.ts

...k/custom-resource-handlers/lib/aws-stepfunctions-tasks/cross-region-aws-sdk-handler/index.ts

nmussy · 2024-05-06T14:31:47Z

...k/custom-resource-handlers/lib/aws-stepfunctions-tasks/cross-region-aws-sdk-handler/index.ts

+
+  try {
+    // esbuild-disable unsupported-require-call -- not esbuildable but that's fine
+    const pkg = require(`@aws-sdk/client-${event.service}`);


Correct me if I'm wrong, but normalizeServiceName is not being ran for your handler? It would allow us to keep the same pattern as other services props, see AwsSdkCall:

aws-cdk/packages/@aws-cdk/aws-custom-resource-sdk-adapter/lib/api-call.ts

Line 65 in fff9cf6

this.service = normalizeServiceName(service);

aws-cdk/packages/@aws-cdk/aws-custom-resource-sdk-adapter/lib/sdk-info.ts

Lines 10 to 15 in fff9cf6

export function normalizeServiceName(service: string) {

service = service.toLowerCase(); // Lowercase

service = service.replace(/^@aws-sdk\/client-/, ''); // Strip the start of a V3 package name

service = v2ToV3Mapping()?.[service] ?? service; // Optionally map v2 name -> v3 name

return service;

}

I'd like to keep this feature and API as simple as possible to lower the maintenance cost. AwsSdkCall code has become such complicated mostly for backward compatibility reason. Since this feature is new, we don't have to introduce the complexity at all.

Also the API is aimed to be compatible with the existing CallAwsService construct (and it should), which also does not support normalizing service names.

I'll let the maintainer weigh in, I think it's a nice piece of QoL, but I understand your maintainability concern

nmussy · 2024-05-06T14:32:06Z

packages/aws-cdk-lib/aws-stepfunctions-tasks/lib/lambda/call-aws-service-cross-region.ts

+ */
+export interface CallAwsServiceCrossRegionProps extends sfn.TaskStateBaseProps {
+  /**
+   * The AWS service to call in AWS SDK for JavaScript v3 style.


See comment about normalizeServiceName

nmussy · 2024-05-06T14:33:53Z

...k/custom-resource-handlers/lib/aws-stepfunctions-tasks/cross-region-aws-sdk-handler/index.ts

+    // esbuild-disable unsupported-require-call -- not esbuildable but that's fine
+    const pkg = require(`@aws-sdk/client-${event.service}`);
+    const Client = findV3ClientConstructor(pkg);
+    const Command = findCommandClass(pkg, event.action);


Similarly, you should be able to use normalizeActionName, see AwsSdkCall:

aws-cdk/packages/@aws-cdk/aws-custom-resource-sdk-adapter/lib/sdk-info.ts

Lines 10 to 16 in fff9cf6

export function normalizeServiceName(service: string) {

service = service.toLowerCase(); // Lowercase

service = service.replace(/^@aws-sdk\/client-/, ''); // Strip the start of a V3 package name

service = v2ToV3Mapping()?.[service] ?? service; // Optionally map v2 name -> v3 name

return service;

}

Same as service name, I'd like to intentionally only support action names in camelCase.

nmussy · 2024-05-06T14:34:08Z

packages/aws-cdk-lib/aws-stepfunctions-tasks/lib/lambda/call-aws-service-cross-region.ts

+  /**
+   * The API action to call.
+   *
+   * Use camelCase.


See comment about normalizeActionName

packages/aws-cdk-lib/aws-stepfunctions-tasks/lib/lambda/call-aws-service-cross-region.ts

tmokmss · 2024-05-07T04:15:09Z

Hi @nmussy, thanks for the comments.

reuse the existing CallAwsService, and just to add an optional crossRegionTarget prop

I considered this approach as well but chose the current approach, because with the current approach we can easily avoid from future breaking changes when SFn will officially start to support cross-region API call; we can just deprecate CallAwsServiceCrossRegion construct and add the official functionality to CallAwsService construct.

nmussy

LGTM, just a couple of pending comments

aws-cdk-automation · 2024-05-14T15:47:41Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
Commit ID: e387716
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

feat(aws-stepfunctions-tasks): Step Functions task for cross-region A…

214f00f

…WS API call

github-actions bot added effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2 star-contributor [Pilot] contributed between 25-49 PRs to the CDK labels May 4, 2024

aws-cdk-automation requested a review from a team May 4, 2024 14:35

aws-cdk-automation previously requested changes May 4, 2024

View reviewed changes

tmokmss changed the title ~~feat(aws-stepfunctions-tasks): Step Functions task for cross-region AWS API call~~ feat(stepfunctions-tasks): step functions task for cross-region AWS API call May 4, 2024

Merge branch 'main' into cross-region-api-sfn

19fbe96

nmussy reviewed May 4, 2024

View reviewed changes

...k/custom-resource-handlers/lib/aws-stepfunctions-tasks/cross-region-aws-sdk-handler/index.ts Show resolved Hide resolved

Update call-aws-service-cross-region.ts

87eb546

aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label May 4, 2024

update comments

0769697

github-actions bot mentioned this pull request May 6, 2024

Weekly PR metrics report #30073

Closed

nmussy suggested changes May 6, 2024

View reviewed changes

aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label May 6, 2024

address review comments

9ec0fa6

nmussy approved these changes May 7, 2024

View reviewed changes

aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label May 7, 2024

update comments

e387716

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(stepfunctions-tasks): step functions task for cross-region AWS API call #30061

feat(stepfunctions-tasks): step functions task for cross-region AWS API call #30061

tmokmss commented May 4, 2024 •

edited

aws-cdk-automation left a comment

nmussy left a comment

nmussy May 6, 2024

tmokmss May 7, 2024 •

edited

nmussy May 7, 2024

nmussy May 6, 2024

nmussy May 6, 2024

tmokmss May 7, 2024

nmussy May 6, 2024

tmokmss commented May 7, 2024 •

edited

nmussy left a comment

aws-cdk-automation commented May 14, 2024

	export function normalizeServiceName(service: string) {
	service = service.toLowerCase(); // Lowercase
	service = service.replace(/^@aws-sdk\/client-/, ''); // Strip the start of a V3 package name
	service = v2ToV3Mapping()?.[service] ?? service; // Optionally map v2 name -> v3 name
	return service;
	}

feat(stepfunctions-tasks): step functions task for cross-region AWS API call #30061

Are you sure you want to change the base?

feat(stepfunctions-tasks): step functions task for cross-region AWS API call #30061

Conversation

tmokmss commented May 4, 2024 • edited

Issue # (if applicable)

Reason for this change

Description of changes

Description of how you validated changes

Checklist

aws-cdk-automation left a comment

Choose a reason for hiding this comment

nmussy left a comment

Choose a reason for hiding this comment

nmussy May 6, 2024

Choose a reason for hiding this comment

tmokmss May 7, 2024 • edited

Choose a reason for hiding this comment

nmussy May 7, 2024

Choose a reason for hiding this comment

nmussy May 6, 2024

Choose a reason for hiding this comment

nmussy May 6, 2024

Choose a reason for hiding this comment

tmokmss May 7, 2024

Choose a reason for hiding this comment

nmussy May 6, 2024

Choose a reason for hiding this comment

tmokmss commented May 7, 2024 • edited

nmussy left a comment

Choose a reason for hiding this comment

aws-cdk-automation commented May 14, 2024

AWS CodeBuild CI Report

tmokmss commented May 4, 2024 •

edited

tmokmss May 7, 2024 •

edited

tmokmss commented May 7, 2024 •

edited