Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(aws-stepfunctions-tasks): implement service integration patterns for tasks #3114

Closed
1 of 5 tasks
wqzoww opened this issue Jun 27, 2019 · 7 comments · Fixed by #3115
Closed
1 of 5 tasks

refactor(aws-stepfunctions-tasks): implement service integration patterns for tasks #3114

wqzoww opened this issue Jun 27, 2019 · 7 comments · Fixed by #3115
Labels
@aws-cdk/aws-stepfunctions Related to AWS StepFunctions

Comments

@wqzoww
Copy link
Contributor

wqzoww commented Jun 27, 2019

Note: for support questions, please first reference our documentation, then use Stackoverflow. This repository's issues are intended for feature requests and bug reports.

  • I'm submitting a ...

    • 🪲 bug report
    • 🚀 feature request
    • 📚 construct library gap
    • ☎️ security issue or vulnerability => Please see policy
    • ❓ support request => Please see note at the top of this template.
  • What is the current behavior?
    Today, to define a task state with another aws services, users should specify their service integration pattern with booleans, such as "waitForTaskToken" and "synchronous".

  • What is the expected behavior (or behavior of feature suggested)?
    As users are supposed to choose exactly one pattern among all the existing ones, I would like to introduce a single enum variable to indicate service integration pattern.

  • What is the motivation / use case for changing the behavior or adding this feature?

  1. Instead of dealing with multiple boolean variables and unnecessary validation ("waitForTaskToken" and "synchronous" cannot be both true), users can choose their intended integration pattern with one single variable.
  2. It is easy for CDK to keep pace with Step Functions, if there are new patterns or any pattern changes for existing integrated services.
@wqzoww wqzoww added the needs-triage This issue or PR still needs to be triaged. label Jun 27, 2019
wqzoww added a commit to wqzoww/aws-cdk that referenced this issue Jun 27, 2019
…erns for tasks

Step Functions allows users to call different integrated services in different ways. They are also called service integration patterns, including Request Response, Run a Job and Wait for Callback. Users must choose exactly one of them and specify it in the "Resource" field.

This commit introduces a new member variable, serviceIntegrationPattern, in the interface of properties within each existing integrated service. This helps to avoid using multiple boolean variables in the service such as ECS, which supports different service integration patterns. It is also beneficial for code maintenances: if Step Functions adds new integrated services or updates the existing integration patterns in the future, keeping pace with these changes will be simply updating this variable of enum type.

BREAKING CHANGE: To define a callback task, users should specify "serviceIntegrationPattern: sfn.ServiceIntegrationPattern.WAIT_FOR_CALLBACK" instead of "waitForTaskToken: true". For a sync task, users should use "serviceIntegrationPattern: sfn.ServiceIntegrationPattern.RUN_A_JOB" in the place of "synchronous: true". In addition, this commit enables users to define callback task with ECS.
**@aws-cdk/aws-stepfunctions-tasks**

Closes aws#3114
wqzoww added a commit to wqzoww/aws-cdk that referenced this issue Jun 28, 2019
…erns for tasks

Step Functions allows users to call different integrated services in different ways. They are also called service integration patterns, including Request Response, Run a Job and Wait for Callback. Users must choose exactly one of them and specify it in the "Resource" field.

This commit introduces a new member variable, serviceIntegrationPattern, in the interface of properties within each existing integrated service. This helps to avoid using multiple boolean variables in the service such as ECS, which supports different service integration patterns. It is also beneficial for code maintenances: if Step Functions adds new integrated services or updates the existing integration patterns in the future, keeping pace with these changes will be simply updating this variable of enum type.

BREAKING CHANGE: To define a callback task, users should specify "serviceIntegrationPattern: sfn.ServiceIntegrationPattern.WAIT_FOR_CALLBACK" instead of "waitForTaskToken: true". For a sync task, users should use "serviceIntegrationPattern: sfn.ServiceIntegrationPattern.RUN_A_JOB" in the place of "synchronous: true". In addition, this commit enables users to define callback task with ECS.
**@aws-cdk/aws-stepfunctions-tasks**

Closes aws#3114
wqzoww added a commit to wqzoww/aws-cdk that referenced this issue Jun 28, 2019
…erns for tasks

Step Functions allows users to call different integrated services in different ways. They are also called service integration patterns, including Request Response, Run a Job and Wait for Callback. Users must choose exactly one of them and specify it in the "Resource" field.

This commit introduces a new member variable, serviceIntegrationPattern, in the interface of properties within each existing integrated service. This helps to avoid using multiple boolean variables in the service such as ECS, which supports different service integration patterns. It is also beneficial for code maintenances: if Step Functions adds new integrated services or updates the existing integration patterns in the future, keeping pace with these changes will be simply updating this variable of enum type.

BREAKING CHANGE: To define a callback task, users should specify "serviceIntegrationPattern: sfn.ServiceIntegrationPattern.WAIT_FOR_CALLBACK" instead of "waitForTaskToken: true". For a sync task, users should use "serviceIntegrationPattern: sfn.ServiceIntegrationPattern.RUN_A_JOB" in the place of "synchronous: true". In addition, this commit enables users to define callback task with ECS.
**@aws-cdk/aws-stepfunctions-tasks**

Closes aws#3114
@NGL321 NGL321 added gap @aws-cdk/aws-stepfunctions Related to AWS StepFunctions and removed needs-triage This issue or PR still needs to be triaged. labels Jul 1, 2019
@NGL321
Copy link
Contributor

NGL321 commented Jul 1, 2019

Hi @wqzoww!

Thank you for addressing this, and putting in a PR!

We are working hard to stabilize the CDK APIs and tuning them to meet our consistency guidelines. While we work on getting the APIs aligned with our guidelines, we are pausing work on most community PRs. Please continue to report issues and submit feature requests, of course. We expect to get back to work on community PRs within a few weeks.

We will absolutely take a took at your PR as soon as we are able.

@MichaelHindley
Copy link

Ran into a use case today that requires using Task Tokens, judging by this issue and my inability to find any docs or sourcecode based on the keyword "waitForTaskToken", is it safe to assume this functionality doesn't exist yet in the CDK?

@wqzoww
Copy link
Contributor Author

wqzoww commented Jul 2, 2019

Hi @MichaelHindley,

Which integrated service do you use in your case ?

Today, CDK supports "waitForTaskToken" for Lambda, SNS and SQS with an additional boolean property "waitForTaskToken".

The following link shows how to define a callback task (waitForTaskToken) for Lambda:
https://github.com/awslabs/aws-cdk/blob/cd1b16f42abca205f3ab6ab1fafd92a4d043a3e3/packages/%40aws-cdk/aws-stepfunctions-tasks/test/run-lambda-task.test.ts#L48

Hope my answer helps.

@MichaelHindley
Copy link

@wqzoww I'm using lambda through InvokeFunction and was unaware RunLambdaTask even exists, thank you very much!

This search https://github.com/awslabs/aws-cdk/search?q=waitForTaskToken&unscoped_q=waitForTaskToken was returning 0 results yesterday or I did some major typos, either way this completely solves my use case :)

Just for my own sanity, there is no DynamoDB support for stepfunction tasks yet, right?

@wqzoww
Copy link
Contributor Author

wqzoww commented Jul 3, 2019

@MichaelHindley

I'm using lambda through InvokeFunction and was unaware RunLambdaTask even exists

I updated the README.md file and provided use cases to cover both classes in the corresponding PR of this issue. Hope my PR will be reviewed and accepted soon.

Just for my own sanity, there is no DynamoDB support for stepfunction tasks yet, right?

To confirm, DDB is not supported in stepfunctions-task today.

@lazharichir
Copy link

Hope it will be merged soon as my StepFunctions make big use of DynamoDB/PUT to update the statuses of the current job. Thanks guy – aws-cdk is game-changing!

@MichaelHindley
Copy link

Hope it will be merged soon as my StepFunctions make big use of DynamoDB/PUT to update the statuses of the current job. Thanks guy – aws-cdk is game-changing!

Yep same here, we're currently using a generic lambda that accepts an DDB operation as it's input as a workaround, which gets the job done but I wouldn't want to maintain this in LTS.

wqzoww added a commit to wqzoww/aws-cdk that referenced this issue Aug 7, 2019
…erns for tasks

Step Functions allows users to call different integrated services in different ways. They are also called service integration patterns, including Request Response, Run a Job and Wait for Callback. Users must choose exactly one of them and specify it in the "Resource" field.

This commit introduces a new member variable, serviceIntegrationPattern, in the interface of properties within each existing integrated service. This helps to avoid using multiple boolean variables in the service such as ECS, which supports different service integration patterns. It is also beneficial for code maintenances: if Step Functions adds new integrated services or updates the existing integration patterns in the future, keeping pace with these changes will be simply updating this variable of enum type.

BREAKING CHANGE: To define a callback task, users should specify "serviceIntegrationPattern: sfn.ServiceIntegrationPattern.WAIT_FOR_TASK_TOKEN" instead of "waitForTaskToken: true". For a sync task, users should use "serviceIntegrationPattern: sfn.ServiceIntegrationPattern.SYNC" in the place of "synchronous: true". In addition, this commit enables users to define callback task with ECS.
**@aws-cdk/aws-stepfunctions-tasks**

Closes aws#3114
wqzoww added a commit to wqzoww/aws-cdk that referenced this issue Aug 7, 2019
…erns for tasks

Step Functions allows users to call different integrated services in different ways. They are also called service integration patterns, including Request Response, Run a Job and Wait for Callback. Users must choose exactly one of them and specify it in the "Resource" field.

This commit introduces a new member variable, serviceIntegrationPattern, in the interface of properties within each existing integrated service. This helps to avoid using multiple boolean variables in the service such as ECS, which supports different service integration patterns. It is also beneficial for code maintenances: if Step Functions adds new integrated services or updates the existing integration patterns in the future, keeping pace with these changes will be simply updating this variable of enum type.

BREAKING CHANGE: To define a callback task, users should specify "serviceIntegrationPattern: sfn.ServiceIntegrationPattern.WAIT_FOR_TASK_TOKEN" instead of "waitForTaskToken: true". For a sync task, users should use "serviceIntegrationPattern: sfn.ServiceIntegrationPattern.SYNC" in the place of "synchronous: true". In addition, this commit enables users to define callback task with ECS.
**@aws-cdk/aws-stepfunctions-tasks**

Closes aws#3114
@mergify mergify bot closed this as completed in #3115 Aug 8, 2019
mergify bot pushed a commit that referenced this issue Aug 8, 2019
* refactor(aws-stepfunctions-tasks): implement service integration patterns for tasks

Step Functions allows users to call different integrated services in different ways. They are also called service integration patterns, including Request Response, Run a Job and Wait for Callback. Users must choose exactly one of them and specify it in the "Resource" field.

This commit introduces a new member variable, serviceIntegrationPattern, in the interface of properties within each existing integrated service. This helps to avoid using multiple boolean variables in the service such as ECS, which supports different service integration patterns. It is also beneficial for code maintenances: if Step Functions adds new integrated services or updates the existing integration patterns in the future, keeping pace with these changes will be simply updating this variable of enum type.

BREAKING CHANGE: To define a callback task, users should specify "serviceIntegrationPattern: sfn.ServiceIntegrationPattern.WAIT_FOR_TASK_TOKEN" instead of "waitForTaskToken: true". For a sync task, users should use "serviceIntegrationPattern: sfn.ServiceIntegrationPattern.SYNC" in the place of "synchronous: true". In addition, this commit enables users to define callback task with ECS.
**@aws-cdk/aws-stepfunctions-tasks**

Closes #3114

* serviceIntegrationPattern -> integrationPattern
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-stepfunctions Related to AWS StepFunctions
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants