-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
aws-stepfunctions: Distributed Map State Item Processor executionType doesn't recognize ProcessorType.EXPRESS #30194
Comments
Hey @Commas929 , thanks for reporting this bug. On investigating this issue, I checked the CDK Docs and found that this PR introduced the support for itemProcessor() where
However later on this value was overwritten by change introduced in this PR -
So I tried reunning the code with giving value of Sharing the code - const distributedMapState = new DistributedMap(this, 'Distributed Map State', {
maxConcurrency: 1000,
itemReader: new S3CsvItemReader({
bucket: bucket,
key: JsonPath.stringAt('$.s3Key'),
csvHeaders: CsvHeaders.useFirstRow()
}),
mapExecutionType: StateMachineType.EXPRESS,
resultPath: JsonPath.DISCARD,
toleratedFailureCount: 0,
}).itemProcessor(new Pass(this, 'Pass State', {}), {
mode: ProcessorMode.DISTRIBUTED,
executionType: ProcessorType.STANDARD,
})
const fileProcessor = new StateMachine(this, 'File Processor', {
definitionBody: DefinitionBody.fromChainable(distributedMapState),
stateMachineName: 'File-Processor-Sfn',
stateMachineType: StateMachineType.STANDARD,
}) and synthesized template -
You could try using Escape hatches for workaround. Marking the issue as appropriate. |
I encountered this problem before , and I’m working on it. I think the The In the current implementation, the The |
… executionType in the ProcessorConfig (#30301) ### Issue # (if applicable) Closes #30194 ### Reason for this change In #27913, the ItemProcessor was introduced for use with the Map Class. With the executionType in the ProcessorConfig, it was possible to specify the executionType for the Map. On the other hand, in #28821, the DistributedMap Class was introduced. The mapExecutionType of the DistributedMap class always overwrites the executionType of the ProcessorConfig. Therefore, when using the DistributedMap class, the implementation ignores the executionType of the ProcessorConfig. However, this behavior cannot be inferred from the documentation. ### Description of changes * Added to the docs that when using the DistributedMap Class, the executionType in the ProcessorConfig is ignored. * Also added a warning. ### Description of how you validated changes Add unit tests and integ tests ### Checklist - [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
|
… executionType in the ProcessorConfig (aws#30301) ### Issue # (if applicable) Closes aws#30194 ### Reason for this change In aws#27913, the ItemProcessor was introduced for use with the Map Class. With the executionType in the ProcessorConfig, it was possible to specify the executionType for the Map. On the other hand, in aws#28821, the DistributedMap Class was introduced. The mapExecutionType of the DistributedMap class always overwrites the executionType of the ProcessorConfig. Therefore, when using the DistributedMap class, the implementation ignores the executionType of the ProcessorConfig. However, this behavior cannot be inferred from the documentation. ### Description of changes * Added to the docs that when using the DistributedMap Class, the executionType in the ProcessorConfig is ignored. * Also added a warning. ### Description of how you validated changes Add unit tests and integ tests ### Checklist - [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
… executionType in the ProcessorConfig (aws#30301) ### Issue # (if applicable) Closes aws#30194 ### Reason for this change In aws#27913, the ItemProcessor was introduced for use with the Map Class. With the executionType in the ProcessorConfig, it was possible to specify the executionType for the Map. On the other hand, in aws#28821, the DistributedMap Class was introduced. The mapExecutionType of the DistributedMap class always overwrites the executionType of the ProcessorConfig. Therefore, when using the DistributedMap class, the implementation ignores the executionType of the ProcessorConfig. However, this behavior cannot be inferred from the documentation. ### Description of changes * Added to the docs that when using the DistributedMap Class, the executionType in the ProcessorConfig is ignored. * Also added a warning. ### Description of how you validated changes Add unit tests and integ tests ### Checklist - [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one. |
Describe the bug
I have the following piece of CDK code but when I deploy it and check in AWS console, the step function execution type for the map run is
Standard
instead ofExpress
.The code:
The generated State Machine definition copied from AWS console:
Expected Behavior
The ItemProcessor execution type should be Express
Current Behavior
The ItemProcessor execution type is Standard and the specified type in CDK is ignored.
Reproduction Steps
Possible Solution
No response
Additional Information/Context
No response
CDK CLI Version
2.136.0
Framework Version
No response
Node.js Version
NodeJS-18
OS
macOS Sonoma 14.4.1
Language
TypeScript
Language Version
No response
Other information
No response
The text was updated successfully, but these errors were encountered: