(stepfunctions): Problem with Choice and afterwards() with loop in state machine

[edisongustavo]

❓ General Issue

The Question

I've tried to create a loop in a State Machine and I'm getting the following error:

Error: State 'Get Job status' already has a next state

I have reproduced the problem by modifying the example from the stepfunctions readme (converted to Python):

    def _create_state_machine2(self):
        submitJob = Pass(self, "Submit Job")
        waitX = Wait(
            self, "Wait X seconds", time=WaitTime.seconds_path("$.waitSeconds")
        )
        getStatus = Pass(self, "Get Job status")
        registerJobStatus = Succeed(self, "Register Final Job Status")
        definition = (
            submitJob
            #
            .next(waitX)
            #
            .next(getStatus)
            #
            .next(
                Choice(self, "Job Complete?")
                .when(
                    Condition.string_equals("$.status", "FAILED"),
                    Fail(self, "Job Failed"),
                )
                .when(
                    Condition.string_equals("$.status", "SUCCEEDED"),
                    Pass(self, "Job succeeded"),
                )
                .when(Condition.string_equals("$.status", "RUNNING"), waitX)
                .afterwards()
            )
            #
            .next(registerJobStatus)
        )
        StateMachine(
            self, "StateMachine", definition=definition, timeout=Duration.days(7)
        )

From what I can see, the problem is that there is a loop together with usage of afterwards():

waitX -> Choice -> waitX

I can make the problem go away by not using afterwards() and adding the rest of the transitions inside the Condition:

        definition = (
            submitJob
            #
            .next(waitX)
            #
            .next(getStatus)
            #
            .next(
                Choice(self, "Job Complete?")
                .when(
                    Condition.string_equals("$.status", "FAILED"),
                    Fail(self, "Job Failed"),
                )
                .when(
                    Condition.string_equals("$.status", "SUCCEEDED"),
                    registerJobStatus,
                )
                .when(Condition.string_equals("$.status", "RUNNING"), waitX)
            )
        )

This is not ideal because it will really hurt the readability of the code defining the state machine because I can't read it "top to bottom", instead I have to read it by inspecting each Choice.

In my real-life example I will have more loops, so that's important.

Environment

• CDK CLI Version: 1.45.0 (build 0cfab15)
• Module Version: 1.45.0
• Node.js Version: v14.2.0
• OS: Ubuntu 18.04
• Language (Version): 3.7.4

Other information

I found this very similar question but it got closed for staleness.

A longer stack trace:

  Error: State 'Get Job status' already has a next state
      at Pass.makeNext (/tmp/jsii-kernel-58Nz7s/node_modules/@aws-cdk/aws-stepfunctions/lib/states/state.js:172:19)
      at Pass.next (/tmp/jsii-kernel-58Nz7s/node_modules/@aws-cdk/aws-stepfunctions/lib/states/pass.js:65:15)
      at Chain.next (/tmp/jsii-kernel-58Nz7s/node_modules/@aws-cdk/aws-stepfunctions/lib/chain.js:43:22)

[shivlaks]

hey @edisongustavo - what's the desired state machine flow you're trying to get to. What would be ideal authoring workflow?

Would it be possible to share a visual representation / ASL definition of the desired state machine?

I want to make sure I understand your readability issue first before I think of suggestions or potential improvement(s) on the CDK side of things.

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[l0b0]

I'm struggling with the same issue. Here's a specific use case:

dataset_version_creation_definition = (
    check_stac_metadata_task.lambda_invoke.next(content_iterator_task.lambda_invoke)
    .next(
        aws_stepfunctions.Choice(  # type: ignore[arg-type]
            self, "check_files_checksums_maybe_array"
        )
        .when(
            aws_stepfunctions.Condition.number_equals("$.content.iteration_size", 1),
            check_files_checksums_single_task.batch_submit_job,
        )
        .otherwise(check_files_checksums_array_task.batch_submit_job)
        .afterwards()
    )
    .next(
        aws_stepfunctions.Choice(self, "content_iteration_finished")
        .when(
            aws_stepfunctions.Condition.number_equals("$.content.next_item", -1),
            validation_summary_task.lambda_invoke.next(
                aws_stepfunctions.Choice(  # type: ignore[arg-type]
                    self, "validation_successful"
                )
                .when(
                    aws_stepfunctions.Condition.boolean_equals(
                        "$.validation.success", True
                    ),
                    import_dataset_task.lambda_invoke.next(
                        success_task  # type: ignore[arg-type]
                    ),
                )
                .otherwise(validation_failure_lambda_invoke)
            ),
        )
        .otherwise(content_iterator_task.lambda_invoke)
    )
)

I'd like to write this in a flat structure, like this:

dataset_version_creation_definition = (
    check_stac_metadata_task.lambda_invoke.next(content_iterator_task.lambda_invoke)
    .next(
        aws_stepfunctions.Choice(  # type: ignore[arg-type]
            self, "check_files_checksums_maybe_array"
        )
        .when(
            aws_stepfunctions.Condition.number_equals("$.content.iteration_size", 1),
            check_files_checksums_single_task.batch_submit_job,
        )
        .otherwise(check_files_checksums_array_task.batch_submit_job)
        .afterwards()
    )
    .next(
        aws_stepfunctions.Choice(self, "content_iteration_finished")
        .when(
            aws_stepfunctions.Condition.number_equals("$.content.next_item", -1),
            validation_summary_task.lambda_invoke,
        )
        .otherwise(content_iterator_task.lambda_invoke)
        .afterwards()
    )
    .next(
        aws_stepfunctions.Choice(self, "validation_successful")  # type: ignore[arg-type]
        .when(
            aws_stepfunctions.Condition.boolean_equals("$.validation.success", True),
            import_dataset_task.lambda_invoke,
        )
        .otherwise(validation_failure_lambda_invoke)
        .afterwards()
    )
    .next(success_task)  # type: ignore[arg-type]
)

This puts all the next calls on the same level, making the code easier to read (at least for me), but runs into the same problem as @edisongustavo with the "already has a next state" error.

[MrPeker]

I have the same issue as well, any update?

I resolved the issue after removing the next state of Choice in the default chain

[nebur395]

Hi there,

I have kinda the same use case and issue. @edisongustavo have you found a solution?

Since this issue has been closed for a while, I've created this new one. @shivlaks please, find there a more detailed explanation and a visual representation

Thanks a lot!!
best regards