"Switch-case" statement (or smoother patterns for conditionally running steps)

[tomasgareau]

Curious to hear anyone's thoughts on some good patterns for more complex conditional behaviour based on the exit code of some Concourse task.

In the simple 0/1 (success/fail) scenario, we're laughing: say I wanted to send a slack alert if a task failed

jobs:
  - name: my-job
    plan:
      - get: repo
        trigger: true
      - task: my-task
        file: repo/ci/tasks/my-task.yml
        on_failure: # ez-pz, very clean
          put: slack
          params:
            text: "Oh no, my-task failed!"

What about a task that has richer exit codes? A concrete example is terraform plan's -detailed-exitcode flag. Terraform will return:

• 0, if no changes are required
• 1, for error
• 2, if changes are required

In this case, I might want to do three different things:

• 0: job succeeds, do nothing
• 1: job fails, send a slack alert
• 2: job succeeds, push the output of the plan task to some storage bucket & send a slack message to ask someone to review it before manually triggering an apply step

I could hack up something specific to this case like so:

jobs:
  - name: my-job
    plan:
      - get: repo
        trigger: true

      - task: terraform-plan
        # this task is configured to return:
        #   * SUCCESS if there is no diff (0) or an error occurred (1)
        #   * FAILURE if there is a diff (2)
        file: repo/ci/tasks/terraform-plan.yml
        vars:
          target: repo/terraform/my-tf-dir

        # there was a diff (2)
        on_failure:
          do:
            - put: bucket
              params:
                file: tfplan/plan.out
            - put: slack
              params:
                text: "terraform plan, pls review"

        # there was either no diff (0) or an error occurred (1)
        # in the no diff case (0), there _will_ be an output plan
        # in the error case (1), there will not be any output
        on_success:
          try:
            task: check-exists
            file: repo/ci/tasks/check-exists.yml
            input_mapping:
              input: tfplan
            vars:
              target: input/plan.out

            # no output, which means an error occurred (1)
            on_failure:
              put: slack
              params:
                text: "error while planning terraform :("

            # output, which means there was no diff (0)
            on_success:
              put: slack
              params:
                text: "hooray, no diff"

but man is that hard to follow, and it might not generalize well (imagine if the "no-diff" case didn't output a file -- I'd need to manually kludge something in in my terraform-plan.yml task)

If I had a case statement, it might look something more like:

jobs:
  - name: my-job
    plan:
      - get: repo
        trigger: true

      - task: terraform-plan
        # this task just exits with the output from:
        #   terraform plan -out=tfplan/plan.out -detailed-exitcode
        file: repo/ci/tasks/terraform-plan.yml
        vars:
          target: repo/terraform/my-tf-dir

        case:
          0: # success, no diff
            put: slack
            params:
              text: "hooray, no diff"
          1: # error occurred while terraform planning
            put: slack
            params:
              text: "error while planning terraform :("
          2: # success, diff detected (apply required)
            do:
              - put: bucket
                params:
                  file: tfplan/plan.out
              - put: slack
                params:
                  text: "terraform plan, pls review"

Ahhhh, now that's a mastapeece: no intermediate tasks, no confusing try trickery, and lots of flexibility in reacting differently to different outputs.

That said, I have not given much thought here to whether this is in line with the Concourse philosophy or whether it's a big footgun for other use-cases! Would love to hear if:

• there would be any interest in a feature like this or
• there's an easier/cleaner way to accomplish this with existing features

[allwalte]

Came in here to try to find how to do this exact thing, fwiw

[taylorsilva]

Casing on exit codes is an interesting idea.

One thing I see missing in your final code block is what will the result of the job be when it falls into case 3 (exit code 2)?

In the current world, this would result in a failed build, while your comment indicates it should probably be marked as successful (assuming the remaining steps succeed).

---

My thoughts around scenarios like these have come down to two thoughts:

1. Don't try and fit if-else logic into your pipeline config. It hurts (like you pointed out) and just isn't natural in Concourse pipelines.
2. Make as many things idempotent as possible. It's more work up front but the pay off later is sooooo good.

I've found making the effort to make stuff idempotent is a nicer pay off then trying to jam in if-else logic into my pipelines. This can be hard if the resources you're using aren't idempotent though and throw errors :(

[tomasgareau]

One thing I see missing in your final code block is what will the result of the job be when it falls into case 3 (exit code 2)?

Hmmm good question. I suppose in my head I was picturing exit codes either being handled or bubbled up, e.g.:

• exit 0, 1, or 2: -> result of the job is simply the result of the case handler
• any other code: -> job fails with that exit code

Make as many things as idempotent as possible

Agreed that this is very nice. Terraform is pretty good here (I can idempotently plan & apply a Terraform module -- it will do nothing if there are no changes).

Where I tend to stumble a little more is on the "workflow" side of things (i.e., signaling someone for review if there are changes). It's something I could certainly write an idempotent "helper" task (or job) for but it's additional boilerplate that might not translate to similar scenarios.

[suhlig]

You could wrap the command in a script that writes the exit code to a file (and make the script itself exit 0).
The file would be in one of the task's outputs. The next task(s) read that file and either do nothing (the file contents are not for them), or do whatever needs to be done.

jobs:
  - name: deploy-terraform
    plan:
    - task: terraform-plan
      config:
        platform: linux
        image_resource:
          type: registry-image
          source: { repository: busybox }
        outputs: [ name: terraform-result ]
        run:
          path: sh
          args:
          - -c # note the absence of -e
          - |
            (exit 1) # simulate a command that produces exit code 1
            echo $? > terraform-result/exit-code
    - in_parallel:
      - try: # do not mark the job as failed if one of the task arms fails
          task: case-0
          config:
            platform: linux
            image_resource:
              type: registry-image
              source: { repository: busybox }
            inputs: [ name: terraform-result ]
            params: { code: 0 } # with a parameter, the script itself is always the same
            run:
              path: sh
              args:
              - -ec # here, we want to fail the task if the exit code is non-zero
              - |
                [ "$(cat terraform-result/exit-code)" -eq "$code" ]
          on_success:
            put: telegram-notification
            params:
              chat_id: ((telegram-chat-id))
              text: hooray, no diff
      - try:
          task: case-1
          config:
            platform: linux
            image_resource:
              type: registry-image
              source: { repository: busybox }
            inputs: [ name: terraform-result ]
            params: { code: 1 }
            run:
              path: sh
              args:
              - -ec
              - |
                [ "$(cat terraform-result/exit-code)" -eq "$code" ]
          on_success:
            put: telegram-notification
            params:
              chat_id: ((telegram-chat-id))
              text: error while planning terraform :(
      - try:
          task: case-2
          config:
            platform: linux
            image_resource:
              type: registry-image
              source: { repository: busybox }
            inputs: [ name: terraform-result ]
            params: { code: 2 }
            run:
              path: sh
              args:
              - -ec
              - |
                [ "$(cat terraform-result/exit-code)" -eq "$code" ]
          on_success:
            put: telegram-notification
            params:
              chat_id: ((telegram-chat-id))
              text: terraform plan, pls review

resource_types:
  - name: telegram-notification
    type: registry-image
    source:
      repository: w32blaster/concourse-telegram-notifier

resources:
  - name: telegram-notification
    type: telegram-notification
    source:
      bot_token: ((telegram-token))

The result looks pretty neat in the web UI - there is one task for each arm of the simulated case, and you could even improve on the task names by making them state what the exit code means.

[tomasgareau]

Ooooh that does come out quite clean... I like that the "check case" script would be generic. Shame that you'd still need to wrap the original task but not the end of the world.