How should one work with different runtimes (Docker, Kubernets etc) with run.container task?

[mrsimonemms]

I'm the maintainer of Zigflow, a DSL for Temporal built upon Serverless Workflow.

I want to implement the run.container task, but need to be able to support different container runtimes. As Zigflow may be deployed in many ways (bare metal, VMs, in a containerised environment etc), I want to give users the ability to configure their tasks to target a specific environment. Of course, one can make educated guesses about the environment, but does anyone have any experience with how this may be done?

My plan is:

• if the docker command exists, default to deploying using docker run <image name>
• if certain Kubernetes exist, deploy as a Kubernetes pod
• use metadata to allow specific configuration

This might look like this:

do:
  # No metadata - this will autodetect based on the environment
  - autoDetect: 
      run:
        container: {}
  # This will run using `docker run <image name>`
  - docker:
      metadata:
        runContainer:
          runtime: docker
          # Everything optional
          pullPolicy: Always
          args: # Any custom Docker run args (eg)
            - --rm
            - -it
      run:
        container: {}
  # This will be deployed to Kubernetes
  - kubernetes
      metadata:
        runtime: kubernetes
        # Everything optional
        kubeconfig: /path/to/config/file # For out-of-cluster deployment
        serviceAcccount: <service account token> # For in-cluster deployment
      run:
        container: {}

Thoughts? Does anyone have any experience of this?

Additions to the Serverless Workflow spec

The only thing I'd add to the formal Serverless Workflow spec would be the pullPolicy. This would accept IfNotPresent, Always and Never, defaulting to IfNotPresent (inspired by Kubernetes, but I'm ok with different wordings).

[cdavernas]

@mrsimonemms In Synapse, we deploy to either Docker or Kubernetes depending on the environment the workflow is running in. Personally, I would avoid having separate task or workflow definitions to decide whether something runs on Docker vs Kubernetes. That choice should be an environmental concern, not something the task itself needs to encode.

I would also strongly recommend avoiding any need to “expose” Kubernetes-specific configuration (like kubeconfig paths or service accounts) to the workflow definition. Doing so quickly becomes cumbersome and error-prone, and it leaks implementation details into the workflow layer. This is especially problematic when workflow authors or users may not be deeply familiar with Kubernetes internals.

In my view, workflows should remain as runtime-agnostic as possible, with the execution environment responsible for resolving how and where containers are actually run.

The only thing I'd add to the formal Serverless Workflow spec would be the pullPolicy. This would accept IfNotPresent, Always and Never, defaulting to IfNotPresent (inspired by Kubernetes, but I'm ok with different wordings).

That's a very good idea! Care to open a feature request, and possibly the PR to address it?

[mrsimonemms]

Thanks @cdavernas, good advice. I think you're right about implementation.

Yes, I'll raise a feature request and a PR.

[mrsimonemms]

#1138