comparison.rst

API Comparisons

Comparison between Ray Core APIs and Workflows

Ray Workflows is built on top of Ray, and offers a mostly consistent subset of its API while providing durability. This section highlights some of the differences:

func.remote vs func.bind

With Ray tasks, func.remote will submit a remote task to run eagerly; func.bind will generate a node in a DAG, it will not be executed until the DAG is been executed.

Under the context of Ray Workflow, the execution of the DAG is deferred until workflow.run(dag, workflow_id=...) or workflow.run_async(dag, workflow_id=...) is called on the DAG. Specifying the workflow id allows for resuming of the workflow by its id in case of cluster failure.

Other Workflow Engines

Note: these comparisons are inspired by the Serverless workflows comparisons repo.

Argo API Comparison

The original source of these comparisons can be found here.

Conditionals

../../../python/ray/workflow/examples/comparisons/argo/conditionals_argo.yaml

../../../python/ray/workflow/examples/comparisons/argo/conditionals_workflow.py

DAG

../../../python/ray/workflow/examples/comparisons/argo/dag_argo.yaml

../../../python/ray/workflow/examples/comparisons/argo/dag_workflow.py

Multi-step Workflow

../../../python/ray/workflow/examples/comparisons/argo/multi_step_argo.yaml

../../../python/ray/workflow/examples/comparisons/argo/multi_step_workflow.py

Exit Handler

../../../python/ray/workflow/examples/comparisons/argo/exit_handler_argo.yaml

../../../python/ray/workflow/examples/comparisons/argo/exit_handler_workflow.py

Loops

../../../python/ray/workflow/examples/comparisons/argo/loops_argo.yaml

../../../python/ray/workflow/examples/comparisons/argo/loops_workflow.py

Recursion

../../../python/ray/workflow/examples/comparisons/argo/recursion_argo.yaml

../../../python/ray/workflow/examples/comparisons/argo/recursion_workflow.py

Retries

../../../python/ray/workflow/examples/comparisons/argo/retry_argo.yaml

../../../python/ray/workflow/examples/comparisons/argo/retry_workflow.py

Metaflow API Comparison

The original source of these comparisons can be found here.

Foreach

../../../python/ray/workflow/examples/comparisons/metaflow/foreach_metaflow.py.txt

../../../python/ray/workflow/examples/comparisons/metaflow/foreach_workflow.py

Cadence API Comparison

The original source of these comparisons can be found here.

Sub Workflows

../../../python/ray/workflow/examples/comparisons/cadence/sub_workflow_cadence.java

../../../python/ray/workflow/examples/comparisons/cadence/sub_workflow_workflow.py

File Processing

../../../python/ray/workflow/examples/comparisons/cadence/file_processing_cadence.java

../../../python/ray/workflow/examples/comparisons/cadence/file_processing_workflow.py

Trip Booking

../../../python/ray/workflow/examples/comparisons/cadence/trip_booking_cadence.java

../../../python/ray/workflow/examples/comparisons/cadence/trip_booking_workflow.py

Google Cloud Workflows API Comparison

The original source of these comparisons can be found here.

Data Conditional

../../../python/ray/workflow/examples/comparisons/google_cloud_workflows/data_cond_google.yaml

../../../python/ray/workflow/examples/comparisons/google_cloud_workflows/data_cond_workflow.py

Concat Array

../../../python/ray/workflow/examples/comparisons/google_cloud_workflows/concat_array_google.yaml

../../../python/ray/workflow/examples/comparisons/google_cloud_workflows/concat_array_workflow.py

Sub Workflows

../../../python/ray/workflow/examples/comparisons/google_cloud_workflows/sub_workflows_google.yaml

../../../python/ray/workflow/examples/comparisons/google_cloud_workflows/sub_workflows_workflow.py

Prefect API Comparison

The original source of these comparisons can be found here.

Looping

../../../python/ray/workflow/examples/comparisons/prefect/compute_fib_prefect.py.txt

../../../python/ray/workflow/examples/comparisons/prefect/compute_fib_workflow.py

AirFlow API Comparison

The original source of these comparisons can be found here.

ETL Workflow

../../../python/ray/workflow/examples/comparisons/airflow/etl_airflow.py.txt

../../../python/ray/workflow/examples/comparisons/airflow/etl_workflow.py