feat: Implement workflow system for multi-step operations

[joshrotenberg]

Summary

Implement a workflow system to handle complex multi-step operations that require multiple API calls, waiting for async operations, and conditional logic.

Motivation

Many common Redis operations require multiple steps with waiting and error handling:

• Initializing an Enterprise cluster (bootstrap → wait → create users → create databases)
• Creating HA databases (create subscription → create database → configure replication → set up monitoring)
• Migrating databases (backup → create new → restore → verify → switch traffic)

Currently users must script these manually or run many commands sequentially.

Implementation Plan

Phase 1: Built-in Workflows (Rust)

Implement core workflows as Rust commands compiled into the binary.

Initial workflows to implement:

• enterprise workflow init-cluster - Bootstrap and configure Enterprise cluster
• cloud workflow create-ha-database - Create high-availability database with best practices
• workflow migrate-database - Migrate database between regions/clusters
• workflow backup-restore - Full backup and restore workflow
• workflow list - List available workflows

Command structure:

# Enterprise workflows
redisctl enterprise workflow init-cluster --name prod-cluster --nodes 3
redisctl enterprise workflow setup-monitoring --prometheus-url http://prometheus:9090

# Cloud workflows  
redisctl cloud workflow create-ha-database --name prod-cache --size 2gb --region us-east-1
redisctl cloud workflow migrate-database --source 12345 --target-region us-west-2

# Common workflows (auto-detect deployment type)
redisctl workflow list
redisctl workflow describe init-cluster

Phase 2: Scriptable Workflows (Future)

Add scripting support for custom workflows using Rhai + YAML/JSON definitions.

YAML workflow definition:

name: init-cluster
description: Initialize Enterprise cluster with standard configuration
parameters:
  - name: cluster_name
    required: true
  - name: admin_email
    default: admin@redis.local
    
steps:
  - name: bootstrap
    command: enterprise.bootstrap.create
    data:
      name: "{{ cluster_name }}"
    wait: true
    
  - name: create_admin
    command: enterprise.user.create
    data:
      email: "{{ admin_email }}"
      role: admin
      
  - name: create_default_db
    command: enterprise.database.create
    data:
      name: default
      memory_limit: 1gb

Rhai scripting:

// Direct Rhai script for complex logic
let cluster = enterprise.cluster.get();
if cluster.status != "ready" {
    let result = enterprise.bootstrap.create(#{
        name: params.cluster_name,
        nodes: params.nodes || 3
    });
    result.wait(300)?;
}

// Create databases
for db in params.databases {
    enterprise.database.create(db).wait()?;
}

Success Criteria

• Users can run complex multi-step operations with a single command
• Workflows handle waiting for async operations automatically
• Clear progress output and error handling
• Ability to resume failed workflows (Phase 2)
• Custom workflows without recompilation (Phase 2)

API Design

Rust Implementation (Phase 1)

// crates/redisctl/src/workflows/mod.rs
pub trait Workflow {
    fn name(&self) -> &str;
    fn description(&self) -> &str;
    fn execute(&self, args: WorkflowArgs) -> Result<WorkflowResult>;
}

// crates/redisctl/src/workflows/enterprise/init_cluster.rs
pub struct InitClusterWorkflow;

impl Workflow for InitClusterWorkflow {
    fn execute(&self, args: WorkflowArgs) -> Result<WorkflowResult> {
        // Step 1: Bootstrap
        println!("Initializing cluster...");
        let bootstrap_result = enterprise.bootstrap(args.get("name"))?;
        wait_for_operation(bootstrap_result)?;
        
        // Step 2: Create admin user
        println!("Creating admin user...");
        // ...
        
        Ok(WorkflowResult::success())
    }
}

Related Issues

• Mentioned in Reorganize documentation for better clarity and user experience #243 (Documentation reorganization - Common Recipes section)
• Will address many use cases in future issues

Priority

High - This will significantly improve user experience for complex operations