Skip to content

Add docs/OPERATIONALIZATION_PLAN.md#43

Merged
intel352 merged 2 commits intomainfrom
copilot/create-operationalization-plan
Feb 22, 2026
Merged

Add docs/OPERATIONALIZATION_PLAN.md#43
intel352 merged 2 commits intomainfrom
copilot/create-operationalization-plan

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Feb 22, 2026

Adds the master operationalization plan documenting how the workflow engine will be deployed across SaaS/PaaS, self-hosted, and embedded modes.

Contents

  • 5-layer architecture — Core Engine → Developer Tooling → Plugin Ecosystem → Platform Services → Applications
  • Layer-by-layer breakdown — current state, actions, and deliverables for each layer (semver releases, wfctl CLI expansion, plugin registry, SaaS control plane, license server)
  • Three deployment modes — SaaS, self-hosted, and embedded (Ratchet-style) with ASCII flow diagrams
  • Repo strategy — ownership and visibility for workflow, ratchet, workflow-registry, workflow-cloud
  • 7 implementation phases — Weeks 1–14, mapping to tracked issues Decouple Admin UI from Go Binary: Serve as Static Asset via static.fileserver #34Cleanup Legacy UI and Remove Deprecated Scaffolding #39 and future work (plugin registry, license server, infra)
  • Stakeholder value matrix — SaaS customers, self-hosted operators, embedded developers, plugin authors, platform operators, org revenue
Original prompt

Create a comprehensive operationalization plan document at docs/OPERATIONALIZATION_PLAN.md that serves as the master plan for how the workflow engine will be operationalized across multiple modes (SaaS, self-hosted, embedded).

The document should contain the full plan below verbatim (in Markdown). Do NOT summarize or abbreviate — include every section, table, code block, and detail exactly as written.


Operationalization Plan: Layered Approach for Workflow Engine

Overview

This document is the master plan for operationalizing the GoCodeAlone/workflow engine across three modes of operation: SaaS/PaaS, Self-Hosted, and Embedded (like GoCodeAlone/ratchet). The approach is organized into five layers, each building on the one below.

Architecture Layers

┌──────────────────────────────────────────────────────────────────────────┐
│  Layer 5: Applications                                                    │
│  Ratchet, SaaS control plane, customer apps — consume everything below   │
├──────────────────────────────────────────────────────────────────────────┤
│  Layer 4: Platform Services  (workflow-cloud, private)                    │
│  Multi-tenancy, billing, license validation, SaaS control plane          │
├──────────────────────────────────────────────────────────────────────────┤
│  Layer 3: Plugin Ecosystem  (registry repo + marketplace)                │
│  Community plugins, premium plugins, UI plugins, external go-plugin      │
├──────────────────────────────────────────────────────────────────────────┤
│  Layer 2: Developer Tooling  (wfctl, SDKs, templates)                    │
│  CLI, project scaffolding, CI helpers, shared UI library                 │
├──────────────────────────────────────────────────────────────────────────┤
│  Layer 1: Core Engine  (GoCodeAlone/workflow, public)                    │
│  Modules, plugins, pipelines, config, static.fileserver, go-plugin       │
└──────────────────────────────────────────────────────────────────────────┘

Layer 1: Core Engine (GoCodeAlone/workflow — public)

The open-source workflow engine library and server binary. Everything in the workflow repo today, stabilized and released with semver.

Current State

  • 30+ module types, config-driven app building, visual editor, external plugin system (go-plugin/gRPC), deployment strategies (rolling, blue-green, canary), Helm chart, Prometheus/Grafana dashboards
  • No tagged semver releases — Ratchet depends on v0.0.0 with replace ../workflow
  • Admin UI embedded in binary via go:embed
  • CI exists (ci.yml, release.yml, helm-lint.yml, osv-scanner.yml, dependency-update.yml)

Actions

# Action Detail
1.1 Semver releases with GitHub Actions Tag v0.1.0 as baseline. Enhance release.yml: run tests + lint, build binaries (linux/darwin amd64/arm64), build admin UI, create GitHub Release with checksums. Every merge to main creates a pre-release; tag push creates stable release.
1.2 Remove replace directive from Ratchet Once workflow has tagged releases, Ratchet's go.mod changes from replace github.com/GoCodeAlone/workflow => ../workflow to require github.com/GoCodeAlone/workflow v0.x.y.
1.3 Admin UI decoupling Tracked: issue #34. Admin UI becomes standalone build artifact, not embedded via go:embed.
1.4 Plugin-driven admin navigation Tracked: issue #35. Remove hardcoded FALLBACK_PAGES, PLUGIN_VIEW_COMPONENTS.
1.5 UI plugin hot-reload Tracked: issue #38. go-plugin based UI plugin system.
1.6 API stability contract Document which Go packages are public API (config, plugin, module, schema, capability, store) vs internal. Breaking changes require major version bump.
1.7 License Engine stays open-source (MIT or Apache 2.0). Enterprise features gated at Layer 4.

Deliverables

  • github.com/GoCodeAlone/workflow as a versioned Go module (e.g., v0.1.0)
  • workflow-server binary downloadable from GitHub Releases
  • workflow-admin-ui tarball as a separate release artifact
  • Docker image: ghcr.io/gocodalone/workflow:<version>

Layer 2: Developer Tooling

Everything a software engineer needs to build, test, and deploy a workflow application.

2a. wfctl CLI (already at cmd/wfctl/)

Current capabilities: validate, run, serve. Needs expansion:

Command Purpose Status
wfctl init Scaffold a new workflow application project New
wfctl validate Validate workflow YAML config Exists
wfctl run Run a workflow locally Exists
wfctl build-ui Build UI assets, validate output, package New (tracked: #37)
wfctl plugin init Scaffold a new plugin project New
wfctl plugin test Run plugin in-process test mode New
wfctl publish Package and push a plugin to the registry New (Layer 3)
wfctl deploy ...

This pull request was created from Copilot chat.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: intel352 <77607+intel352@users.noreply.github.com>
Copilot AI changed the title [WIP] Add comprehensive operationalization plan document Add docs/OPERATIONALIZATION_PLAN.md Feb 22, 2026
Copilot AI requested a review from intel352 February 22, 2026 09:37
@intel352 intel352 marked this pull request as ready for review February 22, 2026 09:43
@intel352 intel352 merged commit c24b7c4 into main Feb 22, 2026
4 checks passed
@intel352 intel352 deleted the copilot/create-operationalization-plan branch February 22, 2026 09:43
intel352 added a commit that referenced this pull request Apr 24, 2026
…egistry, typed gRPC args, migrate image, teardown

Five features bundled into v0.19.0 for shared config-file shape (wfctl.yaml +
.wfctl-lock.yaml) and release boundary. Each addresses architectural debt
surfaced during BMW tonight's deploy blocker chain.

Features:
- A. Plugin manifest + lockfile split (tasks #42/#43)
- B. Multi-registry + IaCProvider.EnsureRegistryAuth (task #48)
- C. Typed-args refactor for IaCProvider gRPC (task #41)
- D. Official workflow-migrate Docker image (task #49)
- E. wfctl infra teardown with mandatory dry-run + --approve flag (new)

Non-goals: constraint-based plugin resolution (v0.20.0), transitive plugin
deps, OCI chart/artifact registries, cross-registry mirroring.

Autonomous pipeline target: v0.19.0 after BMW post-teardown stabilizes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
intel352 added a commit that referenced this pull request Apr 24, 2026
* docs: v0.19.0 architectural cleanup design — plugin manifest, multi-registry, typed gRPC args, migrate image, teardown

Five features bundled into v0.19.0 for shared config-file shape (wfctl.yaml +
.wfctl-lock.yaml) and release boundary. Each addresses architectural debt
surfaced during BMW tonight's deploy blocker chain.

Features:
- A. Plugin manifest + lockfile split (tasks #42/#43)
- B. Multi-registry + IaCProvider.EnsureRegistryAuth (task #48)
- C. Typed-args refactor for IaCProvider gRPC (task #41)
- D. Official workflow-migrate Docker image (task #49)
- E. wfctl infra teardown with mandatory dry-run + --approve flag (new)

Non-goals: constraint-based plugin resolution (v0.20.0), transitive plugin
deps, OCI chart/artifact registries, cross-registry mirroring.

Autonomous pipeline target: v0.19.0 after BMW post-teardown stabilizes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: v0.19.0 design — add Features F, G, H (outputs, verify, secret sinks)

Scope expanded from 5 to 7 features per user feedback on BMW CI gap audit:
- F. wfctl infra outputs with masked-by-default sensitivity + GHA ::add-mask::
- G. wfctl deploy verify with multi-target healthcheck + retry/timeout gate
- H. Declarative secret sinks (outputs.<field>.sinks[]) — plaintext never
  leaves wfctl process; built-in github_secret + github_env handlers;
  aws/gcp/azure sinks via plugin fan-out in v0.19.x

Motivation: BMW's Capture staging DB URL step uses doctl + awk + gh secret
set shell pipeline, leaking DATABASE_URL plaintext through stdout/env/argv.
Declarative sink pattern (like terraform's output-to-secret-manager) writes
the value in-process directly to the GitHub secrets API with libsodium
encryption. Matches user's stated principle: "if BMW CI has provider-specific
shell, fix it in workflow/wfctl so the CI stays declarative."

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: v0.19.0 implementation plan — 7 features × 9 phases

Matches design doc 2026-04-24-v0.19.0-architectural-cleanup-design.md:
- Phase 1 alpha.1: Feature A (plugin manifest + lockfile)
- Phase 2 alpha.2: Feature C client-side (typed gRPC args)
- Phase 3 (DO plugin v0.8.0): Feature C server-side + integration tests
- Phase 4 alpha.3: Feature B (multi-registry)
- Phase 5 (DO plugin v0.8.1): Feature B server-side (EnsureRegistryAuth)
- Phase 6a rc1: Feature D (workflow-migrate image)
- Phase 6b rc2: Feature E (wfctl infra teardown)
- Phase 6c rc3: Features F + G + H (outputs + verify + sinks)
- Phase 7: v0.19.0 final + changelog + docs
- Phase 8: Plugin fan-out (aws/gcp/azure/tofu) in parallel
- Phase 9: BMW migration PR (after v0.19.0 stabilizes)

Timing: all phases can merge independently; final v0.19.0 tag and Phase 9
hold until BMW's tonight deploy chain reaches prod /healthz green (task #26).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: address PR #474 review — reconcile feature count, flag naming, source task column

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
intel352 added a commit that referenced this pull request Apr 24, 2026
…nfra_output (#476)

* docs: v0.19.0 architectural cleanup design — plugin manifest, multi-registry, typed gRPC args, migrate image, teardown

Five features bundled into v0.19.0 for shared config-file shape (wfctl.yaml +
.wfctl-lock.yaml) and release boundary. Each addresses architectural debt
surfaced during BMW tonight's deploy blocker chain.

Features:
- A. Plugin manifest + lockfile split (tasks #42/#43)
- B. Multi-registry + IaCProvider.EnsureRegistryAuth (task #48)
- C. Typed-args refactor for IaCProvider gRPC (task #41)
- D. Official workflow-migrate Docker image (task #49)
- E. wfctl infra teardown with mandatory dry-run + --approve flag (new)

Non-goals: constraint-based plugin resolution (v0.20.0), transitive plugin
deps, OCI chart/artifact registries, cross-registry mirroring.

Autonomous pipeline target: v0.19.0 after BMW post-teardown stabilizes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: v0.19.0 design — add Features F, G, H (outputs, verify, secret sinks)

Scope expanded from 5 to 7 features per user feedback on BMW CI gap audit:
- F. wfctl infra outputs with masked-by-default sensitivity + GHA ::add-mask::
- G. wfctl deploy verify with multi-target healthcheck + retry/timeout gate
- H. Declarative secret sinks (outputs.<field>.sinks[]) — plaintext never
  leaves wfctl process; built-in github_secret + github_env handlers;
  aws/gcp/azure sinks via plugin fan-out in v0.19.x

Motivation: BMW's Capture staging DB URL step uses doctl + awk + gh secret
set shell pipeline, leaking DATABASE_URL plaintext through stdout/env/argv.
Declarative sink pattern (like terraform's output-to-secret-manager) writes
the value in-process directly to the GitHub secrets API with libsodium
encryption. Matches user's stated principle: "if BMW CI has provider-specific
shell, fix it in workflow/wfctl so the CI stays declarative."

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: v0.19.0 implementation plan — 7 features × 9 phases

Matches design doc 2026-04-24-v0.19.0-architectural-cleanup-design.md:
- Phase 1 alpha.1: Feature A (plugin manifest + lockfile)
- Phase 2 alpha.2: Feature C client-side (typed gRPC args)
- Phase 3 (DO plugin v0.8.0): Feature C server-side + integration tests
- Phase 4 alpha.3: Feature B (multi-registry)
- Phase 5 (DO plugin v0.8.1): Feature B server-side (EnsureRegistryAuth)
- Phase 6a rc1: Feature D (workflow-migrate image)
- Phase 6b rc2: Feature E (wfctl infra teardown)
- Phase 6c rc3: Features F + G + H (outputs + verify + sinks)
- Phase 7: v0.19.0 final + changelog + docs
- Phase 8: Plugin fan-out (aws/gcp/azure/tofu) in parallel
- Phase 9: BMW migration PR (after v0.19.0 stabilizes)

Timing: all phases can merge independently; final v0.19.0 tag and Phase 9
hold until BMW's tonight deploy chain reaches prod /healthz green (task #26).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: address PR #474 review — reconcile feature count, flag naming, source task column

* docs: v0.18.9 phase-continuation design — env-resolution consistency

BMW deploy run 24888583717 created a duplicate DO App Platform app because
wfctl infra apply used env-resolved name "bmw-staging" while wfctl ci run
--phase deploy used base module name "bmw-app". Both paths call driver.Read
by name; with different names they find different resources (or none) and
each calls Create, producing duplicates.

Root cause: cmd/wfctl/deploy_providers.go:769 reads m.Name directly after
ResolveForEnv has been applied. Same class as v0.18.7's Task #32 fix but
in the deploy-phase code path.

Fix: refactor resolveModCfg closure to return *ResolvedModule, use
resolved.Name at call sites. Audit + patch infra_output source resolution
(task #56) with the same pattern. Ship as v0.18.9.

Does not require state-sharing between IaC and CI phases; the bug is about
names, not state. Both phases use driver.Read by name; aligning the names
aligns the lookups.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: v0.18.9 phase-continuation implementation plan

9 tasks across Phase 1 (core fixes: deploy_providers.go + infra_secrets.go
+ regression tests) and Phase 2 (release + BMW unblock: PR, merge, tag,
BMW bump, teardown, redeploy).

Same-class fix as v0.18.7 Task #32: env-resolved Name used consistently
wherever modules are consumed. Target: v0.18.9 hotfix; unblocks BMW
staging deploy from run 24888583717 duplicate-resource failure.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(wfctl): ci run deploy uses env-resolved module name (not base)

Refactored resolveModCfg closure in deploy_providers.go to return
*config.ResolvedModule so callers see both resolved.Name (env-override
lifted from Config["name"]) and resolved.Config. All three call sites
(iac.provider lookup, findByType, fallback loop) now read resolved.Name
instead of m.Name.

Same class as v0.18.7 Task #32 fix for ResourceSpec.Name — env override
of Config["name"] was lifted into ResolvedModule.Name but deploy_providers.go
read m.Name directly, ignoring the override. Caused BMW deploy run
24888583717 to create duplicate DO apps (bmw-app vs bmw-staging).

Regression tested via:
- TestPluginDeployProvider_UsesEnvResolvedName (new, was failing)
- TestPluginDeployProvider_FallsBackToModuleNameWhenNoEnv (new, baseline)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wfctl): infra_output source module name flows through env resolution

Introduces resolveInfraOutput(wfCfg, source, envName, stateOutputs)
which translates the base module name in a "module.field" source string
to its env-resolved name before looking up state. State is persisted
under the env-resolved name (e.g. "bmw-staging-db"), so "bmw-database.uri"
with --env staging now correctly finds the state entry.

syncInfraOutputSecrets now accepts wfCfg and envName so the new
resolution is applied for every infra_output secret in the generate list.
The call site in infra.go (runInfraApply) loads the workflow config and
passes it through.

Closes task #56. Regression tested via:
- TestInfraOutput_EnvResolvesModuleSource (new, was failing)
- TestInfraOutput_NoEnvUsesBaseName (new, baseline)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs: CHANGELOG v0.18.9 entry

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wfctl): stateKeys actually sorts keys (comment matched implementation)

Agent-Logs-Url: https://github.com/GoCodeAlone/workflow/sessions/a0429849-a053-4485-914d-ccb115be94e8

Co-authored-by: intel352 <77607+intel352@users.noreply.github.com>

* fix(wfctl): address 4 Copilot round-1 findings on v0.18.9 (#476)

- resolveInfraOutput: ResolveForEnv ok=false now errors (config error)
  instead of silently falling back to base module name — prevents
  the env-resolution fix from being bypassed on misconfigured envs
- stateKeys: add sort.Strings so error messages list available modules
  in deterministic order (comment already said "sorted")
- infra.go: surface config.LoadFromFile error instead of discarding it —
  silent failure would regress env resolution to the pre-fix nil-wfCfg path
- CHANGELOG: replace "Closes task #60" (ambiguous GitHub issue ref) with
  "Root cause from BMW deploy run 24888583717"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wfctl): accurate error message + test for explicitly-disabled module in resolveInfraOutput

Agent-Logs-Url: https://github.com/GoCodeAlone/workflow/sessions/3accbfdf-259b-4b98-a44e-8b538d3f5857

Co-authored-by: intel352 <77607+intel352@users.noreply.github.com>

* fix(wfctl): gate LoadFromFile on envName + infra_output presence (#476)

Skip config.LoadFromFile when env resolution is not needed:
- envName="" → no env resolution, wfCfg=nil is correct
- no infra_output generators → syncInfraOutputSecrets ignores wfCfg

Avoids unnecessary file I/O on every infra apply when the caller
has no infra_output secrets or is not running with --env.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: intel352 <77607+intel352@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants