Skip to content

feat(templates): auto-discover flowfile templates on sync#413

Merged
jahvon merged 1 commit into
mainfrom
feat/template-autodiscovery
Jul 26, 2026
Merged

feat(templates): auto-discover flowfile templates on sync#413
jahvon merged 1 commit into
mainfrom
feat/template-autodiscovery

Conversation

@jahvon

@jahvon jahvon commented Jul 26, 2026

Copy link
Copy Markdown
Member

Summary

Flowfile templates previously had to be registered by name (flow template add) in the global config to be usable by name. This PR makes them auto-discovered from *.flow.tmpl files within workspaces during flow sync — the same way executables (.flow files) are found — so dropping a template into a workspace makes it available with zero configuration.

Registration still works and is preserved for templates that live outside a workspace or should be available from every workspace.

Resolution order: -f <path> → registered name → discovered name.

What changed

  • Shared discovery walk (pkg/filesystem/executables.go): extracted the workspace tree walk into a shared findFiles(cfg, filter, match); executable discovery is behavior-preserving.
  • Template loader (pkg/filesystem/templates.go): LoadWorkspaceFlowFileTemplates, mirroring the executable loader.
  • TemplateCache (pkg/cache/templates_cache.go): mirrors ExecutableCacheImpl — validates each template, warns on cross-workspace name collisions, persists to the store. Added as a third stage in cache.UpdateAll, so flow sync discovers templates.
  • App context (pkg/context/context.go): Context.TemplateCache constructed and propagated in ShallowCopy.
  • Command wiring (cmd/internal/): name resolution falls back to the cache; template list merges registered + discovered (registered wins on collision); template add now stores absolute paths.
  • Workspace-scoped filter (types/workspace/schema.yaml): a templates include/exclude block in flow.yaml, same semantics as executables.
  • Docs: updated the templating guide with an auto-discovery section, scoping, and collision handling.
  • Fixes: canonicalized .tmpl.flow.flow.tmpl in test fixtures.

Only files with the exact .flow.tmpl extension are discovered, so artifact partials like deployment.yaml.tmpl are never treated as template generators. For cross-workspace name collisions, a workspace/name reference disambiguates.

Testing

  • Unit: cache (discovery, invalid-skip, dup-warn, qualified resolution), filesystem discovery (extension matching + exclusions), mergeTemplates precedence.
  • E2E: seed an unregistered .flow.tmpl, flow sync, then list and generate it.
  • flow validate passes (generate → lint → unit + e2e → schemas).

🤖 Generated with Claude Code

Templates previously had to be registered by name in the global config to
be usable via name. They are now also auto-discovered from *.flow.tmpl files
within workspaces during `flow sync`, mirroring how executables are found.

- share the workspace walk between executable and template discovery
- add a TemplateCache stage to cache.UpdateAll, wired into the app context
- resolve templates by name: -f path > registered name > discovered name;
  support a workspace/name form to disambiguate cross-workspace collisions
- merge registered + discovered templates in `template list`
- add a workspace-scoped `templates` include/exclude filter (flow.yaml)
- normalize registered template paths to absolute
- fix .tmpl.flow test fixtures to the canonical .flow.tmpl extension

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

@jahvon
jahvon merged commit 121aeef into main Jul 26, 2026
15 checks passed
@jahvon
jahvon deleted the feat/template-autodiscovery branch July 26, 2026 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant