Skip to content

feat(engine): add workflow.dir, workflow.file, workflow.name template variables#121

Merged
jrob5756 merged 4 commits intomicrosoft:mainfrom
PolyphonyRequiem:feat/workflow-dir-template
May 4, 2026
Merged

feat(engine): add workflow.dir, workflow.file, workflow.name template variables#121
jrob5756 merged 4 commits intomicrosoft:mainfrom
PolyphonyRequiem:feat/workflow-dir-template

Conversation

@PolyphonyRequiem
Copy link
Copy Markdown
Member

Summary

Adds three new template variables available in all agent contexts:

  • {{ workflow.dir }} — absolute path to the workflow YAML's parent directory
  • {{ workflow.file }} — absolute path to the workflow YAML file itself
  • {{ workflow.name }} — workflow name from the YAML config

Motivation

Script agents specify script paths in args, but these resolve relative to CWD (where conductor run is invoked). When workflows live in a registry (e.g., ~/.conductor/registries/twig/), co-located scripts can't be found because CWD is the user's project directory, not the registry.

{{ workflow.dir }} solves this by letting scripts build paths relative to the workflow file:

agents:
  - name: detector
    type: script
    command: pwsh
    args:
      - "-File"
      - "{{ workflow.dir }}/scripts/detect-state.ps1"

This matches how !file tags already resolve prompt file paths relative to the workflow file.

Changes

  • src/conductor/engine/context.py: Add workflow_dir, workflow_file, workflow_name fields to WorkflowContext. Include them in build_for_agent() output under the workflow dict (all context modes).
  • src/conductor/engine/workflow.py: Set the new fields from workflow_path during WorkflowEngine.__init__().
  • tests/test_engine/test_context.py: Tests for metadata in accumulate/explicit modes, and empty-string omission.

Design decisions

  • Available in all context modes — workflow metadata is lightweight (3 strings) and fundamental. Unlike agent outputs, these shouldn't be filtered by explicit mode.
  • Empty strings omitted — when workflow_path is None (e.g., tests), the fields are empty strings and excluded from the context dict to avoid polluting templates.
  • No breaking changes — existing workflow.input behavior is unchanged. New fields are additive.

Testing

  • 139 context + workflow engine tests pass
  • New tests cover accumulate mode, explicit mode, and empty-metadata omission

… variables

Adds three new template variables available in all agent contexts:
- {{ workflow.dir }}  - absolute path to the workflow YAML's directory
- {{ workflow.file }} - absolute path to the workflow YAML file
- {{ workflow.name }} - workflow name from the YAML config

These enable script agents to resolve co-located scripts relative to
the workflow file rather than CWD, which is critical for registry-based
workflows where scripts live alongside the YAML in the registry directory.

Example:
  args:
    - -File
    - {{ workflow.dir }}/scripts/detect-state.ps1

Available in all context modes (accumulate, last_only, explicit).
Empty strings are omitted from context to avoid polluting templates.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 25, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@7655dc7). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #121   +/-   ##
=======================================
  Coverage        ?   84.95%           
=======================================
  Files           ?       53           
  Lines           ?     7452           
  Branches        ?        0           
=======================================
  Hits            ?     6331           
  Misses          ?     1121           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Collaborator

@jrob5756 jrob5756 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge conflict and some small comments. Thanks!

Available in templates as ``{{ workflow.file }}``."""

workflow_name: str = ""
"""Name of the workflow from the YAML config.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resume drops these fields.

The new workflow_dir/file/name fields aren't included in to_dict() (line 486) or restored by from_dict() (line 502). On resume, cli/run.py:1538 calls engine.set_context(restored_context), which replaces the engine's context (built with the metadata in __init__) with one that has the metadata wiped to empty strings. After resume, {{ workflow.dir/file/name }} will silently disappear from templates — exactly the registry-based script-path scenario this PR exists for.

Recommended fix: repopulate metadata inside WorkflowEngine.set_context() from self.workflow_path / self.config. Keeps absolute paths out of checkpoint files (which would otherwise become stale if the workflow moves) and keeps the source of truth co-located with the engine that knows the path.

Please also add a regression test in tests/test_engine/test_resume.py (or similar) asserting that workflow.dir survives a checkpoint round-trip + resume.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you're right, the round trip drops them and set_context was happily clobbering the engine's view. Fixed in eaf6c9f: set_context now repopulates workflow_dir/file/name from self.workflow_path and self.config after the replace, so the engine stays the source of truth and we keep absolute paths out of the checkpoint. Added test_set_context_repopulates_workflow_metadata that round-trips through to_dict/from_dict, calls set_context, and verifies via build_for_agent that templates actually resolve. Also covered the no-workflow_path case in test_set_context_without_workflow_path_still_sets_name.


assert "dir" not in agent_ctx["workflow"]
assert "file" not in agent_ctx["workflow"]
assert "name" not in agent_ctx["workflow"]
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: add an engine-level wiring test.

These tests verify WorkflowContext.build_for_agent() formatting given pre-set fields, but nothing asserts that WorkflowEngine.__init__(config, workflow_path=...) actually wires the path through into those fields (the change at workflow.py:311-315). A small test in tests/test_engine/test_workflow.py like:

def test_engine_populates_workflow_metadata(tmp_path):
    wf_file = tmp_path / "wf.yaml"
    wf_file.write_text("...")  # or use a fixture config
    engine = WorkflowEngine(config, workflow_path=wf_file)
    assert engine.context.workflow_dir == str(tmp_path.resolve())
    assert engine.context.workflow_file == str(wf_file.resolve())
    assert engine.context.workflow_name == config.workflow.name

would catch any future regression in the wiring (e.g., if someone refactors __init__ and reverts to the old WorkflowContext()).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair, there was no guard that __init__ was actually wiring the path into the context. Added test_engine_populates_workflow_metadata and test_engine_workflow_metadata_empty_without_path in tests/test_engine/test_workflow.py to lock that down. Same commit, eaf6c9f.

Daniel Green and others added 3 commits May 4, 2026 11:56
Resume drops workflow_dir/file/name because WorkflowContext.from_dict()
omits absolute path metadata (intentionally, to keep checkpoints portable).
Without this, set_context() during the resume path silently wipes the
workflow.dir/file/name template variables that the rest of this PR exists
to provide.

set_context() now repopulates from self.workflow_path and self.config —
the engine knows the current path and is the source of truth.

Tests:
- test_set_context_repopulates_workflow_metadata: round-trip + assert
  metadata survives, including end-to-end via build_for_agent
- test_set_context_without_workflow_path_still_sets_name: path-less case
- test_engine_populates_workflow_metadata: wiring guard for __init__
- test_engine_workflow_metadata_empty_without_path: no-pollution guard

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…emplate

# Conflicts:
#	src/conductor/engine/workflow.py
Resolve conflict in build_for_agent: combine main's local-render agent
input handling (initial_workflow_inputs from microsoft#119) with this PR's
workflow_meta exposure.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jrob5756 jrob5756 merged commit 2e51912 into microsoft:main May 4, 2026
3 checks passed
@PolyphonyRequiem PolyphonyRequiem deleted the feat/workflow-dir-template branch May 4, 2026 20:19
jrob5756 added a commit that referenced this pull request May 5, 2026
…, #121-#123, #125, #129, #130, #131, #139, #141-#144, #146)

CHANGELOG: add 6 newer PRs (#119, #121, #122, #123, #125, #113, #130, #131, #141, #146) to [Unreleased] alongside the previously documented batch.

docs/workflow-syntax.md:
  - Add metadata + instructions fields to the workflow configuration block.
  - Add input_mapping and max_depth to Sub-Workflow Steps; correct stale claims that circular references are rejected and that workflow steps cannot be used in for_each groups.
  - Add 'Sub-workflows in for_each groups' subsection with example.
  - Add JSON stdout auto-parsing note + example to Script Steps output section.
  - Add type-appropriate zero values table to Workflow Inputs.
  - Add 'Workflow Metadata Variables' subsection covering workflow.dir, workflow.file, workflow.name.
  - Update on_start hook context list to include the new workflow.dir/file vars.

docs/cli-reference.md:
  - Document --metadata/-m, --workspace-instructions, and --instructions flags on conductor run.
  - Add 'Metadata and Instructions' examples block.
  - Update conductor validate to describe the new template-reference error/warning checks added in #125.

docs/providers/claude.md, docs/providers/comparison.md:
  - Replace stale 'All models support a 200K token context window' / '200K (all models)' claims with notes that the dashboard now sources context_window_max from each provider's SDK at runtime (#144).

README.md:
  - Refresh the Features list to mention sub-workflow composition, dialog mode, workspace instructions, breadcrumb navigation, and the enhanced validate behavior.
  - Add --metadata, --workspace-instructions, --instructions to the conductor run options table.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
jrob5756 added a commit that referenced this pull request May 5, 2026
* docs: changelog + doc updates for unreleased PRs (#100, #109-#111, #119, #121-#123, #125, #129, #130, #131, #139, #141-#144, #146)

CHANGELOG: add 6 newer PRs (#119, #121, #122, #123, #125, #113, #130, #131, #141, #146) to [Unreleased] alongside the previously documented batch.

docs/workflow-syntax.md:
  - Add metadata + instructions fields to the workflow configuration block.
  - Add input_mapping and max_depth to Sub-Workflow Steps; correct stale claims that circular references are rejected and that workflow steps cannot be used in for_each groups.
  - Add 'Sub-workflows in for_each groups' subsection with example.
  - Add JSON stdout auto-parsing note + example to Script Steps output section.
  - Add type-appropriate zero values table to Workflow Inputs.
  - Add 'Workflow Metadata Variables' subsection covering workflow.dir, workflow.file, workflow.name.
  - Update on_start hook context list to include the new workflow.dir/file vars.

docs/cli-reference.md:
  - Document --metadata/-m, --workspace-instructions, and --instructions flags on conductor run.
  - Add 'Metadata and Instructions' examples block.
  - Update conductor validate to describe the new template-reference error/warning checks added in #125.

docs/providers/claude.md, docs/providers/comparison.md:
  - Replace stale 'All models support a 200K token context window' / '200K (all models)' claims with notes that the dashboard now sources context_window_max from each provider's SDK at runtime (#144).

README.md:
  - Refresh the Features list to mention sub-workflow composition, dialog mode, workspace instructions, breadcrumb navigation, and the enhanced validate behavior.
  - Add --metadata, --workspace-instructions, --instructions to the conductor run options table.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: bump version to 0.1.11 and changelog #148

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants