Skip to content

feat(gates): markdown rendering and auto-linkification in human gate prompts#131

Merged
jrob5756 merged 11 commits intomicrosoft:mainfrom
PolyphonyRequiem:feat/gate-markdown-rendering
May 5, 2026
Merged

feat(gates): markdown rendering and auto-linkification in human gate prompts#131
jrob5756 merged 11 commits intomicrosoft:mainfrom
PolyphonyRequiem:feat/gate-markdown-rendering

Conversation

@PolyphonyRequiem
Copy link
Copy Markdown
Member

Summary

Adds full Markdown rendering support to human gate prompts in both the terminal CLI and the web dashboard.

Changes

Rich Terminal Rendering

  • Gate prompts are now rendered through Rich Markdown in the terminal, supporting headings, bold, lists, code blocks, and tables

Auto-Linkification (\linkify.py)

  • New markdown-aware post-processor that converts bare file paths and URLs in rendered gate prompts into clickable links
  • File paths are validated against the workflow root (path traversal blocked)
  • Skips fenced code blocks, inline code, and existing markdown links
  • Applied to both terminal (Rich) and web dashboard rendering

Web Dashboard: File Viewing

  • \GET /api/files/{path}\ endpoint serves local files relative to the workflow root with security guards (path traversal protection, extension allowlist, 1MB size limit)
  • \FileViewer\ modal component renders linked files with Markdown support
  • Relative links in gate prompts open in the FileViewer; external URLs open in new tabs

Web Dashboard: GFM Table Support

  • Added
    emark-gfm\ plugin for GitHub Flavored Markdown table parsing
  • Added table/th/td component overrides to both GateDetail and FileViewer

Documentation

  • Added 'Markdown in Gate Prompts' section to \docs/workflow-syntax.md\ documenting all supported features with examples

Tests

  • 13 file API security tests (path traversal, extension allowlist, size limits)
  • 3 gate Markdown rendering tests
  • Auto-linkify unit tests (213 lines covering URLs, file paths, code block skipping, edge cases)

Daniel Green and others added 8 commits May 1, 2026 15:31
- Terminal gates: wrap prompt text in Rich Markdown for proper heading,
  bold, list, and link rendering in the CLI
- Web dashboard: add GET /api/files/{path} endpoint to serve local files
  relative to the workflow root with security guards (path traversal
  protection, extension allowlist, 1MB size limit)
- Web dashboard: add FileViewer modal component for viewing linked files
  with Markdown rendering support
- Web dashboard: intercept relative links in gate prompts to open the
  FileViewer instead of navigating away; external URLs still open in
  new tabs
- Tests: 13 file API security tests + 3 gate Markdown rendering tests
  (79 total tests pass across both test suites)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a markdown-aware post-processor that converts bare file paths and
URLs in rendered gate prompts into clickable markdown links. This fixes
the issue where Jinja2-rendered file paths appeared as plain grey text
in the web dashboard instead of interactive file links.

Processing:
- Normalizes Jinja2 whitespace artifacts (collapses excessive blank lines)
- Auto-detects bare http(s):// URLs and wraps in markdown link syntax
- Auto-detects relative file paths (verified against workflow root),
  wraps in markdown links that the dashboard FileViewer can open
- Markdown-aware: skips fenced code blocks, inline code, existing links
- Reuses the server's extension allowlist for file detection
- Security: path traversal outside workflow root is blocked

Applied to both web dashboard (gate_presented event) and terminal
(Rich markdown rendering in HumanGateHandler).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
GateDetail's PromptMarkdown component was missing table/th/td handlers,
causing markdown tables to render as raw pipe-separated text. Added
the same table component overrides already used in FileViewer.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ReactMarkdown requires the remark-gfm plugin to parse GitHub Flavored
Markdown extensions like tables. Without it, pipe-delimited tables
render as plain text inside paragraph elements.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Documents the markdown formatting support in gate prompts including
Rich terminal rendering, GFM table support, auto-linkification of
file paths and URLs, and the web dashboard interactive features.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 1, 2026

Codecov Report

❌ Patch coverage is 90.69767% with 4 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@1c620f0). Learn more about missing BASE report.

Files with missing lines Patch % Lines
src/conductor/web/server.py 86.20% 4 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #131   +/-   ##
=======================================
  Coverage        ?   85.73%           
=======================================
  Files           ?       56           
  Lines           ?     8144           
  Branches        ?        0           
=======================================
  Hits            ?     6982           
  Misses          ?     1162           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Collaborator

@jrob5756 jrob5756 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments before we merge.

Comment thread src/conductor/executor/linkify.py Outdated
try:
candidate = (base_dir / stripped).resolve()
# Security: must be within base_dir
if not str(candidate).startswith(str(base_dir.resolve())):
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important — fragile string-prefix containment check.

if not str(candidate).startswith(str(base_dir.resolve())):

This has the classic prefix-confusion bug: /foo/bar is a string prefix of /foo/bar2. The new server.py (line ~290) correctly uses target.relative_to(self._workflow_root) in a try/except ValueError — please apply the same pattern here, or use candidate.is_relative_to(base_dir.resolve()) (stdlib since 3.9; project requires 3.12+).

Not exploitable in practice today because candidate is also resolved and gate roots don't usually have sibling-prefix names, but it's a footgun worth removing.

".ps1",
".plan.md",
}
)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important — extension allowlist already drifting between modules.

This LINKABLE_EXTENSIONS (21 entries, includes .plan.md) and web/server.py's _FILE_ALLOWED_EXTENSIONS (20 entries, missing .plan.md) are intended to stay in sync — the docstring above even says so. They've already drifted on this very PR.

Suggest: have server.py import this constant directly, e.g. from conductor.executor.linkify import LINKABLE_EXTENSIONS.

Side note: the .plan.md entry here is dead anyway — Path.suffix returns only .md for foo.plan.md, so it's already covered by the .md entry in server.py. Consolidating to one source of truth makes this kind of confusion go away.

Comment thread src/conductor/executor/linkify.py Outdated
r"(?!https?://)" # not a URL
r"(?:[a-zA-Z0-9_.][a-zA-Z0-9_./-]*[a-zA-Z0-9_]" # path chars with at least one /
r")"
)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion — dead regex.

_FILE_PATH_RE is defined here but never referenced (grep -rn _FILE_PATH_RE src/ tests/ returns only this definition). The actual file-path linkification uses re.split(r"(\s+)", segment) token-by-token in _try_linkify_path. Safe to delete (along with the explanatory comment block immediately above).

Comment thread src/conductor/executor/linkify.py Outdated
def _has_linkable_extension(path: str) -> bool:
"""Check if a path ends with a known linkable extension."""
lower = path.lower()
return any(lower.endswith(ext) for ext in LINKABLE_EXTENSIONS)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit — str.endswith accepts a tuple natively.

def _has_linkable_extension(path: str) -> bool:
    lower = path.lower()
    return any(lower.endswith(ext) for ext in LINKABLE_EXTENSIONS)

Can be:

_LINKABLE_SUFFIXES = tuple(LINKABLE_EXTENSIONS)
...
def _has_linkable_extension(path: str) -> bool:
    return path.lower().endswith(_LINKABLE_SUFFIXES)

No Python-level loop, slightly faster, more idiomatic.

Comment thread src/conductor/executor/linkify.py Outdated

# Build markdown link with forward slashes (for dashboard API)
link_target = normalized
return f"{prefix}[{stripped}]({link_target}){suffix}"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit — dead local.

link_target = normalized
return f"{prefix}[{stripped}]({link_target}){suffix}"

Inline normalized directly into the f-string and drop the link_target line.

return JSONResponse(
{"error": "Absolute paths are not allowed"},
status_code=403,
)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit — hand-rolled absolute-path detection.

The combined absolute / UNC / drive-letter check could be PurePosixPath(file_path).is_absolute() or PureWindowsPath(file_path).is_absolute() (or PurePath). Keep the "://" scheme check separate. Reduces a brittle hand-rolled check to a documented stdlib call.

Comment thread src/conductor/engine/workflow.py Outdated
# If we bail early when ``has_connections()`` is False, a later click
# in the dashboard pushes a message to ``_gate_response_queue`` that
# nobody is awaiting, and the workflow hangs forever.
gate_base = Path(self.workflow_path).resolve().parent if self.workflow_path else None
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion — triplicated gate_base derivation.

The same expression appears at lines 834, 843, and 1433-1436 (and a similar pattern at line 583 for workflow_dir):

gate_base = Path(self.workflow_path).resolve().parent if self.workflow_path else None

Extract a cached property:

@property
def _workflow_dir(self) -> Path | None:
    return Path(self.workflow_path).resolve().parent if self.workflow_path else None

and reuse at all four call sites.

Merged upstream/main into feat/gate-markdown-rendering. The only conflict
was in src/conductor/web/static/index.html (competing built-asset hashes).
Resolved by rebuilding the frontend to produce a single bundle that includes
both the gate-markdown-rendering features and upstream's breadcrumb/subworkflow
changes. Fixed TypeScript errors from the merge in GroupDetail, GateNode,
WorkflowGraph, WorkflowNode, YamlViewer, and use-viewed-context.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
jrob5756

This comment was marked as resolved.

Review feedback addressed:

1. linkify.py: Use is_relative_to() for path containment check instead
   of fragile string-prefix comparison (resolved via upstream merge).

2. linkify.py: Remove dead .plan.md entry from LINKABLE_EXTENSIONS
   (already covered by .md via Path.suffix). server.py now imports
   LINKABLE_EXTENSIONS from linkify — single source of truth.

3. linkify.py: Remove dead _FILE_PATH_RE regex (never referenced).

4. linkify.py: Use tuple str.endswith() idiom for _has_linkable_extension
   (no Python-level loop, more idiomatic).

5. linkify.py: Inline dead link_target local into the f-string.

6. server.py: Replace hand-rolled absolute-path detection with
   PurePosixPath/PureWindowsPath.is_absolute() (stdlib, documented).

7. workflow.py: Extract _workflow_dir cached property to eliminate
   triplicated gate_base/gate_base_dir derivation at three call sites.

Also merged upstream/main and rebuilt frontend assets.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@PolyphonyRequiem
Copy link
Copy Markdown
Member Author

Addressed all 7 review comments in commit 6c37af1:

  1. Path containment (Important) - Resolved via upstream merge; now uses is_relative_to() instead of the fragile string-prefix comparison.

  2. Extension allowlist drift (Important) - server.py now imports LINKABLE_EXTENSIONS directly from linkify.py (single source of truth). Removed the dead .plan.md entry (Path.suffix returns .md anyway).

  3. Dead regex - Removed _FILE_PATH_RE (defined but never referenced).

  4. endswith tuple idiom (Nit) - _has_linkable_extension now uses path.lower().endswith(_LINKABLE_SUFFIXES) with a pre-computed tuple. No Python-level loop.

  5. Dead local (Nit) - Inlined normalized directly into the f-string, removed link_target.

  6. Absolute-path detection (Nit) - Replaced hand-rolled check with PurePosixPath(file_path).is_absolute() or PureWindowsPath(file_path).is_absolute(). Kept the :// scheme check separate as suggested.

  7. Triplicated gate_base derivation (Suggestion) - Extracted a _workflow_dir property on WorkflowEngine and replaced all three call sites.

All 87 linkify+server tests and 517 engine tests pass.

- tests/test_web/test_server.py: union imports (Path + traceback/patch
  both now needed)
- src/conductor/web/static/index.html and assets/: regenerated via
  npm run build (new hashes: index-ZAvmqxaV.js, index-CPAEvQwD.css)
- frontend/tsconfig.tsbuildinfo: regenerated by tsc -b

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jrob5756 jrob5756 merged commit f18a646 into microsoft:main May 5, 2026
7 checks passed
jrob5756 added a commit that referenced this pull request May 5, 2026
…, #121-#123, #125, #129, #130, #131, #139, #141-#144, #146)

CHANGELOG: add 6 newer PRs (#119, #121, #122, #123, #125, #113, #130, #131, #141, #146) to [Unreleased] alongside the previously documented batch.

docs/workflow-syntax.md:
  - Add metadata + instructions fields to the workflow configuration block.
  - Add input_mapping and max_depth to Sub-Workflow Steps; correct stale claims that circular references are rejected and that workflow steps cannot be used in for_each groups.
  - Add 'Sub-workflows in for_each groups' subsection with example.
  - Add JSON stdout auto-parsing note + example to Script Steps output section.
  - Add type-appropriate zero values table to Workflow Inputs.
  - Add 'Workflow Metadata Variables' subsection covering workflow.dir, workflow.file, workflow.name.
  - Update on_start hook context list to include the new workflow.dir/file vars.

docs/cli-reference.md:
  - Document --metadata/-m, --workspace-instructions, and --instructions flags on conductor run.
  - Add 'Metadata and Instructions' examples block.
  - Update conductor validate to describe the new template-reference error/warning checks added in #125.

docs/providers/claude.md, docs/providers/comparison.md:
  - Replace stale 'All models support a 200K token context window' / '200K (all models)' claims with notes that the dashboard now sources context_window_max from each provider's SDK at runtime (#144).

README.md:
  - Refresh the Features list to mention sub-workflow composition, dialog mode, workspace instructions, breadcrumb navigation, and the enhanced validate behavior.
  - Add --metadata, --workspace-instructions, --instructions to the conductor run options table.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
jrob5756 added a commit that referenced this pull request May 5, 2026
* docs: changelog + doc updates for unreleased PRs (#100, #109-#111, #119, #121-#123, #125, #129, #130, #131, #139, #141-#144, #146)

CHANGELOG: add 6 newer PRs (#119, #121, #122, #123, #125, #113, #130, #131, #141, #146) to [Unreleased] alongside the previously documented batch.

docs/workflow-syntax.md:
  - Add metadata + instructions fields to the workflow configuration block.
  - Add input_mapping and max_depth to Sub-Workflow Steps; correct stale claims that circular references are rejected and that workflow steps cannot be used in for_each groups.
  - Add 'Sub-workflows in for_each groups' subsection with example.
  - Add JSON stdout auto-parsing note + example to Script Steps output section.
  - Add type-appropriate zero values table to Workflow Inputs.
  - Add 'Workflow Metadata Variables' subsection covering workflow.dir, workflow.file, workflow.name.
  - Update on_start hook context list to include the new workflow.dir/file vars.

docs/cli-reference.md:
  - Document --metadata/-m, --workspace-instructions, and --instructions flags on conductor run.
  - Add 'Metadata and Instructions' examples block.
  - Update conductor validate to describe the new template-reference error/warning checks added in #125.

docs/providers/claude.md, docs/providers/comparison.md:
  - Replace stale 'All models support a 200K token context window' / '200K (all models)' claims with notes that the dashboard now sources context_window_max from each provider's SDK at runtime (#144).

README.md:
  - Refresh the Features list to mention sub-workflow composition, dialog mode, workspace instructions, breadcrumb navigation, and the enhanced validate behavior.
  - Add --metadata, --workspace-instructions, --instructions to the conductor run options table.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: bump version to 0.1.11 and changelog #148

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants