Skip to content

Refactor workflow syntax tool and improve variable validation#796

Merged
larryro merged 4 commits into
mainfrom
refactor/workflow-syntax-tool
Mar 15, 2026
Merged

Refactor workflow syntax tool and improve variable validation#796
larryro merged 4 commits into
mainfrom
refactor/workflow-syntax-tool

Conversation

@larryro
Copy link
Copy Markdown
Collaborator

@larryro larryro commented Mar 15, 2026

Summary

  • Rename workflow_examples tool to workflow_syntax with a streamlined syntax reference focused on documentation over examples
  • Namespace workflow variables with config.* / variables.* prefixes and validate sources (bare names now error with suggestions)
  • Rename output step outputMapping to mapping (with backward compat) and add nextSteps port validation per step type
  • Surface available integrations in the workflow syntax tool and expand builder agent tool access

Test plan

  • Added workflow_syntax_tool.test.ts for the renamed tool
  • Added validate_variable_references_known_sources.test.ts for namespaced variable validation
  • Added validate_llm_output_path.test.ts for LLM output format awareness
  • Added validate_next_steps_ports.test.ts for port validation
  • Updated output.test.ts for the mapping rename

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • New workflow syntax reference tool providing category-based documentation and integration context
    • Enhanced variable reference validation with support for config and variables reference types
    • Step port validation for workflow definitions
  • Refactor

    • Output step configuration field renamed from outputMapping to mapping (backward compatible)
    • Variable storage structure optimized with namespace nesting
    • Workflow creation instructions simplified and consolidated
  • Tests

    • Comprehensive test suites added for validation, syntax tools, and output path handling

larryro added 4 commits March 15, 2026 19:45
…fy syntax reference

Rename the workflow_examples tool to workflow_syntax to better reflect its purpose of providing syntax documentation rather than just examples. Streamline the syntax reference helpers and reduce redundant instructions in workflow tool descriptions.
…tSteps port validation

Rename the output step config field from `outputMapping` to `mapping` with
backward compatibility for existing workflows. Add validation that nextSteps
port names match allowed ports per step type (e.g. start/llm/action use
"success", condition uses "true"/"false"). Expand syntax reference with
hello_world example and explicit port documentation to reduce agent mistakes.
… and validate sources

Introduce explicit namespacing for variable references: config.* for workflow
config variables, variables.* for set_variables action output. Bare variable
names are now flagged as errors with suggestions. Also adds LLM output format
awareness to validation (text vs json schemas) and detects invalid sub-field
access on scalar outputs.
Copy link
Copy Markdown

@greptile-apps greptile-apps Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

@larryro larryro merged commit 5fd1eda into main Mar 15, 2026
16 of 17 checks passed
@larryro larryro deleted the refactor/workflow-syntax-tool branch March 15, 2026 14:12
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 15, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 64967a37-022a-4535-b5cb-c3908969eb0f

📥 Commits

Reviewing files that changed from the base of the PR and between 217699b and 9402990.

⛔ Files ignored due to path filters (1)
  • services/platform/convex/_generated/api.d.ts is excluded by !**/_generated/**
📒 Files selected for processing (40)
  • services/platform/app/features/automations/components/step-create-dialog.tsx
  • services/platform/app/features/chat/components/workflow-creation-approval-card.tsx
  • services/platform/app/features/custom-agents/components/tool-selector.tsx
  • services/platform/convex/agent_tools/tool_names.ts
  • services/platform/convex/agent_tools/tool_registry.ts
  • services/platform/convex/agent_tools/workflows/__tests__/workflow_syntax_tool.test.ts
  • services/platform/convex/agent_tools/workflows/create_workflow_tool.ts
  • services/platform/convex/agent_tools/workflows/helpers/syntax_reference.ts
  • services/platform/convex/agent_tools/workflows/save_workflow_definition_tool.ts
  • services/platform/convex/agent_tools/workflows/update_workflow_step_tool.ts
  • services/platform/convex/agent_tools/workflows/workflow_examples_tool.ts
  • services/platform/convex/agent_tools/workflows/workflow_syntax_tool.ts
  • services/platform/convex/custom_agents/system_defaults.ts
  • services/platform/convex/lib/variables/build_context.ts
  • services/platform/convex/workflow_engine/action_defs/set_variables_action.ts
  • services/platform/convex/workflow_engine/helpers/step_execution/execute_step_by_type.ts
  • services/platform/convex/workflow_engine/helpers/step_execution/initialize_execution_variables.ts
  • services/platform/convex/workflow_engine/helpers/step_execution/persist_execution_result.ts
  • services/platform/convex/workflow_engine/helpers/validation/steps/output.test.ts
  • services/platform/convex/workflow_engine/helpers/validation/steps/output.ts
  • services/platform/convex/workflow_engine/helpers/validation/validate_next_steps_ports.test.ts
  • services/platform/convex/workflow_engine/helpers/validation/validate_next_steps_ports.ts
  • services/platform/convex/workflow_engine/helpers/validation/validate_workflow_definition.ts
  • services/platform/convex/workflow_engine/helpers/validation/variables/index.ts
  • services/platform/convex/workflow_engine/helpers/validation/variables/parse.ts
  • services/platform/convex/workflow_engine/helpers/validation/variables/step_schemas.ts
  • services/platform/convex/workflow_engine/helpers/validation/variables/types.ts
  • services/platform/convex/workflow_engine/helpers/validation/variables/validate.ts
  • services/platform/convex/workflow_engine/helpers/validation/variables/validate_llm_output_path.test.ts
  • services/platform/convex/workflow_engine/helpers/validation/variables/validate_variable_references_known_sources.test.ts
  • services/platform/convex/workflow_engine/helpers/validation/variables/validate_variable_references_known_sources.ts
  • services/platform/convex/workflow_engine/instructions/core_instructions.ts
  • services/platform/convex/workflow_engine/types/nodes.ts
  • services/platform/convex/workflow_engine/workflow_syntax_compact.ts
  • services/platform/convex/workflows/definitions/activate_version.ts
  • services/platform/convex/workflows/definitions/create_draft_from_active.ts
  • services/platform/convex/workflows/definitions/publish_draft.ts
  • services/platform/convex/workflows/schema.ts
  • services/platform/convex/workflows/steps/validators.ts
  • services/platform/convex/workflows/validators.ts

📝 Walkthrough

Walkthrough

This pull request refactors the workflow automation tooling and variable handling system. It replaces the workflow_examples_tool with a new workflow_syntax_tool that provides syntax references by category and supports integration introspection for the action category. The output step configuration field is migrated from outputMapping to mapping with backward compatibility fallbacks and deprecation warnings. Syntax reference modules are reorganized, removing legacy categories and adding a hello_world example. Variable scope handling is restructured to nest configuration and step variables under a dedicated namespace. New validation utilities are introduced for validating nextSteps port names and variable reference sources. Schema changes add optional mapping and inputMapping fields with deprecation markers. Tool registrations, agent defaults, core instructions, and syntax documentation are updated to reference the new workflow_syntax tool and mapping field names.

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~95 minutes

Possibly related PRs

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch refactor/workflow-syntax-tool
📝 Coding Plan
  • Generate coding plan for human review comments

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant