Skip to content

Comments

fix: correct {step_N} placeholder regex in executor#161

Merged
ShuxinLin merged 4 commits intofeat/156-fmsr-mcp-serverfrom
fix/160-placeholder-regex
Feb 20, 2026
Merged

fix: correct {step_N} placeholder regex in executor#161
ShuxinLin merged 4 commits intofeat/156-fmsr-mcp-serverfrom
fix/160-placeholder-regex

Conversation

@ShuxinLin
Copy link
Collaborator

Summary

Fixes #160.

_PLAN_PROMPT in planner.py is rendered with str.format(). The template contains {{step_N}} (double braces), which Python's .format() converts to {step_N} (single braces) in the prompt text the LLM receives. The LLM therefore generates args such as:

{"asset_id": "{step_2}"}

The old placeholder detection regex was:

_PLACEHOLDER_RE = re.compile(r"\{\{step_(\d+)\}\}")   # matches {{step_N}}

This never matched the single-brace {step_2} the LLM produced, so _has_placeholders() always returned False and the raw placeholder string was forwarded directly to the MCP tool — causing errors like:

{"error": "unknown asset_id {step_2} or no sensors found"}

Fix: change _PLACEHOLDER_RE to match single braces:

_PLACEHOLDER_RE = re.compile(r"\{step_(\d+)\}")        # matches {step_N}

All test fixtures updated from "{{step_N}}" to "{step_N}" to match actual LLM output format.

Test plan

  • All 55 mcp/plan_execute/tests/ unit tests pass
  • test_has_placeholders_true"{step_1}" correctly detected
  • test_executor_resolves_placeholder_via_llm — placeholder resolved via LLM call
  • test_executor_no_placeholder_skips_llm — no-placeholder steps still skip LLM

- Clear stale "in progress" comment from mcp/servers/fmsr/__init__.py
- Add python-dotenv as explicit runtime dependency in pyproject.toml
- Remove stale Pydantic V1 warning note from INSTRUCTIONS.md (warning
  is now suppressed)
- Document get_failure_mode_sensor_mapping scalability limitation in
  tool docstring (sequential LLM calls; keep lists small)

Signed-off-by: Shuxin Lin <shuxin.lin@ibm.com>

Signed-off-by: Shuxin Lin <linshuhsin@gmail.com>
_PLAN_PROMPT uses str.format(), so {{step_N}} in the template renders
to {step_N} (single braces) in the prompt the LLM receives.  The LLM
therefore generates args like {"asset_id": "{step_2}"}, but the old
regex r"\{\{step_(\d+)\}\}" only matched double-brace {{step_N}} —
so _has_placeholders() always returned False and the raw placeholder
string was forwarded directly to the MCP tool.

Fix: change _PLACEHOLDER_RE to r"\{step_(\d+)\}" (single braces).
Update all test fixtures from "{{step_N}}" to "{step_N}" to reflect
the actual format produced by the LLM.

Signed-off-by: Shuxin Lin <shuxin.lin@ibm.com>

Signed-off-by: Shuxin Lin <linshuhsin@gmail.com>
The original bug was undetected because all test fixtures built PlanStep
objects directly with "{{step_N}}" (double braces), which matched the
buggy double-brace regex.  No test exercised the real path where
parse_plan() receives LLM output containing {step_N} (single braces).

Add three tests that would have caught the bug:

test_planner.py:
  - test_placeholder_args_preserved_as_string: verifies parse_plan stores
    {step_N} placeholder strings verbatim in tool_args
  - test_placeholder_in_parsed_args_detected: verifies _has_placeholders()
    returns True for args produced by parse_plan with {step_N} placeholders

test_runner.py:
  - test_pipeline_resolves_placeholder_from_planner_output: end-to-end
    pipeline test — planner LLM returns a plan with {step_N} in args,
    executor resolves it via LLM, tool is called with the resolved value
    not the literal placeholder string

Signed-off-by: Shuxin Lin <shuxin.lin@ibm.com>

Signed-off-by: Shuxin Lin <linshuhsin@gmail.com>
Add tool and tool_args fields to StepResult so the executor records
which tool was called and with what arguments (after placeholder
resolution).  The CLI --show-history flag now prints:

  [OK ] Step 2 (IoTAgent): List all assets at site MAIN
       tool: assets  args: {'site_name': 'MAIN'}
        {"site_name": "MAIN", "total_assets": 1, ...}

The --json output also gains tool and tool_args per history entry.

Signed-off-by: Shuxin Lin <shuxin.lin@ibm.com>

Signed-off-by: Shuxin Lin <linshuhsin@gmail.com>
@ShuxinLin ShuxinLin merged commit e6ab0f0 into feat/156-fmsr-mcp-server Feb 20, 2026
1 check passed
@ShuxinLin ShuxinLin deleted the fix/160-placeholder-regex branch February 20, 2026 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant