Search before asking
Description
Found during YAML doc verification (#742) while running an end-to-end agent
with a real LLM.
SafeFormatter in python/flink_agents/api/prompts/utils.py:22-41 uses the
regex \{([^{}]+)\} to substitute {key} placeholders, and offers no
escape syntax for emitting a literal { or }:
{name} → substituted with the value of name
{{, }}, {{name}} → all preserved as-is in the output
Python f-strings and str.format use {{ and }} for literal { and }.
A user porting prompt content from f-string-shaped examples to a Flink
Agents prompt template will reasonably double the braces in JSON examples
to "escape" them — and find that the LLM sees literal {{ and }} in the
prompt, then sometimes mimics the doubled-brace form in its response,
breaking downstream JSON parsing.
This was hit during end-to-end YAML verification: a system prompt with
double-brace JSON examples occasionally produced LLM responses with
{{ ... }} instead of { ... }, which then failed json.loads in the
downstream action and silently dropped the output (related: #780).
The framework's behavior is reasonable from an implementation perspective,
but it is undocumented and diverges from a common convention.
Suggested fix
Pick one:
-
Document only — add a short note to
docs/content/docs/development/prompts.md (and a cross-reference from
yaml.md's prompt section) stating:
{name} is the only substitution syntax
{ and } characters that are not placeholders pass through verbatim
- there is no
{{ / }} escape — if you need a literal { or }, write
it as-is
- reminder that LLM examples inside the prompt should use single braces
-
Document + code — extend SafeFormatter (and the Java equivalent in
Prompt.formatMessages) to recognize {{ → { and }} → }, matching
Python f-string / str.format conventions, then update the doc
accordingly. This is a behavior change; consider a deprecation cycle if
anyone might depend on {{ being literal today.
How to reproduce
In a YAML prompt, write a JSON example with doubled braces (a natural copy
from Python f-string examples):
prompts:
- name: p
messages:
- role: system
content: |
Respond as JSON like:
{{
"result": "ok"
}}
- role: user
content: "{input}"
The rendered prompt the LLM sees contains literal {{ and }}. The LLM
may then mimic the doubled-brace form in its response.
Version and environment
Flink Agents 0.3.0 (main). Python SafeFormatter; the Java equivalent
Prompt.formatMessages should be checked for parity.
Are you willing to submit a PR?
Search before asking
Description
Found during YAML doc verification (#742) while running an end-to-end agent
with a real LLM.
SafeFormatterinpython/flink_agents/api/prompts/utils.py:22-41uses theregex
\{([^{}]+)\}to substitute{key}placeholders, and offers noescape syntax for emitting a literal
{or}:{name}→ substituted with the value ofname{{,}},{{name}}→ all preserved as-is in the outputPython f-strings and
str.formatuse{{and}}for literal{and}.A user porting prompt content from f-string-shaped examples to a Flink
Agents prompt template will reasonably double the braces in JSON examples
to "escape" them — and find that the LLM sees literal
{{and}}in theprompt, then sometimes mimics the doubled-brace form in its response,
breaking downstream JSON parsing.
This was hit during end-to-end YAML verification: a system prompt with
double-brace JSON examples occasionally produced LLM responses with
{{ ... }}instead of{ ... }, which then failedjson.loadsin thedownstream action and silently dropped the output (related: #780).
The framework's behavior is reasonable from an implementation perspective,
but it is undocumented and diverges from a common convention.
Suggested fix
Pick one:
Document only — add a short note to
docs/content/docs/development/prompts.md(and a cross-reference fromyaml.md's prompt section) stating:{name}is the only substitution syntax{and}characters that are not placeholders pass through verbatim{{/}}escape — if you need a literal{or}, writeit as-is
Document + code — extend
SafeFormatter(and the Java equivalent inPrompt.formatMessages) to recognize{{→{and}}→}, matchingPython f-string / str.format conventions, then update the doc
accordingly. This is a behavior change; consider a deprecation cycle if
anyone might depend on
{{being literal today.How to reproduce
In a YAML prompt, write a JSON example with doubled braces (a natural copy
from Python f-string examples):
The rendered prompt the LLM sees contains literal
{{and}}. The LLMmay then mimic the doubled-brace form in its response.
Version and environment
Flink Agents 0.3.0 (
main). PythonSafeFormatter; the Java equivalentPrompt.formatMessagesshould be checked for parity.Are you willing to submit a PR?