fix(parsers): MermaidParser no longer silently drops edge lines, plus metadata pass - #3
Merged
Merged
Conversation
MermaidParser discarded any edge line it could not match, without a word.
Two ways that turned into a silently wrong diagram:
- Inline node definitions on an edge line were mis-parsed. For
`start([Start]) --> check_user{Is user authenticated?}` the edge parser
failed, the node parser then matched whichever shape came first in its
pattern list, and the edge plus the remaining node vanished.
- Non-canonical link spellings were dropped whole. gpt-4o-mini emits
`check_user --|Yes|--> show_dashboard` (valid Mermaid is
`check_user -->|Yes| show_dashboard`); the line went, and with it the
`check_user{...}` decision node it referenced, so a diagram that should
hold a DECISION node held none.
The parser now:
- resolves each edge endpoint separately, keeping inline node shapes
- normalizes the `--|label|-->` drift, plus the valid `-- label -->` and
`-. label .->` forms
- parses chained edge lines (`a --> b --> c`)
- reports what it cannot read instead of dropping it: such lines land in
MermaidParser.unparsed_lines and raise a MermaidParseWarning, or a
ValueError under the new strict=True
- recognizes comments, subgraph/style/classDef directives and the legacy
`graph` keyword, so valid Mermaid does not trip that new warning
Node shapes are now one table driving both the node and endpoint patterns.
LLMConverter routes its three parse sites through _parse_mermaid, which
refuses output holding no valid diagram instead of returning an empty one.
Claude-Session: https://claude.ai/code/session_01VipiLaG4xy7WctqY9w2475
Both real-API tests ran on mere presence of OPENAI_API_KEY, so any developer or agent with a key exported silently made non-hermetic, network-dependent, cost-incurring gpt-4o-mini calls -- and inherited their model drift. Require an explicit opt-in instead. CI behaviour is unchanged: it sets neither variable, so both tests were and remain skipped there. Claude-Session: https://claude.ai/code/session_01VipiLaG4xy7WctqY9w2475
- license: the [project.license] table carrying the non-SPDX value "mit" becomes the inline SPDX string license = "MIT". No License:: trove classifier is added alongside it -- PEP 639 rejects that combination. - authors: was empty, so PyPI showed no author. - classifiers: none were declared; the 3.10/3.12 pair mirrors the CI matrix. - .editorconfig: added, copied from the fleet template. - ci.yml: dropped two stale migration comments. No setup.cfg exists, and PYPI_PASSWORD is provisioned -- Publish has already succeeded. Repo metadata set out of band: homepage -> the gh-pages site, plus the seven topics mirroring the pyproject keywords. Claude-Session: https://claude.ai/code/session_01VipiLaG4xy7WctqY9w2475
This was referenced Jul 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
MermaidParserdiscarded any edge line it could not match, without a word. Alive gpt-4o-mini call produced this:
check_user --|Yes|--> show_dashboardis invalid Mermaid — the valid form ischeck_user -->|Yes| show_dashboard. The parser dropped both of those lines,and the first line fared no better: an edge line carrying inline node
definitions failed the edge patterns, fell through to the node parser, which
matched whichever shape came first in its pattern list and threw the rest away.
Net result — the parser returned this, reporting nothing wrong:
check_user)check_user{Is user authenticated?}was gone.show_dashboardandshow_loginwere typedSTART. A diagram that should contain a decisioncontained none, and nothing anywhere said so. This is how model drift becomes a
silently wrong diagram rather than an error.
This is not hypothetical drift: it is exactly why
test_llm_converter_real_api_with_decisionfails today(
assert len(decision_nodes) >= 1→assert 0 >= 1) on any machine withOPENAI_API_KEYexported. TheA[Start] --> B[End]idiom — the most commonMermaid form there is — was equally affected; the pre-existing
test_llm_converter_code_block_cleanupused it and passed anyway, because itonly asserted
validate(), which an under-populated diagram satisfies.The fix
ij/parsers/mermaid.py:edge line are kept and typed correctly.
ARROW|label|form: the invalid-- |label| -->drift, plus the validMermaid alternatives
a -- label --> banda -. label .-> bwhich werealso being dropped.
a --> b --> c) yield one edge per link. Previouslyonly the first link survived.
MermaidParser.unparsed_linesand raised as aMermaidParseWarning; the newkeyword-only
strict=Trueturns it into theValueErrorthatparse'sdocstring has always promised but never raised.
%%),subgraph/end/direction/style/classDef/click/linkStyledirectives, bare node ids, and the legacy
graphkeyword are now recognizedrather than falling through.
instead of two hand-maintained pattern lists.
ij/converters/llm_converter.py:convert,refineandconvert_with_examplesall route through a new_parse_mermaid, which refusesoutput holding no valid diagram (with the raw model output in the message)
instead of returning an empty one.
MermaidParseWarningis exported fromijso a caller can escalate it with
warnings.simplefilter("error", ...).Confirmed against the live API: with this fix,
test_llm_converter_real_api_with_decisionpasses against real gpt-4o-mini —the root cause was the parser, not the model.
Tests
priv test-dependents ij)Eight new hermetic tests (no network). Six in
tests/test_parsers.pypin:inline node definitions surviving; the
--|label|-->normalization (assertedunder
simplefilter("error"), so a dropped line fails the test); the-- label -->/-. label .->forms; chained edges; the warning andunparsed_lineson genuinely unreadable input;strict=Trueraising; and aguard that a valid diagram full of comments, subgraphs and style directives
produces no warning at all. Two in
tests/test_llm_converter.pypin theend-to-end drift case (the captured model output above, asserting the DECISION
node survives — the hermetic stand-in for the live test) and the refusal of
unusable output.
Live-API tests are now opt-in
Both real-API tests were gated on mere presence of
OPENAI_API_KEY, so anydeveloper or agent with a key exported silently made non-hermetic,
network-dependent, cost-incurring calls. They now require an explicit
IJ_RUN_REAL_API_TESTS. CI behaviour is unchanged — it sets neither variable,so both were and remain skipped there.
Metadata pass
license: the[project.license]table with the non-SPDX value"mit"becomes the inline SPDX string
license = "MIT". Verified in the builtwheel:
License-Expression: MIT,Metadata-Version: 2.4. NoLicense :: OSI Approved ::trove classifier is added alongside it — PEP 639rejects that combination.
authors: was[], so PyPI showed no author.classifiers: none were declared. The 3.10/3.12 pair mirrors the CI matrix..editorconfig: added, byte-identical to the fleet template.ci.yml: dropped two stale migration comments (nosetup.cfgexists;PYPI_PASSWORDis provisioned and Publish has succeeded). No behaviouralchange.
the seven topics mirroring the pyproject keywords. Description was already
correct and was left alone.
Deliberately left out of scope
LLMConverter. The parser fix plus a hardfailure on unusable output covers the observed drift; a retry loop is a
bigger design question (retry budget, prompt repair, cost) and belongs in its
own PR.
a --- b,a -.- b) are still unsupported — but theynow warn instead of vanishing.
ij.__version__is"0.2.0"while pyproject and PyPI are at0.1.5.Pre-existing drift, untouched.
testpaths = ["tests"]means the wads CIinvocation collects only
tests/, soij/'s doctests are not executedanywhere. Two are broken or non-hermetic:
LLMConverter.refine's raisesNameError: name 'converter' is not defined,and
ij/export/image.py's writes adiagram.pnginto the repo root. Changingtestpathswould change what the gate collects, so it is left for a separatedecision.
https://claude.ai/code/session_01VipiLaG4xy7WctqY9w2475