BUG: Fix mermaid.ink visual comparison test sensitivity to environment#99
Merged
Carwyn Pelley (cpelley) merged 1 commit intomainfrom Oct 30, 2025
Merged
Conversation
Rob Purvis (mo-robert-purvis)
approved these changes
Oct 30, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Opted for simply checking that images are successfully returned by mermaid.ink and that such files are non-zero in size.
Can't justify adding and maintaining code to DAGrunner testing for handling tolerant image comparison (svg/png/jpg).
Image generation is not operational or even part of the visualisation UI so doing anything more than checking that mermaid.ink actually returns a file cannot be justified.
Background
Mermaid cannot return consistent results since it works by dynamically generating a users diagram -- floating-point value precision differences leading to logic differences with layout.
Even svg element IDs are themselves allocated. We can enforce a consistent seed but this has its own implications on svg support.
Issues
Note
Markdown file (.md) generation (which we do care about) continues to be fully tested (i.e. content verified against reference file).