Organize Mermaid

[Yash-Singh1]

I think that MermaidJS is a great idea, a language to write diagrams, but this repository needs some organizing because I have seen dead documentation and inconsistent code-style in many places.

Ideas

These are some ideas that I came up with to make mermaid a better project.

• Enforce Code styling
  I have seen the code having inconsistent code style in many places
  • Add a GitHub Action
  • Add a lint:fix script (enable eslint fix with prettier-plugin-jsdoc and eslint-plugin-jsdoc #2471)
  • Lint all files
    • Note that for the HTML files, the HTML will have to be escaped to prevent parsing errors and messed up the formatting
    • Use eslint-plugin-markdown (lint markdown : enable eslint-plugin-markdown #2530)
    • Use eslint-plugin-html (lint html : enable eslint-plugin-html #2531)
    • Use prettier on all files (not only src/**/*.js)
      • Many files of many types have inconsistent prettier code-style
      • Use prettier-plugin-jsdoc (enable eslint fix with prettier-plugin-jsdoc and eslint-plugin-jsdoc #2471)
  • Add a commit hook using Husky (Add a pre-commit lint hook for staged files. #1878) (setup lint-staged and husky #2522)
  • Enforce .editorconfig is followed using linters
  • Use stylelint
  • Use eslint-plugin-jsdoc https://npmjs.com/package/eslint-plugin-jsdoc (enable eslint fix with prettier-plugin-jsdoc and eslint-plugin-jsdoc #2471)
    • When implementing this remember to turn off conflicting rules with prettier-plugin-jsdoc
• Documentation
  • Synchronize README and docs/README (see https://github.com/Yash-Singh1/eslint-plugin-userscripts/blob/main/.github/workflows/readme-in-sync.yml)
  • Synchronize CHANGELOG and docs/CHANGELOG
  • Merge flowchart docs in diagrams-and-syntax-and-examples and at root of docs and move all diagram documentation to diagrams-and-syntax-and-examples
  • Remove theme_themed.css
  • Remove redundant example and code (see Use monaco-mermaid for syntax highlighting and many syntax highlighting improvements #2449 (comment)) (Use monaco-mermaid for syntax highlighting and many syntax highlighting improvements #2449)
  • Group images in landing inside a new img folder
  • Use markdownlint so the documentation is more maintainable
  • Grammar and casing in multiple files
  • Finish up n00b-advanced.md
  • Reference and update the upgrading.md and breakingChanges.md
  • Search for and remove dead documentation and images
  • Keep syntax and docs in sync (there are many parsing features that are in the code but not in the docs, e.g. Add a milestone to docs #2340, Document curve interpolation #2275)
    • Maybe make it necessary to document changes to syntax
• Metadata
  • Update descriptions in package.json
  • Actually install defined Husky Hooks (setup lint-staged and husky #2522)
  • Move todo.md somewhere else
    • I understand it might be useful for personal notes, but putting that in GitHub projects, or making it more formal as a ROADMAP.md would be better
  • Mark dist/* as linguist-generated
  • Have more specific issue templates (Use issue templates and add diagram, theme and syntax proposal issue forms #2739)
    • Diagram Proposals
    • Syntax Proposals
    • Theme Proposals
  • Don't lock/unlock issues (Automatically lock/unlock issues on close/reopen #967) (Don't lock issues on close #2503)
    • This prevents users from being able to give their ideas against why the issues are closed because there are times when the issue is closed and then another idea is brought up.
  • Add some sort of deploy bot to allow preview of documentation on PRs (maybe Netlify because that is what we use in mermaid-live-editor)
  • Standardize commit message with https://www.conventionalcommits.org/en/v1.0.0/ (ci commitlint : setup commitlint #2529)
• Bulid
  • Don't push dist to GitHub
  • Group webpack files
• Source
  • Add JSDOC to helper functions to help contributors and add IntelliSense (Add jsdoc #2464)
  • Switch from Annex B escape and unescape to something else

[knsv]

Hey, @Yash-Singh1 you seems to have many ideas! Ping me on slack if you want to be a member of the core team.

[sidharthv96]

@knsv , Yash has been an active contributor in live editor too.. Happy to have him onboard the team!

[Andre601]

I'm considering working on some issue templates, perhaps even incorporating GitHub's new Issue forms for a more consistent issue formatting.

On another note would I personally suggest to enable and use Discussions in this repositories. While you can use issues for general questions would Discussions be better suited for that, especially thanks to features such as marking a reply as answer, which can help in building a healthy knowledgebase over time.
Issues could also be converted to discussions.

[financelurker]

My understanding of extending mermaid with custom diagrams is, that they need to be placed in the "diagrams" folder before building your own package... If that's right so far, I'd see a mechanism where you can develop your diagrams in dedicated npm packages and mermaid can make use of it (when at runtime the libraries are loaded within the same context).

Like, some kind of "diagram plugin" mechanism. What's your take on that one?

[knsv]

@financelurker thats a good idea. I think that we are reaching a point where the base package of mermaid needs to be split up. If you don't care about gantt charts why include and load them? I have been thinking about a lazy loading. This could be another way or even better a part of that setup.

There are some refactoring activities that need to happen first though. The code that initially handled two diagrams is not as clean when handling 10+ diagrams after"organic growth". We need to define what a diagram is so that mermaid can treat them all diagrams in the same way with a basic interface. This is high up on my prioritisation list and diagram organisation is a part of that.

[financelurker]

That sounds good to me. Tho, the magic in my opinion would be to extract all necessary information into an API package, which developers can import in their projects and code against, so that you can fully take care of quality assurance in your IDE or build process without the need to also pull the main library source itself.

[Yash-Singh1]

@financelurker thats a good idea. I think that we are reaching a point where the base package of mermaid needs to be split up. If you don't care about gantt charts why include and load them? I have been thinking about a lazy loading. This could be another way or even better a part of that setup.

There are some refactoring activities that need to happen first though. The code that initially handled two diagrams is not as clean when handling 10+ diagrams after"organic growth". We need to define what a diagram is so that mermaid can treat them all diagrams in the same way with a basic interface. This is high up on my prioritisation list and diagram organisation is a part of that.

Seems like a place where we can use a monorepo structure: https://yarnpkg.com/features/workspaces.

[brunolnetto]

It seems an opportunity to show a structure up I found very adequate to what @financelurker described as "code against". At the end of the day, a developer wants either to find something easy to use or to build himself based on some prior imported knowledge. Code-based documentation is less visual than its IDE counterpart. In the long run, I see mermaid as a responsive react ad-hoc web application.

[sakurawald]

mark

[financelurker]

This isn't really the right spot, but I'm trying to get the mermaid-live-editor running with my own custom diagrams - that's a real pain (okay, and I gotta admit that I'm fairly new to node/npm development too). How do you customise your diagrams and test it? Such a development environment seems pretty complex in my eyes. (but again: this probably isn't the right spot to mention this)

[brunolnetto]

I just developed shortly a parser from an artifact we had at my previous job to a .mmd file. Their diagrams have similar structure to mermaid flowcharts/state diagrams, so I gave it a try. I divided the diagram into styles for nodes and edges. There are some samples to both Flowbuild and Mermaid diagrams here.

[financelurker]

And one major thing I would wish to see for Christmas: one should not need to extend Diagram.js, mermaidAPI.js, utils.js and config.js to register new diagram types. Or did I try a wrong approach to introduce a newly customised diagram type?

[brunolnetto]

@financelurker I described my approach on my last post for diagram customization. The main configuration file is here: https://github.com/dot-quiver/dot-quiver-api/blob/main/utils/workflow/diagramConfig.json

[financelurker]

Thank you @brunolnetto - I'll give it a look.

[financelurker]

@brunolnetto did I get it right from having a look on your library: that library composes a Graph and translates it into a mermaid "graph TD" diagram? That's not what I was meaning with "customising diagrams". What I meant is the challenge of adding completely new diagram types to mermaid.

@knsv I've created a dedicated issue, since I think it's going off-topic here (?): #3061

[brunolnetto]

@financelurker You understand correctly. Which diagram do you mean to integrate to Mermaid?

[financelurker]

@brunolnetto I'm thinking of UML component diagrams (with lollipop notation), deployment diagrams, cmmn diagrams, etc. (some "custom diagram types") which are not supported in mermaid by now. "Hacking" these semantics into existing supported diagram types would be okay for the start but in the long run isn't feasible for my users (because semantics in diagrams matter).

[brunolnetto]

@financelurker I still do not comprehend if the desired diagrams are super-diagrams (categories of existing diagrams) or totally different ones.

[financelurker]

Well, I don't know either. But having a look at #177 there's a lot of demand of different and new diagram types. And cleaning up the mermaid architecture (described in my issue here: #3061 ) could make adding new diagram types more easy and streamlined (especially for people who don't want to understand the orchestration-layer of mermaid).

[jgreywolf]

@financelurker @Yash-Singh1 What is the current status of this item?

Should new issues be created for specific tasks remaining?

[Yash-Singh1]

I'm currently occupied with other projects, however, anyone interested in contributing is more than welcome to do so.

[whinee]

Hello, I would like to add to this (arguably) dead issue.

For background, I am making an electron app that takes mermaid.js config and an array of mermaid.js code from pipe, then renders those graphs, take a screenshot of the screen, then spits out the base64. this is due to the fact that mermaid.js cli requires a browser installed in a machine, while this solution does not. But I digress.

I have this python code that acts as an intermediary between that electron mess and the user, and I want that python code to validate the config for mermaid.js. And so, I got the default config then fed it into chatGPT, requesting to make a validation code. But you see, I saw a problem. For example, with the top-level key of theme, there are only 5 allowed strings: default, neutral, dark, forest, and base. You see, I also want to validate these types of things, but if I have not dug deep enough in the documentation, I would have not even knew of it in the first place.

So yeah, my suggestion is, along with the goals of improving the documentation, to document the config too.

This includes documenting:

• what each key on the config does (behavior, scope, etc)
• what each key's values are allowed, and its type

Sorry if this is not allowed in here, or if this comment is too messy, or insufferable, or anything.

[sidharthv96]

@whinee we have an issue #3397, but that hasn't been prioritized yet.

[whinee]

Ahh okay, thank you!

[xenkuo]

It's a good idea to have some re-organize or refactor to such a big project at current stage.
I know there's still new diagram type support request, but to be honest, mermaid should be the most comprehensive diagram tool now as far as I know. Current priority should be the stability, performance and user experience.
In terms of user experience, one humble suggestion is: seems like there's no any dark theme could properly display all that diagram types, there's some color design conflict in some diagrams, like the C4 Diagram.
I tried all the dark theme, no one can display it well, even some of them can work, if you apply same theme to other type, most likely quite bad display.

[Yokozuna59]

We're currently converting and standardizing diagram definitions, i.e., #4486 and #4501. There's an idea of creating a cli script that creates a skeleton for a new diagram following the new structure (it might be better to file an issue for better visibility, but the structure hasn't finalized yet). The discussed structure is in PR #4499, or at least that's what I'm following (because it hasn't been approved yet).

...there's some color design conflict in some diagrams, like the C4 Diagram.
I tried all the dark theme, no one can display it well, even some of them can work, if you apply same theme to other type, most likely quite bad display.

I guess that could be resolved by doing the file part of this #4499 (comment).

[jgreywolf]

Closing as per comment from @Yokozuna59