Sankey diagrams

[nirname]

📑 Summary

Adding new type of diagram - Sankey diagrams

Resolves #4492
I closed First PR in favor of this one, as latter has proper branch name and better description

📏 Design

Sankey diagram represents a flow from one object to another.
GoJS, as well as pages from D3 use csv for that:

source,target,value
Agricultural 'waste',Bio-conversion,124.729
Bio-conversion,Liquid,0.597
Bio-conversion,Losses,26.862
Bio-conversion,Solid,280.322

Instead of heading we expect keyword "sankey".
That is the basic idea.

Lets call these object 'nodes' and these transition 'links'/
We also need a possibility for tuning their attributes, and graph attributes as well, such as:

• link sort
• node sort
• coloring strategy for links (source, target, transition)
• graph alignment (left, right, width)
• node color
• node title
• node width
• node padding
• graph margin

📋 To Do

That is what I plan to do, not everything will have been done at the first approach.

• 📖 have read the contribution guidelines
• 🔖 targeted develop branch
• implement csv syntax
• implement renderer
• add demo
• allow a node to be wrapped in double quotes
• recognize floating point numbers as values
• graph with nodes but without flows works
• remove all comments from graph before parsing (check that this is done automatically, is it related to csv style?)
• squash some commits and write correct messages for all of them
• check what has happened with .lock (some deletions, probably undesired) and resolve package json
• check multiple links between 2 nodes
• fix conflicts in grammar
• 💻 have added unit tests
• fix width (part of the node is cropped at the right)
• allow to configure height and width
• 💻 have added e2e tests
• 📓 have added documentation

[codecov-commenter]

Codecov Report

Merging #4502 (afb979f) into develop (4570f82) will decrease coverage by 19.60%.
The diff coverage is 86.53%.

Additional details and impacted files

@@             Coverage Diff              @@
##           develop    #4502       +/-   ##
============================================
- Coverage    79.71%   60.11%   -19.60%     
============================================
  Files          137       53       -84     
  Lines        16296     9109     -7187     
  Branches       496       21      -475     
============================================
- Hits         12990     5476     -7514     
- Misses        3208     3633      +425     
+ Partials        98        0       -98     

Flag	Coverage Δ
e2e	?
unit	60.11% <86.53%> (+2.11%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...s/mermaid/src/diagram-api/diagram-orchestration.ts	30.23% <33.33%> (-9.06%)	⬇️
...ages/mermaid/src/diagrams/sankey/sankeyDetector.ts	75.00% <75.00%> (ø)
packages/mermaid/src/config.type.ts	100.00% <100.00%> (ø)
packages/mermaid/src/defaultConfig.ts	93.48% <100.00%> (-0.33%)	⬇️

... and 130 files with indirect coverage changes

[Yokozuna59]

I'm not a member of the mermaid project, but just to put my two cents in:
 
First of all, thanks for your contribution! Adding more diagrams would really help the mermaid project! I'm aware that you haven't been getting much support or guidance from the community while working with the new diagram, but as you can see, they were busy with other issues.

---

• add cypress docker support

I'm not really sure that this should be in this PR; it might be better to have it in a separate PR if necessary. Because adding Docker would mean it should be continuously updated and maintained, I'm not sure if that is a top priority. (correct if I'm wrong)

It might be better to commit with a meaningful message for a better change log; if someone wants to see changes made in a commit, it would be hard with a Wip message.

I haven't seen the code yet, but you could follow new guidelines for diagram definitions in discussed in this PR #4499, info diagram is an example applying those rules.

[nirname]

That is how graph looks at the moment

[nirname]

@Yokozuna59 thanks for reply. Have taken all you'd said into consideration

[sidharthv96]

@nirname I haven't been much active in Slack. Please try to leave questions and comments related to the PR here, so everyone following the PR has the complete view and can pitch in with suggestions.

I'll also curate a list of new diagram PRs, so people who want to add new diagrams can refer to them, see discussions, and understand why we made some decisions.

The current diagram looks good btw. 🚀

[nirname]

@sidharthv96 Thanks a lot

[nirname]

Syntax

I have not received any feedback on syntax yet, there is a topic about that #4492, but it does not really matter for me where it is being discussed.
Probably, this feature is not so popular (as many other diagrams), otherwise there would be more people giving feedback on that topic. But this is a good starting point for me.

As for the syntax there are several options (mainly 2). One inspired by dot, and the other by researching on the topic.

Arrow style

(which is the current one)

"Coal"[color=black] // node attributes
"Coal" -> 343.1235 -> "Energy" -> "42.34" -> "Heating" [color=blue] // link attributes

Pros:

• Allows chaining, that is reasonable and descriptive to my mind
• We can apply same attributes for many links at once (or not, see attributes in place)
• Relatively descriptive
• Ordering of nodes can be setup depending on their order of appearance
• Duplicating nodes and redefining their attributes may be easily implemented

Cons:

• may be too complicated and entangled, there is no style recommendation on how to write that
• we need separate place to specify node attributes (that is an advantage in case of ordering, but probably a disadvantage from different perspective)
• we cannot define link attributes separately (see attributes in place below)

We can also take a step further and allow defining attributes in place:

"Coal"[color=black] -> 343.1235[color=red] -> "Energy"[color=blue] -> "42.34" -> "Heating" // link and node attributes are mixed

ps
Initially I had an Idea of something like

a -> {
  10 -> b
  12 -> c
} -> 30 -> d -> {10, 20}

but that led to ambiguous behavior so I rejected this option

CSV-style

that is quite often used for that purposes, simply because data is stored that way:

Coal, Energy, 343.1235
Energy, Heating, 42.34

Pros

• widely used
• uniform (3 columns always), can be formatted in columns (using spaces)
• can be copied and paste directly CSV, and extended hereafter
• It can be even more descriptive, because we see source and target nodes one after another, thereby we can easy perception of what is going on

Cons (well, I dont see any significant):

• May be it is a little bit excessive, but the worst case scenario for arrow style result in the same length of textual representation code

---

Fusion

I also think about something heterogeneous, comma based:

a[label="Coal"], b[label="Energy"], 10[color='red']
a[label="Coal"] {
  b[label="Energy"], 10
  e[label="Forging"], 20
}

or arrow based:

a[label="Coal"] -> b[label="Energy"] = 10[color='red']
a[label="Coal"] {
  b[label="Energy"] = 10[color='blue'] // node attributes, link attributes
  e[label="Forging"] = 20[color='red']
} [opacity=50] // common attributes for all links from a to b

Allows pretty much everything that 2 previous styles do, as well as grouping

[sidharthv96]

2 more test cases and we're done!

[nirname]

This is last commit message:

Added cypress container (from their official image),
may be reconsidered later to choose our own
The only minor problem is node JS version mistmatch
package.json 18.16.0
docker-compose mermaid 18.16.1
docker-compose cypress 18.16.0

Host option in cypress docker container must be removed
in favor of possible configuration option.
http://localhost:9000 are currently hard-coded, that is bad

Updated ./run script with better documentation
and added some styles too it as well

Started sankey.spec.js as an example (will be reworked)

There was a discussion about the need of adding cypress docker support in this PR, firstly I agreed that it could be postponed but now I've reconsidered my views and would rather stick to my initial proposal. We need this, because it is impossible to write tests without running them locally, so that integration is tightly couple with this PR, because no one before me, obviously, was developing it in Docker

[sidharthv96]

@nirname can you add some more imgSnapshotTests in a different PR?

[sidharthv96]

Thanks for the great work @nirname !

Will try to get the release out ASAP.

[sidharthv96]

imgsnapshot is a visual test. It'll compare the rendered image on every run with an approved base image. So we don't have to write any validations manually, and it'll test the entire parsing and rendering stack.

…

On Mon, 3 Jul, 2023, 12:37 am Nikolay Rozhkov, ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In cypress/integration/rendering/sankey.spec.ts <#4502 (comment)>: > + it('should render a simple example', () => { + imgSnapshotTest( + ` + sankey-beta + + sourceNode,targetNode,10 + `, + {} + ); + }); Could you explain how this test works? What is tested there exactly? What is expected value? — Reply to this email directly, view it on GitHub <#4502 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACRVEVOZIV6AZNASM3OMXI3XOHBHFANCNFSM6AAAAAAZJZX6UY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[nirname]

@sidharthv96 I have noticed, that this ImgSnapshot test fails sometimes, because cypress discovers some differences in almost 2 identical images (and they really are). I don't know why, If I face the issue I'll attach screenshot. The diff is somewhere near texts, although texts aren't changed, but it still manages to find some imperfect pixels.

By the way, thanks for explanation.

[aloisklink]

Hi @nirname, sorry I didn't get a chance to look at this PR before it was merged. The new Sankey diagrams look really good!

TypeScript enums have a bunch of issues, so I've made a PR changing these lines of code to remove the enum, see #4580.

Please feel free to review this PR with your feedback!