Big loading times when using many mapping rules

[okorolov]

Valid for 0.3.0 version

Greetings,

We are trying to generate somewhat a dynamic dashboard when the whole JSON structure of the panel is generated dynamically.
Think about a panel that displays ~350 nested square panels and each one should be assign with a metric and change the color according to the metric value.

The visual part works well. We are able to generate the xml part that draws squares.
The issue comes when we start linking squares with the metrics.

For each mxgraph object (square panel) we create a rule where we dynamically link a metric from Prometheus to the id of the mxgraph object.
In the end we have ~350 Mapping rules, as expected resulting in the JSON structure for ~16k lines.

Now if we try to load the page with Chrome the loading time might take ~5 minutes to refresh the panel.
In Firefox it takes ~ 40sec - 1minute.
During the loading time one of the CPU cores goes 100% loaded until the page is loaded completely.

Attaching performance recording from Chrome.

Attaching performance from Firefox

We tried to draw everything with draw.io with coloring the panels, but failed to import it dynamically. It seems that URL source for the XML part is still not working.

Could you, please, have a look in terms of optimizing loading speed with multiple items?

Thank you.

[algenty]

Hi okorolov,
thx for feedback
Wft 350 rules ! amazing ! it is possible to post xml graph after anonymize it ?
https://about.draw.io/tag/anonymize/
What I see in the second screen, the expressions take the most resources, have you Full name of id or juste an expression ? i have an idea to optimize it.

Best reguards
Arnaud

[okorolov]

Yes, we are generating our own IDs and then link them without regex.
We try to omit regex as much as possible, but the metric query in rule uses regex.
Still, when we used full query without regex it wasn't any better, unfortunately.

Attaching an approximate xml template.
White boxes are assigned with metrics eventually.

test_xml.zip

[algenty]

Nice,
i'll come back

[algenty]

Can you try version 0.4.0 snapshot
https://algenty.github.io/flowcharting-repository/ARCHIVES.html
a have added a little optimization : test absolute egality before regex

[okorolov]

@algenty , Thanks a lot for a quick feedback and a new version! Really impressive!

I can confirm that it is working faster now. The page still hangs, but I was able to refresh the panel with 224 metrics in ~12 seconds in Firefox.

It is working much slower in Chrome though, but still much faster than it was. ~77 seconds.

[algenty]

You're welcome,
more optimizations are possible but not now, need more non regression test.
Don't forget, flowcharting is a beta and young plugin (<3 months).

BR,
Arnaud

[algenty]

i close ticket, more optimization come in next versions

[algenty]

Hi @okorolov
Can you try last version 0.4.0, some optimizations.
https://algenty.github.io/flowcharting-repository/ARCHIVES.html
Fist load must be the same but the refresh must be faster

BR
Arnaud

[okorolov]

Hi @algenty
Will try it.

Thank you.

[algenty]

Hi @okorolov
Can u try last version and tell me if is more speed ?
See changelog at
https://algenty.github.io/flowcharting-repository/CHANGELOG.html

BR
Arnaud