Huge memory leaks in Rickshaw

[mkalinkin]

Hello,

We are developing a data analytics application with web visualization. We need to visualize many html charts in real time (e.g., several dashboard with 10-20 line charts having 300 points and updated 5 times per second).

We found that our needs are more or less good served by d3 rickshaw library. Among many other javascript libraries, it seems having best (or nearly best) performance

However, we also detected big memory leaks. Below are the details.

Our code is here: http://depositfiles.com/files/igs2z1o36

rickshaw-master\examples\test_update.html - plotting generated data in real time. Sliding window is organized manually. Memory leak for currently used parameters is ~ 8 Mb/sec.
rickshaw-master\examples\test_update_series.html - plotting generated data in real-time. Sliding window is organized via Rickshaw Series addon. Same memory leak.

In test_update.html, charts are updated using following lines of code:

function updateChart() {
for(var g = 0; g < charts; g++){
yValues[g] += rnd(); data[g].shift(); data[g].push({x: xValue, y: yValues[g]});
}
xValue += 1;
graph.update();
// console.log(yValues)
}
setInterval(updateChart, interval);

While debugging, we have revealed function render (Rickshaw.Graph.Renderer.js:75), where current svg:path (one for each chart) is deleted и and new path with new data is added. If comment that part of code, memory leaks disappear.

In Crome webdevtools dump we found that there are a lot of remaining elements "SVGPathElement" (their amount increase linearly with time). Most part of them seems to be outside DOM (we tried to delete elements by remove() function, but it seems that still somewhere remain links to these elements. Besides of grow of SVGPathElement amount, we observed grow of amount of some arrays (with similar dimension as SVGPathElement ). This might lead to the conclusion, that somewhere links to the arrays with previous data are stored causing memory leaks.

However, despite this considerations, we failed to locate exact reason of the memory leaks and find a solution to this problem.

Best regards,
Mikhail Kalinkin.

[drspin]

I wrote a 'load' tester, generating about 500 graphs all updating in 'realtime' and Chrome will not release memory, however, it seems ok in Safari. I'll be interested to see if you have similar findings.

[mkalinkin]

Hello,

we have done additional investigations and found, that huge leaks occur only in Chrome (ver. 23.0.1271.97). We didn't detect leaks neither in Opera or Firefox, nor in other Webkit-based browsers (e.g., Maxhton).

It seems, that Chrome (or at least the latest version of Chrome) has poor Garbage collector.

[mkalinkin]

We finally resolved the problem by... upgrading Chrome to ver. 24 beta. Actually, in previous version (22, 21, 20, etc) there were also no memory leaks.

It seems, that particular version 23 has poor garbage collector.
Therefore, this was a bug in chrome ver. 23, not in Rickshaw.

[mkalinkin]

Well, basically in Chrome ver. 24 beta memory is still leaking... Now much slower - 1 Gb leakage for 5 hours.

[aburtsev]

During the research it turned out, that memory leak happens in RickShaw due to the function _styleSeries (Rickshaw.Graph.Renderer.js: 96-105).
More precisely, the problem was in JS-function setAttribute() (Rickshaw.Graph.Renderer.js: 101-103). If one replaces function setAttribute() by, for example, jQuery function css() memory leaks disappear.
Why it's happening remains unclear.
Hope it will be useful for someone.

[bhiggins]

My guess is that this is due to this Chrome bug: https://code.google.com/p/chromium/issues/detail?id=172221

[arg-technoholic]

Hi,
I am trying to plot around 200,000 (0.2million) points with X axis representing the time stamp (with resolution of milliseconds) and the Y axis representing the data.Unfortunately, the script hangs in every browser I have tried(Firefox and Google Chrome latest versions). I am not sure what is causing the problem. I am not even updating the data or the graph dynamically with time.
I just need the static version. But still there appears to be an issue of memory leakage with so many data points when I run the script on a pretty latest hardware. When I reduce the number of data points to around 5000, everything runs perfectly fine. Does anybody know what could be the reason and any solution or workaround for this problem?

I would really appreciate any help.
Thanks.

[timelyportfolio]

This is simply too many points for any svg based library and not an issue specific to Rickshaw. If you plan to plot this many points you will need to use a canvas based library such as dygraphs, use d3 canvas renderer, or something that uses webgl.

[leeoniya]

https://github.com/leeoniya/uPlot