Memory leak with surface plots

[samuela]

I'm working in a Jupyter notebook with a number of plotly surface plots. I've noticed that after 4-5 plots are generated, my computer slows down considerably. Upon review, I realized that Chrome was consuming all of my 16gb of memory. Closing the Chrome tab with the plots freed up 12gb of memory.

Here's some simple code that attempts to reproduce the error. Apologies in advance if it crashes your machine.

import numpy as np

import plotly
import plotly.graph_objs as go

import pandas as pd

plotly.offline.init_notebook_mode(connected=True)

for _ in range(50):
    data = [go.Surface(z=np.random.randn(50, 50))]
    layout = go.Layout(
        width=500,
        height=500,
        margin=dict(
            l=65,
            r=50,
            b=65,
            t=90
        )
    )
    fig = go.Figure(data=data, layout=layout)
    plotly.offline.iplot(fig)

Running it once is fine. A chunk of memory is used. No problem. But then if the same cell is reevaluated, removing the previous plots and replacing them with new ones, the memory only continues to climb. (You may need to rerun a few times to get the full effect.) We should expect totally memory usage to remain constant more or less. But things just keep climbing.

I'm running Chrome 53.0.2785.116 on Mac OS X 10.11.6.

[etpinard]

Thanks for the report.

Memory management for WebGL trace type has been long-standing issue for us. Referencing plotly/plotly.py#487 and #718.

Can I ask what version of plotly.py are you using? If you're using plotly.py < 1.12.6, I would recommend updating to the latest version. You should notice some improvements there coming from @monfera 's #724 and #726.

That said, even after updating, I suspect that some memory issues will remain (especially on pages with 4-5 WebGL graphs) . Unfortunately, fixing those issues will require a major changes in our WebGL code base which we planned in #949 but probably won't be ready for primetime before early 2017.

[samuela]

I'm using the plotly.py version 1.12.9.

Glad to hear that this is an issue on the radar and that there are potential fixes in the pipeline! IPython/Jupyter notebooks are highly popular with many plotly users, so I submit that this issue deserves a decent amount of prioritization.

[monfera]

Reproduced it in plotly.js:

    fit('memory leak test', function(done) {
        var tuple = require('@mocks/gl3d_surface-lighting.json')
        Plotly.plot(gd, tuple.data, tuple.layout)
        var i = 0;
        window.setInterval(function() {
            if(i > 8) return;
            Plotly.plot(gd, tuple.data, tuple.layout)
            console.log("iteration: ", i++)
        }, 1000)
    });

On an initial look, pickBuffers and calcdata accumulate to large sizes. We haven't yet shot for memory leaks in a loop like this and the gradual slowdown and halting of the rendering is confirmed.

[etpinard]

After a chat with @mikolalysenko , he thinks that perhaps the memory leaks are related to our typedarray-pool dependency where in some cases pool.free() isn't properly called.

[etpinard]

It's been a few years since we heard reports of memory leaks in surface traces.

I'll close this, but if someone subscribed to this ticket is still experiencing leakage in the latest plotly.js version please open a new issue!