Can't render multiple contour plots in Django

[msaysell]

Whenever (at least) 2 people try to generate a contour plot in my application, at least one of them will receive a random error depending on how far the first person managed to draw.. ("unknown element o", "ContourSet must be in current Axes" are just two of the possibilities)

The following is a cut down test that can produce the error, if you try to load this page in 2 or more tabs at once, the first will render correctly whilst the second will produce an error. (Easiest way I found to do this was to click the refresh page button in chrome with the middle mouse button a couple times)

views.py

def home(request):
    return render(request, 'home.html', {'chart': _test_chart()})


def _test_chart():
    import base64
    import cStringIO
    import matplotlib
    matplotlib.use('agg')
    from matplotlib.mlab import bivariate_normal
    import matplotlib.pyplot as plt
    import numpy as np
    from numpy.core.multiarray import arange

    delta = 0.5

    x = arange(-3.0, 4.001, delta)
    y = arange(-4.0, 3.001, delta)
    X, Y = np.meshgrid(x, y)
    Z1 = bivariate_normal(X, Y, 1.0, 1.0, 0.0, 0.0)
    Z2 = bivariate_normal(X, Y, 1.5, 0.5, 1, 1)
    Z = (Z1 - Z2) * 10

    fig = plt.figure(figsize=(10, 5))
    plt.contour(X, Y, Z, 10, colors='k')

    jpg_image_buffer = cStringIO.StringIO()
    fig.savefig(jpg_image_buffer)

    array = base64.b64encode(jpg_image_buffer.getvalue())
    jpg_image_buffer.close()
    return array

home.html (just this one line is enough)

<img src="data:image/png;base64,{{ chart }}" />

N.B. This is a cross post from stackoverflow. I also don't think this is an issue with django but a bug with how contour plots are created.. this is just how I've managed to replicate it easily

[mdboom]

Are these two plots being rendered from separate threads, perhaps? The pyplot API doesn't in general support multithreading -- I suspect you are ending up with two plots writing to the same figure and putting things into an unsupported state. You can either use the OO interface, or spawn separate processes (using multiprocessing) to generate the figures.

If threading isn't involved, at a minimum you'll need to clear the figure after rendering it to prevent memory usage from increasing on every call.

[msaysell]

@mdboom - Thanks for your reply, I wouldn't be surprised if django was doing some threading, I did manage to "fix" it by using a multiprocessing pool like you suggested but this does appear to add an overhead as you'd expect for spawning a process...

pool = Pool(processes=1)
result = pool.apply(_test_chart)

What is the OO interface you mentioned?

Do you know if theres something different that the contour plot does in order to render itself other than other charts? (i.e polar and graph) these types are also on the same page and do not suffer from these errors. Otherwise, is there a way to specify the figure you are trying to plot to in plt.Contour?

Edit: I found the OO interface and this also seems to solve the problem without the overhead (or wierd looking workaround!). I'm still quite curious about what the difference is though

[mdboom]

The difference is that pyplot is used for convenient plotting at the commandline and keeps around global state. For example, when you say plt.figure() it adds the figure to a global list and then sets the "current figure" pointer to the most recently created figure. Then subsequent plotting commands automatically write to that figure. Obviously, that's not threadsafe. We recommend always using the OO interface from application code (rather than quick scripts and notebook/console tinkering) to avoid these sides effects.

[msaysell]

Thanks, for the great explanation. This always felt a bit odd the way I was writing it...