Makie memory leak?

[LaurentPlagne]

The following MWE consumes more an more memory until crash:

using GLMakie,AbstractPlotting,ImageCore
function leak(img,i)
    fn=string(i)*".png"
    scene = AbstractPlotting.image(rotr90(img), show_axis = false);
    AbstractPlotting.save(fn, scene; resolution = (1920,1088), px_per_unit = 72)
    nothing
end

function nleak(nf)
    nx,ny=1920,1088

    for i=1:nf
        img = rand(RGB{N0f8}, nx, ny)
        leak(img,i)
        @info "Size of AbstractPlotting: $(Base.format_bytes(Base.summarysize(AbstractPlotting)))"
        GC.gc(true)
    end
end
nleak(200)

Status `~/Projects/video_grid.jl/MWE/Project.toml`
  [537997a7] AbstractPlotting v0.14.3
  [3da002f7] ColorTypes v0.10.9
  [5ae59095] Colors v0.12.5
  [e9467ef8] GLMakie v0.1.21

julia> versioninfo()
Julia Version 1.5.3
Commit 788b2c77c1 (2020-11-09 13:37 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i5-9300H CPU @ 2.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = "/snap/code/52/usr/share/code/code"
  JULIA_NUM_THREADS = 

[SimonDanisch]

Ok, I'm not 100% sure how exactly it's happening, but the screen is holding on to memory via some observable.
For save the screen is reused, so it will just stay the same screen and grow.
You can do this as a work around:

using GLMakie,AbstractPlotting,ImageCore
function leak(img, i)
    fn=string(i)*".png"
    scene = AbstractPlotting.image(rotr90(img), show_axis = false);
    AbstractPlotting.save(fn, scene; resolution = (1920,1088), px_per_unit = 72)
    empty!(GLMakie.global_gl_screen())
    nothing
end

[LaurentPlagne]

Hi Simon,
thank you for your support.
Unfortunately the workaround does not fix the leak on my computer...
Does it work for you ?

[SimonDanisch]

Yes it does as far as I can tell...
Base.format_bytes(Base.summarysize(AbstractPlotting)) stays the same...
You can also try close(GLMakie.global_gl_screen())! You also may want to check if you're on AbstractPlotting 0.14.4 and GLMakie 0.1.22

[LaurentPlagne]

OK, it works with close !

empty! makes no difference on my env:

Status `~/Projects/video_grid.jl/MWE/Project.toml`
  [537997a7] AbstractPlotting v0.14.4 `https://github.com/JuliaPlots/AbstractPlotting.jl#master`
  [e9467ef8] GLMakie v0.1.22 `https://github.com/JuliaPlots/GLMakie.jl#master`

Thank you again !

[JTreehorn]

I have the same issue but with different implementation, I don't use save at all but just want to display an image as it changes. my ultimate goal is displaying this thing "live" without running into memory problems. Any ideas? maybe use another package? imshow had no leaks, but was too slow with my implementation.

Platform Info:
OS: macOS (x86_64-apple-darwin19.6.0)
CPU: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-11.0.1 (ORCJIT, haswell)
Environment:
JULIA_REVISE_WORKER_ONLY = 1

using GLMakie
using Colors

function needs_mem_draw(imag, num_squares, square_size=200) 
	
	#requires as much memory as num_squares copies of imag
	
	image_size = size(imag)
	if square_size > min(size(imag)...) return 0 end
	
	f = Figure(resolution = (image_size[2],image_size[1]));
	
	GLMakie.Axis(f[1, 1], aspect = DataAspect(), xticks = [], yticks=[]); 
 
	display(f)
	
	for i in rand(1:min(image_size...)-square_size-1,num_squares) 
		wndw = view((imag) ,i:i+square_size,i:i+square_size)
	 	wndw .= 0.75
		image!(Gray.(imag));
		sleep(.005)
	end 
end

imag = rand(Float32,(1000,1000));
needs_mem_draw(imag,20)

using close(GLMakie.global_gl_screen()) returns the memory, but also closes the window and defeats my original purpose. empty!(GLMakie.global_gl_screen()) seems to have no effect on memory allocation.

thanks very much

[jkrumbiegel]

Don't plot an additional image in each frame, that adds them all up (not a "memory leak"). Plot one image with an observable and change that observable in each frame.

data = Node(randn(100, 100))
image(data)
# then you can change data
data[] = randn(100, 100)

[JTreehorn]

Thanks! And in your example above, it's not possible to get data to update by just changing an element, e.g. data[][loc] = val I haven't been able to get Node(data::Array) to trigger on these events (and push!(data[], val) doesn't work either). An array of Nodes technically works, but is a ridiculous solution to me when there are more than a handful of elements.

I can force an update with data[] = data[], but this takes too long.

Is there some simple way to trigger data without calling data[]= ...?

Also about the original code, calling image!() multiple times... if it isn't exactly a leak, are the old images still accessible? Is there a direct way to free the memory manually?

thanks again!

[SimonDanisch]

You can use the Sampler type, to only update exactly the data that changes:
https://github.com/JuliaPlots/GLMakie.jl/blob/master/test/glmakie_tests.jl#L34

Also about the original code, calling image!() multiple times... if it isn't exactly a leak, are the old images still accessible?

yes, you need to delete the old image:

image_plot = image!(axis, ...)
delete!(axis, image_plot)

I can force an update with data[] = data[], but this takes too long.

How long does it take with how much data?
This should be pretty fast up to ~2000x2000, so if it's slow, maybe its a bug?

[JTreehorn]

Thanks!
re: speed, the following 2k x 2k takes just over 5s for 25 calls of data[] = data[]. so ~< 5 frames/sec on my machine, or 200ms per call. For 1k x 1k i get ~18fps, 56ms/call.... Is that expected?

I feel greedy asking for more speed, but throw in some data processing upstream then even the 1k can become difficult to work with.

	imag=rand(Float32, (2000,2000))
        data = Node(imag)
        n = 25

	f = Figure(resolution = size(imag));
	Makie.Axis(f[1, 1], aspect = DataAspect(), xticks = [], yticks=[]); 
 	image!(data)
	display(f)

	for i in 1:n

		sleep(.001)

		data[] = data[]    	
		#notify(data)
	
	end

And thanks a lot for mentioning Sampler, I got the test to create the RamStepper object that contains the three images.

In my code, I can alter my image object associated w/ Figure by using Sampler, but the changes don't propagate the update to the image! associated with the Figure. Do I also need to use Sampler with UpdatableArray or connect! defined in ShaderAbstractions? I haven't been able to find an example on how to propagate the update to a display in real time

¡thanks again for your work & I'm curious to hear if that 200ms for data[]=data[] call makes sense.

[SimonDanisch]

Damn, seems like this kind of high performance usage of Makie has been slowly rotting away...
First of all, image with a float array hits a slow path, that also converts the sampler to a normal array, which is why your example is so slow and doesn't update with the sampler. You'd need to use Matrix{RGBAf0} for it to work...
If you use heatmap, it works much better, but there is a conversion overload missing to have heatmap work with sampler:

using GLMakie 
using GLMakie.ShaderAbstractions: Sampler

# Missing overload to have Sampler work
AbstractPlotting.el32convert(x::AbstractArray{Float32}) = x

imag = rand(Float32, (2000,2000))
data = Sampler(imag, minfilter=:nearest) # or linear for interpolation
n = 25
f = Figure(resolution = size(imag));
ax = Makie.Axis(f[1, 1], aspect = DataAspect()); 
hidedecorations!(ax)
tightlimits!(ax)
heatmap!(ax, data)
display(f)

@time for i in 1:n
    data[1:400, 1:400] = fill(0f0, 400, 400)
    yield()
end

[JTreehorn]

Wow. So using Sampler (correctly) has resulted in a >100x increase in frame rate to ~500fps in that case.

Damn, seems like this kind of high performance usage of Makie has been slowly rotting away...

use it or lose it?? :)

this has been really helpful; thanks for the help & the work.

[IanButterworth]

I've been trying to figure out why the MWE below isn't releasing memory.

Note that in this example x is unused.

julia> using GLMakie

julia> function foo()
           fig = Figure()
           ax = Axis(fig[1, 1]) 
           x = map(_->rand(2000,2000), 1:500)
           return fig
       end
foo (generic function with 2 methods)

julia> fig = foo()

The window opens and renders. During which ram usage goes up, then drops. It appears to know x is unused. All is good on this first run.

However, the next time foo() is run, the memory rises and stays up, even if you try

julia> fig = foo()

julia> fig = nothing

julia> GC.gc()

julia> close(GLMakie.global_gl_screen())

julia> GC.gc()

julia> GLMakie.Makie._current_figure[] = nothing

julia> GC.gc()

Though, if you do another foo() it seems to replace the memory. There's a slight bump and dip, as if the gc is running around the same time new memory is allocated.

I've tried to find global state that's being retained and nothing I've disabled has made a difference.

In a real world example for a complex plot, this appears to mean that the first run is good, but the 2nd is prone to segfault.

signal (11): Segmentation fault
in expression starting at /home/ian/Documents/GitHub/Foo.jl/test/runtests.jl:43
glDeleteBuffers at /home/ian/.julia/artifacts/a84cc58d5161b950f268bb562e105bbbf4d6004a/lib/libGLdispatch.so.0 (unknown line)
Allocations: 4434875135 (Pool: 4434166195; Big: 708940); GC: 3494
Segmentation fault (core dumped)

This kind of crash is quite painful given the slower TTFP.

[JTreehorn]

I think we may have the same issue... I start a window & run something on it. I can re-run my program to make a new live image. but if i do this more than a couple times (or sometimes even run it twice) i get a seg fault. The way i've lived with this is to manually close the window that opens to display my content after each run. I can run again and again if i close windows fastidiously

also i haven't looked at this code for months but i had the same issue

 # clear screen cache if any, create canvas and Makie objects for display (i thi
empty!(GLMakie.global_gl_screen())
close(GLMakie.global_gl_screen())

color_canvas=fill(RGBAf0(0,0,0.,1),size(pass_image_name));

canvas_sampler = Sampler(color_canvas, minfilter=:nearest)

f = Figure(resolution = size(color_canvas),backgroundcolor = :gray0);
ax = Makie.Axis(f[1, 1], aspect = DataAspect());
hidedecorations!(ax)
tightlimits!(ax)
image!(ax, canvas_sampler)
display(f)

[SimonDanisch]

This feels like there isn't any remaining open issue here anymore... If mistaken, please add an mwe we can test to see whats going wrong!