Visual observations memory leak on Unity side (0.4)

[ngutten]

I'm running experiments on an environment with visual observations on Ubuntu using the Linux Unity Editor Beta, ML-Agents 0.4, and control via a Jupyter notebook. The memory footprint of the environment process seems to be increasing linearly with time when I have the visual observations enabled, and is constant when I disable them.

Any ideas why this might be? The environment itself does very little - it's basically just some rigidbody physics where agent actions control a camera and tries to follow objects.

It may or may not be related, but when I close an environment using close() and then try to open a new one, rather than using reset() to start a new trial, I get a OSError: Broken pipe and the new environment starts to run in real time without waiting for act() calls.

[mbaske]

The problem appears to be in Agent.cs ObservationToTexture method.
var tex = new Texture2D(width, height, TextureFormat.RGB24, false);
creates a new Texture2D at every observation step. Texture count in the profiler keeps constant once I replace the local variable with an instance property.

[ngutten]

I'm not entirely sure I'm doing this the same way (I tried a few variations involving removing the 'new Texture2D(...)' line) but I still seem to be getting increasing memory usage over time. For what it's worth, I'm not seeing increased memory usage when I run the environment directly in the editor - just when I run it via the binary directly.

Edit: Scratch that, I've got it working now by passing a pre-allocated Texture2D to ObservationToTexture in place of tex, and the memory usage seems to be constant.

[awjuliani]

Hi @ngutten and @mbaske,

I have created a hotfix PR that addresses this issue: #935. Please take a look and let me know if it solves your issues. I ended up doing something very similar to what you both described, but there is some additional complication, since the possibility for textures of different sizes need to be taken into account. Hopefully it works just as well in your specific use-cases.

[mbaske]

@awjuliani

if (width != texture2D.width || height != texture2D.height)
{
         texture2D = new Texture2D(width, height, TextureFormat.RGB24, false);
}

Not entirely sure, but my guess is that
texture2D.Resize(width, height);
would be the safer choice here with regard to avoiding a memory leak. Thanks for the quick fix!

[awjuliani]

@mbaske, Thanks for pointing this out! Just changed it to a Resize(). I actually wasn't aware that resize was possible. Nice to know 😃

[awjuliani]

Just merged in the PR. I will close the Issue for now, but if other problems with the fix arise, feel free to re-open it.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.