Linux-only, texture-related memory leaks

[VanessaE]

Let's just call it insane amounts of memory used by the engine, but only on Linux builds. At least two others have confirmed this same issue, and that it does not occur with equivalent Windows setups.

My system is based around an AMD Phenom II X6 1055T (2.8 GHz) with 16GB of RAM. The GPU is an ATI HD6870 with 1GB. There's an extra 16GB of swap residing on a fast SSD, the same disk from which Minetest and its resources are loaded.

Rather than go into a long-winded dissertation about it, below is my pissed-off rant from #minetest-dev (a continuation of a discussion that started in #minetest).. I am so upset here because this problem has persisted for far too long.

[06-17 19:38] <VanessaE> ok, 10+ GB in Linux to use HDX 512px on my Creative server, versus 5.2 GB for Jordach to do the same?  Clearly a memory leak exists in Linux builds.
[06-17 19:38] <VanessaE> this needs fixed.  NOW.
[06-17 19:38] <VanessaE> this kind of shit has to stop.
[06-17 19:40] <Jordach> VanessaE, $5 it's that no extrude patch being leaky
[06-17 19:41] <VanessaE> Jordach: try it and see, then
[06-17 19:42] <Jordach> VanessaE, my client did not use that patch
[06-17 19:42] <Jordach> so recompile without it
[06-17 19:42] <Jordach> (mines fresh from git whenever sfan5 last compiled)
[06-17 19:43] <Jordach> cb3b42e *
[06-17 19:43] <VanessaE> Jordach: it's always used horrendous amounts of RAM for me, since long before I started using that patch
[06-17 19:49] <asl> so now people start caring about the leak
[06-17 19:54] <VanessaE> asl: explain
[06-17 20:00] <asl> i talk about some memory leak in minetest a few days/weeks ago,  after running minetest for awhile, it slowly hogging the ram and before you know it, it use up all the memory and freeze the whole os, exiting to the menu doesn't free the memory, only closing minetest then the memory would be free
[06-17 20:01] <VanessaE> huh.
[06-17 20:01] <asl> it slowly hog more and more memory as more block/chunk of the map get loaded and that's is the only lead i got
[06-17 20:01] <VanessaE> [06-17 20:00] <VanessaE> Jordach: also, 6.133 GB with HDX 256 at Creative's spawn after pitching and panning my view around and letting every last bit of map around me load up
[06-17 20:01] <VanessaE> [06-17 20:01] <VanessaE> no preload, all settings enabled except for parallax, have not opened the inventory yet.
[06-17 20:03] * Jordach has quit (Ping timeout: 244 seconds)
[06-17 20:42] <VanessaE> Ok after enabling some swap, Minetest uses 14.342 GB virtual, and somewhere in the 12 GB resident range, with HDX 512, just standing here looking around at the spawn on my Creative server.
[06-17 20:42] <VanessaE> THAT'S FUCKING INSANE.
[06-17 20:43] <VanessaE> the G*d damn pack is only ~420 MB.
[06-17 20:43] <VanessaE> and I am NOT using preload visuals
[06-17 20:43] <VanessaE> so there's no way in hell it should be using this much, this soon.
[06-17 20:43] <VanessaE> all I'm doing is standing at the spawn, looking around.
[06-17 20:43] <VanessaE> FPS:  3-4.
[06-17 20:45] <VanessaE> and that's with the view range turned all the way down to 5m.
[06-17 20:45] <VanessaE> what is the problem here?
[06-17 20:50] <VanessaE> hello?
[06-17 20:52] <VanessaE> and yes, I'm ranting
[06-17 20:56] <Sokomine> whatever the problem is, hope it can be fixed. less good hardware may profit from such fixes as well
[06-17 21:03] * restcoser has quit (Quit: Leaving)
[06-17 21:04] <VanessaE> well if Jordach can do the same server with less than half the usage, and roughly equivalent CPU and graphics power, but under Windows, then surely it can
[06-17 21:04] <VanessaE> but does anyone care?
[06-17 21:04] <VanessaE> or is everyone obsessed with the server?
[06-17 21:04] <VanessaE> (except RealBadAngel, whose focus is on shaders)
[06-17 22:38] <VanessaE> if I turn off aniso/mipmap, it barely even touches the RAM usage (not enough to even measure).
[06-17 22:38] <VanessaE> (and gives me back a ton of framerate, probably because of GPU overload).
[06-17 22:39] <VanessaE> if I turn off normalmaps, that gains back about 2.5GB, but still leaves me using over 9.5 GB of RAM.
[06-17 22:39] <VanessaE> I simply can't use any less than that, and that's still almost twice the usage Jordach was getting with windows, and he was using normalmaps.
[06-17 22:39] <VanessaE> someone has to explain this.
[06-17 22:45] <Exio4> i've been trying, i seem to end with the resource usage as a lot vs the win build (using latest's sfan5 build at the moment, cb3b42e) against latest git
[06-17 22:45] <Exio4> with same settings it ended being as much as 375% (3.75x :P)
[06-17 22:49] <VanessaE> ^^^ that is, his experience mirrors mine, only worse.

[Megaf]

Well, we got memory leaks on server side too. I believe the memory leak in your client might be just a driver issue.

[Holyphoenix]

I too use more ram running it on top of Arch as opposed to Windows which kind of sucks. But my ram usage isn't anywhere near yours.

[nerzhul]

@VanessaE ping. Can you tell me if it's always here ?

[VanessaE]

The issue still exists. It isn't as bad as it used to be, but it's still there - HDX-512 will take anywhere from 6GB to as much as 11GB of RAM.

[Megaf]

why was this closed? It still leaking badly here.

[VanessaE]

Because numerous leaks and other bugs have since been fixed in the engine. A new report should be opened after the current round of bug-hunting is done (i.e. let the dust settle a bit).

[Megaf]

Ok, I will open a bug report when I have more time then. Here with the default textures it sometimes eats up 4 GB for RAM in about 20 to 30 seconds.