Klipper Estimator does not handle running out of memory well on a RasPi 4 , 4 GB

[psiberfunk]

Tried this out on my RasPi 4, 4 GB (neat program by the way!) and my PC. While certain files work fine on both, for larger files, only my Windows PC is able to handle them. For files somewhere between 20-80 MB of gcode, I start getting failure messages like the following during execution of estimate:

Sequences:
memory allocation of 805306368 bytes failed
Aborted

Strangely, my memory utilization doesn't get anywhere near my full 4 GB before this happens.. i'm not quite sure why.. perhaps this is a malloc() failing in anticipation of needing more than 4 GB? It would be good to realize that on ARM platforms like the Ras Pi that memory is going to be limited, and some sort of caching-to-disk (yes, slower of course) as a slower fallback would be appropriate if the memory-hungry nature of the program can't otherwise be made more efficient.

Since Klipper itself of course is pretty memory efficient and happily runs without complaint with much less memory, I would hope there's ways to run Klipper_estimator in a more memory efficiency manner than is run today ?

[psiberfunk]

I'm not familiar with rust, but my guess here is that one approach that would scale reasonably well would be to analyze a "layer at a time" in-memory in cases of not having enough to take the whole GCODE file all at once, which is, I suspect, the issue from browsing the code briefly. I'm sure there's other ways to chunk it up as well, but I think in most cases a single layer should fit in the memory of most machines you'd want to run this on, even the relatively memory limited smaller Raspis.

[dalegaard]

Estimation mode indeed relies on loading the entire file, so it can't reasonably run on a Pi currently. Post processing mode works fine though, as it runs in constant memory.

I don't really see the use case for estimate on weak machines as it's mainly a tool for optimization. Change something, run estimate, see how time changed, repeat. Since the pi is so slow it'll take a long time for each run, and so isn't particular useful.

For large files on Pi, a simple work around is enabling a swap file.

At some point estimation mode could definitely be improved to aggregate incrementally, the dataset in memory should still be reasonably small.

Best regards,
Lasse

[dalegaard]

I just released a v1.1.0 where estimate now runs in constant space. This should allow it to work on the Raspberry Pi as well, but of course it'll still probably be a bit slow :-)

Best regards,
Lasse

[psiberfunk]

Thanks ! @dalegaard : Slow is relative I guess.. I have 2-3 hour prints estimating in 5 seconds on my RasPi 4 while printing!