Out of memory

[hasse69]

Running a Windows 2008r2 server with a virtualbox guest (Ubuntu 14 server) running rar2fs.
Plex media server is running on the windows 2008r2 server and using the rar2fs server to unpack my movies and series. 

Having issue with rar2fs getting following message after X amount of data transfered: 

rar2fs login: Out of memory: Kill process 1046 (rar2fs) score 858 or sacrifice child 
Killed process 1046(rar2fs) total-vm:2409464kB  anon-rss:1760124kB, file-rss:0kB

When this happens i must run umount -l and remount it again.

I'm running 2 mounts from rar2fs, one with movies (with --seek-length=1) that has no issue so far. 
The other one(with the issue) does not have --seek-length=1 since it can't find all the rar files with it.

Tried increasing the memory from 512mb to 2gb, all it did was postpone the error bit longer - still happens thou. 

Original issue reported on code.google.com by emil@vimmelman.se on 2014-09-05

[hasse69]

Can you please tell what version of rar2fs you are using.

According to the error message rar2fs would have allocated ~2.5GiB of virtual memory.
It obviously looks like a leak of some kind. I do not think it is directly related
to running without seek-depth limit. How long does it take from mount to process death?
Hours? Days?
One thing you could try is to only run the instance of rar2fs that is using no seek-depth
limit. Does it still happen?
If you have top or htop, can you please let both instances run and 

"All modern Linux kernels have a built-in mechanism called “Out Of Memory killer” which
can annihilate your processes under extremely low memory conditions. When such a condition
is detected, the killer is activated and picks a process to kill. The target is picked
using a set of heuristics scoring all processes and selecting the one with the worst
score to kill."

That said, it might still not be rar2fs that is the root cause, even though it seems
likely. But we need make sure before starting a ghost chase.

Original issue reported on code.google.com by hasse69 on 2014-09-06 06:55:38

[hasse69]

Something made the post to break up.

If you have top or htop, can you please let both instances run and make a couple of
samples of VIRT, RES and SHR. 

Original issue reported on code.google.com by hasse69 on 2014-09-06 06:58:55

[hasse69]

It happens periodically, but it feels like it happens a lot more when running more traffic
through it.  For example when Plex server was running its initial  sync/refresh it
happened several times during a couple of hours. When the server wasn't in use for
2-3 days nothing happened.

Some more background info about how its setup:

Windows 2008r2 server running Virtualbox with:
Ubuntu 14.04 server (fully updated)  
1 CPU, 2Gb memory allocated.
rar2fs v1.20.0
unrarsrc 4.0.7
Windows folders shared to ubuntu machine through virtualbox shared folder
Unpacked folder shared back to Windows by samba

Running 2 folders with rar2fs. 
1 with Movies(with --seek-length=1) 
1 with Tv_series (with no seek-length)
Both folders are quite big (around 9TB for tv_series) but i thought 2Gb should suffice?


When it crashes it only takes down tv_series, Movies never get affected. 

Tried today to run it with --seek-length=7 (no idea if this will help) 7 was the minimum
to get most of the tv_series to unpack. 

Just ran a plex scan with htop running. Rar2fs (with TV_series) eats up more and more
memory, when cancelling the scan - it never backs down.  Can send over some screen
grabs by e-mail if interested 

Original issue reported on code.google.com by emil@vimmelman.se on 2014-09-06 08:09:19

[hasse69]

Since this is not going to be an easy task to trouble-shoot I will contact you by private
e-mail. This will most likely require some iterations :(

Original issue reported on code.google.com by hasse69 on 2014-09-06 08:23:31

[hasse69]

Seems related to libunrar 4.0.7. Awaiting feedback from submitter how the system behaves
after upgrade.

Original issue reported on code.google.com by hasse69 on 2014-09-08 07:08:13

[hasse69]

Can not reproduce the problem locally with the combination libunrar 4.0.7 and rar2fs
1.20.0. Upgrade to latest version of libunrar seems to resolve the reported issue.
I will close this case until further information is provided or I receive any additional
reports about libunrar 4.0.7 is causing problems

Original issue reported on code.google.com by hasse69 on 2014-09-15 06:14:31

[hasse69]

I am opening this issue again. A weakness in libunrar 4.0.7 has been identified. Since
some of the library extension are based on the same code also rar2fs may suffer from
the same potential memory leaks. This should not be a serious issue since it only affects
some extreme error cases, but severe enough to investigate it a bit further.

Original issue reported on code.google.com by hasse69 on 2014-09-18 21:41:55

[hasse69]

Just for the record, the potential memory leak was fixed in libunrar 4.2.0 and later.

Original issue reported on code.google.com by hasse69 on 2014-09-18 21:46:35

[hasse69]

A potential fix for this issue has been committed to trunk.

Original issue reported on code.google.com by hasse69 on 2014-10-24 20:03:03

[hasse69]

Missing feedback from submitter about the latest patch.
Will close this issue since a weakness in libunrar 4.0.7 was confirmed and a workaround
in rar2fs is committed to trunk.

Original issue reported on code.google.com by hasse69 on 2014-11-11 20:56:55