Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce memory usage when scanning large folders #1484

Merged
merged 3 commits into from
Feb 7, 2023

Conversation

magnww
Copy link
Contributor

@magnww magnww commented Feb 6, 2023

When audiobookshelf scans a folder with a lot of files, it will take up a lot of memory, because audiobookshelf will start a ffprobe process for each media file at the same time. In order to reduce memory usage, it is necessary to limit the number of ffprobe processes started at the same time.
In addition, after the ffprobe process is executed, it does not exit immediately and becomes a zombie process. It is necessary to call kill() in the exit event to end the process.

Before:
图片

After:
图片

@advplyr
Copy link
Owner

advplyr commented Feb 6, 2023

Thanks! I haven't done any testing on the zombie processes to confirm but I'm wondering if calling kill() in the exit event could cause some issues if the process exited.

In node docs https://nodejs.org/api/child_process.html#subprocesskillsignal

The ChildProcess object may emit an 'error' event if the signal cannot be delivered. Sending a signal to a child process that has already exited is not an error but may have unforeseen consequences.

@magnww
Copy link
Contributor Author

magnww commented Feb 7, 2023

After some investigation I found a docker configuration issue. The zombie process problem can be solved by adding --init parameter to docker run command. So I will revert the change of ffprobe.
https://docs.docker.com/engine/reference/run/#specify-an-init-process

@advplyr
Copy link
Owner

advplyr commented Feb 7, 2023

Thanks!

@advplyr advplyr merged commit 4529fc0 into advplyr:master Feb 7, 2023
@whmountains
Copy link

Hmmm, is it really a good idea to create zombie processes? Having the init system reap them seems like it should be a last resort. I am concerned that few will realize they should be starting their containers with --init. I vaguely knew that was an option, but I had not seen any examples use it and so was not in the habit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants