Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Runs out of memory while doing local scan of big library on Raspberry PI Zero W #1648

Open
Caenhorn opened this issue Nov 30, 2017 · 9 comments

Comments

@Caenhorn
Copy link

@Caenhorn Caenhorn commented Nov 30, 2017

Last output:

INFO     Scanned 18700 of 26522 files in 12201s, ~5103s left.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/DIRECTSO.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/FILEIO.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/FLASHASS.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/FONTASSE.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/FONTXTRA.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/INETURL.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/MACROMIX.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/MIXSERVI.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/MPEG3IMP.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/MUIDIALO.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/NETFILE.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/NETLINGO.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/QT3ASSET.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SOUNDCON.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SOUNDIMP.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SWACMPR.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SWACNVRT.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SWADCMPR.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SWAIMPOR.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SWAOPT.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SWASTRM.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/TEXTASSE.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/TEXTXTRA.X32: No audio found in file.
INFO     Scanned 18800 of 26522 files in 12234s, ~5025s left.

(python:2337): GLib-ERROR **: /build/glib2.0-F5w919/glib2.0-2.50.3/./glib/gmem.c:100: failed to allocate 11017050 bytes
Trace/breakpoint trap

By the way, log file didn't catch that for some reason. Last line in log file is last progress report on scanning.

I'm running:

pi@mopidy-zero:/var/log/mopidy $ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 9.1 (stretch)
Release:        9.1
Codename:       stretch
pi@mopidy-zero:/var/log/mopidy $ uname -a
Linux mopidy-zero 4.9.59+ #1047 Sun Oct 29 11:47:10 GMT 2017 armv6l GNU/Linux

pi@mopidy-zero:/var/log/mopidy $ dpkg -s mopidy
Package: mopidy
Status: install ok installed
Priority: optional
Section: sound
Installed-Size: 853
Maintainer: Stein Magnus Jodal <jodal@debian.org>
Architecture: all
Version: 2.1.0-1
@Caenhorn Caenhorn changed the title Runs out of memory while doing local scan of big library on Raspberry Zero W Runs out of memory while doing local scan of big library on Raspberry PI Zero W Nov 30, 2017
@Caenhorn

This comment has been minimized.

Copy link
Author

@Caenhorn Caenhorn commented Nov 30, 2017

Effective configuration:

pi@mopidy-zero:/var/log/mopidy $ sudo mopidyctl config
Running "/usr/bin/mopidy --config /usr/share/mopidy/conf.d:/etc/mopidy/mopidy.conf config" as user mopidy
[core]
cache_dir = /var/cache/mopidy
config_dir = /etc/mopidy
data_dir = /var/lib/mopidy
max_tracklist_length = 10000
restore_state = false

[logging]
color = true
console_format = %(levelname)-8s %(message)s
debug_format = %(levelname)-8s %(asctime)s [%(process)d:%(threadName)s] %(name)s\n  %(message)s
debug_file = /var/log/mopidy/mopidy-debug.log
config_file = /etc/mopidy/logging.conf

[audio]
mixer = software
mixer_volume =
output = autoaudiosink
buffer_time =

[proxy]
scheme =
hostname =
port =
username =
password =

[moped]
enabled = true

[local-images]
enabled = true
library = json
base_uri = /images/
image_dir =
album_art_files =
  *.jpg
  *.jpeg
  *.png

[iris]
enabled = true
country = NZ
locale = en_NZ
spotify_authorization_url = https://jamesbarnsley.co.nz/auth_spotify.php
lastfm_authorization_url = https://jamesbarnsley.co.nz/auth_lastfm.php

[mpd]
enabled = true
hostname = ::
port = 6600
password =
max_connections = 20
connection_timeout = 60
zeroconf = Mopidy MPD server on $hostname
command_blacklist =
  listall
  listallinfo
default_playlist_scheme = m3u

[http]
enabled = true
hostname = ::
port = 6680
static_dir =
zeroconf = Mopidy HTTP server on $hostname

[stream]
enabled = true
protocols =
  http
  https
  mms
  rtmp
  rtmps
  rtsp
metadata_blacklist =
timeout = 5000

[m3u]
enabled = true
base_dir =
default_encoding = latin-1
default_extension = .m3u8
playlists_dir = /var/lib/mopidy/playlists

[softwaremixer]
enabled = true

[file]
enabled = true
media_dirs =
  $XDG_MUSIC_DIR|Music
  ~/|Home
excluded_file_extensions =
  .jpg
  .jpeg
show_dotfiles = false
follow_symlinks = false
metadata_timeout = 1000

[local]
enabled = true
library = json
media_dir = /mnt/music
scan_timeout = 15000
scan_flush_threshold = 100
scan_follow_symlinks = false
excluded_file_extensions =
  .directory
  .html
  .jpeg
  .jpg
  .log
  .nfo
  .png
  .txt
  .cue
@kingosticks

This comment has been minimized.

Copy link
Member

@kingosticks kingosticks commented Nov 30, 2017

Have you tried with mopidy-local-sqlite? That is the preferred local library to use.

@Caenhorn

This comment has been minimized.

Copy link
Author

@Caenhorn Caenhorn commented Nov 30, 2017

Thanks for the tip! I installed mopidy-local-sqlite, added it as a library of choice in config.
Alas, it crashes with sqlite too.

INFO     Scanned 18500 of 26212 files in 11936s, ~4976s left.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/CDPLUS/Sub_Info.ja: gst-stream-error-quark: Could not determine type of stream. (4)
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/faceplate.dxr: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/MEDIA/media.cst: gst-stream-error-quark: Could not determine type of stream. (4)
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/MEDIA/MOVIES/BLANK.MOV: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/MEDIA/MOVIES/ozzy.mov: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/MEDIA/readme.htm: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/PICTURES/JACKET01.00J: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/PICTURES/JACKET01.00N: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/PICTURES/JACKET01.00S: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/PICTURES/JACKET01.00T: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/WIN_QT4/QuickTimeInstallCache.qdat: gst-stream-error-quark: Could not determine type of stream. (4)
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/BUDAPI.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/DIRECTSO.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/FILEIO.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/FLASHASS.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/FONTASSE.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/FONTXTRA.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/INETURL.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/MACROMIX.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/MIXSERVI.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/MPEG3IMP.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/MUIDIALO.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/NETFILE.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/NETLINGO.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/QT3ASSET.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SOUNDCON.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SOUNDIMP.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SWACMPR.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SWACNVRT.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SWADCMPR.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SWAIMPOR.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SWAOPT.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/SWASTRM.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/TEXTASSE.X32: No audio found in file.
WARNING  Failed local:track:Ozzy%20Osbourne/2001%20-%20Down%20To%20Earth/DATA/XTRAS/TEXTXTRA.X32: No audio found in file.
INFO     Scanned 18600 of 26212 files in 11967s, ~4897s left.
WARNING  Failed local:track:Pain%20of%20Salvation/2004%20-%20Be/01%20-%20Prologue-Animae%20Partus%20%28%27I%20Am%27%29.flac: Timeout after 15000ms

(python:1142): GLib-ERROR **: /build/glib2.0-F5w919/glib2.0-2.50.3/./glib/gmem.c:100: failed to allocate 11017050 bytes
Trace/breakpoint trap
@kingosticks

This comment has been minimized.

Copy link
Member

@kingosticks kingosticks commented Nov 30, 2017

26k files should be fine. This has been used with larger collections. Can you provide some data on the process size? How much ram have you allocated to system memory? Is it crashing when it hits a limit?

@Caenhorn

This comment has been minimized.

Copy link
Author

@Caenhorn Caenhorn commented Dec 1, 2017

I've collected some data on process size and memory dynamics. It looks like it does run out of free+cached+swap memory. Judging by log-file, process itself does not crash.
process memory
memory dynamics

Log-file:
mopidy.log

Here is netdata snapshot. You can use any netdata demo site to import it and check metrics on your own.
netdata-mopidy-zero-20171201-221224-9204.snapshot.gz

Most of the files in my library are lossless, if it matters. I used stretch lite image for mopidy installation. It is a dedicated system for mopidy, so only services coming by default in stretch lite are running. My config.txt is almost vanilla, except for disabled onboard audio and enabled Allo Mini Boss gpio extension board. I could free up some memory by allocating less GPU mem next time.

@mabeltron

This comment has been minimized.

Copy link

@mabeltron mabeltron commented May 9, 2018

I can confirm I have seen this problem with mopidy and with other MPD servers. The scan function appears to have an in-memory cache and will fill memory to the point where the hardware will become unresponsive - I had this on a server with 8GB of RAM as well as RPis. I suspect there needs to be a method of flushing to disk more frequently.

@monte-monte

This comment has been minimized.

Copy link

@monte-monte monte-monte commented Jun 20, 2019

I have the same behavior on a server using mopidy-local-sqlite with option "scan_flush_threshold = 50". It doesn't seem to free memory until it finishes scan. Had to hard reset machine several times before I finished initial scan. This for sure must be fixed.
mopidy version 2.2.2

@kingosticks

This comment has been minimized.

Copy link
Member

@kingosticks kingosticks commented Oct 23, 2019

@jrowberg Can you confirm that your problem is not related to scanning like the others here? Yours is a memory issue when simply playing music, right? And the only difference between this Pi and the other 3 Pis you have is the exact local files being played? What model Rapsberry Pis are you using?

@kingosticks

This comment has been minimized.

Copy link
Member

@kingosticks kingosticks commented Oct 23, 2019

Yes please, a separate issue would be helpful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.