Unable to start container on rasbian buster 32bits

[jrhbcn]

Since yesterday I am unable to start my unifi container on a raspberry pi 4 running buster 10 - 32 bits. It was working fine before a docker-compose pull.

I am using docker-compose with the following config:

  unifi:   
    image: ghcr.io/linuxserver/unifi-controller:arm32v7-latest
    container_name: unifi  
    restart: unless-stopped
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/Madrid
      - MEM_LIMIT=1024M #optional
      - MEM_STARTUP=1024M #optional
    ports:
      - 3478:3478/udp
      - 10001:10001/udp
      - 8080:8080
      - 8081:8081
      - 8443:8443
      - 8843:8843
      - 8880:8880
      - 6789:6789
    volumes:
      - unifi_data:/config  

Container seems to start fine as "docker logs"

[cont-init.d] executing container initialization scripts...
[cont-init.d] 10-adduser: executing... 
usermod: no changes

-------------------------------------
          _         ()
         | |  ___   _    __
         | | / __| | |  /  \ 
         | | \__ \ | | | () |
         |_| |___/ |_|  \__/


Brought to you by linuxserver.io
We gratefully accept donations at:
https://www.linuxserver.io/donate/
-------------------------------------
GID/UID
-------------------------------------

User uid:    1000
User gid:    1000
-------------------------------------

[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 20-config: executing... 
[cont-init.d] 20-config: exited 0.
[cont-init.d] 30-keygen: executing... 
[cont-init.d] 30-keygen: exited 0.
[cont-init.d] 90-custom-folders: executing... 
[cont-init.d] 90-custom-folders: exited 0.
[cont-init.d] 99-custom-scripts: executing... 
[custom-init] no custom files found exiting...
[cont-init.d] 99-custom-scripts: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.

However, mondb logs shows errors:

2021-09-03T09:52:25.090+0200 [initandlisten] MongoDB starting : pid=4458 port=27117 dbpath=/usr/lib/unifi/data/db 32-bit host=5a62f127ae8f
2021-09-03T09:52:25.090+0200 [initandlisten] 
2021-09-03T09:52:25.090+0200 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
2021-09-03T09:52:25.091+0200 [initandlisten] **       32 bit builds are limited to less than 2GB of data (or less with --journal).
2021-09-03T09:52:25.091+0200 [initandlisten] **       Note that journaling defaults to off for 32 bit and is currently off.
2021-09-03T09:52:25.091+0200 [initandlisten] **       See http://dochub.mongodb.org/core/32bit
2021-09-03T09:52:25.091+0200 [initandlisten] 
2021-09-03T09:52:25.091+0200 [initandlisten] db version v2.6.10
2021-09-03T09:52:25.091+0200 [initandlisten] git version: nogitversion
2021-09-03T09:52:25.091+0200 [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
2021-09-03T09:52:25.091+0200 [initandlisten] build info: Linux kishi10 3.2.0-84-highbank #121-Ubuntu SMP PREEMPT Tue May 5 19:44:15 UTC 2015 armv7l BOOST_LIB_VERSION=1_58
2021-09-03T09:52:25.091+0200 [initandlisten] allocator: tcmalloc
2021-09-03T09:52:25.092+0200 [initandlisten] options: { net: { bindIp: "127.0.0.1", port: 27117, unixDomainSocket: { pathPrefix: "/usr/lib/unifi/run" } }, processManagement: { pidFilePath: "/usr/lib/unifi/run/mongod.pid" }, storage: { dbPath: "/usr/lib/unifi/data/db" }, systemLog: { destination: "file", logAppend: true, path: "/usr/lib/unifi/logs/mongod.log" } }
2021-09-03T09:52:25.092+0200 [initandlisten] 
2021-09-03T09:52:25.092+0200 [initandlisten] ** WARNING: Readahead for /usr/lib/unifi/data/db is set to 65532KB
2021-09-03T09:52:25.092+0200 [initandlisten] **          We suggest setting it to 256KB (512 sectors) or less
2021-09-03T09:52:25.092+0200 [initandlisten] **          http://dochub.mongodb.org/core/readahead
2021-09-03T09:52:25.105+0200 [initandlisten] ERROR:   mmap() failed for /usr/lib/unifi/data/db/ace_stat.7 len:536870912 errno:12 Cannot allocate memory
2021-09-03T09:52:25.105+0200 [initandlisten] ERROR: mmap failed with out of memory. You are using a 32-bit build and probably need to upgrade to 64
2021-09-03T09:52:25.105+0200 [initandlisten] Assertion: 16966:_extentManager.init failed: InternalError DataFile::openExisting - mmf.open failed
2021-09-03T09:52:25.111+0200 [initandlisten] 0x7639be 0x72180a 0x70a5f0 0x70a68c 0x297df2 0x29bf7e 0x29f6fa 0x2afeb4 0x2b002c 0x1ee366 0x1ef78a 0x1f009c 0x1c4f8c 0xb679e8aa 
 bin/mongod(_ZN5mongo15printStackTraceERSo+0x25) [0x7639be]
 bin/mongod(_ZN5mongo10logContextEPKc+0x15d) [0x72180a]
 bin/mongod(_ZN5mongo11msgassertedEiPKc+0x8f) [0x70a5f0]
 bin/mongod(_ZN5mongo11msgassertedEiRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x7) [0x70a68c]
 bin/mongod(_ZN5mongo8Database12openAllFilesEv+0xa1) [0x297df2]
 bin/mongod(_ZN5mongo8DatabaseC1EPKcRbRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x1ad) [0x29bf7e]
 bin/mongod(_ZN5mongo14DatabaseHolder11getOrCreateERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_Rb+0x649) [0x29f6fa]
 bin/mongod(_ZN5mongo6Client7Context11_finishInitEv+0x2b) [0x2afeb4]
 bin/mongod(_ZN5mongo6Client7ContextC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES9_b+0x67) [0x2b002c]
 bin/mongod() [0x1ee366]
 bin/mongod(_ZN5mongo14_initAndListenEi+0x5ed) [0x1ef78a]
 bin/mongod(_ZN5mongo13initAndListenEi+0x13) [0x1f009c]
 bin/mongod(main+0x197) [0x1c4f8c]
 /lib/arm-linux-gnueabihf/libc.so.6(__libc_start_main+0x9d) [0xb679e8aa]
2021-09-03T09:52:25.112+0200 [initandlisten] warning database /usr/lib/unifi/data/db ace_stat could not be opened
2021-09-03T09:52:25.112+0200 [initandlisten] DBException 16966: _extentManager.init failed: InternalError DataFile::openExisting - mmf.open failed
2021-09-03T09:52:25.115+0200 [initandlisten] exception in initAndListen: 16966 _extentManager.init failed: InternalError DataFile::openExisting - mmf.open failed, terminating
2021-09-03T09:52:25.115+0200 [initandlisten] dbexit: 
2021-09-03T09:52:25.115+0200 [initandlisten] shutdown: going to close listening sockets...
2021-09-03T09:52:25.115+0200 [initandlisten] shutdown: going to flush diaglog...
2021-09-03T09:52:25.116+0200 [initandlisten] shutdown: going to close sockets...
2021-09-03T09:52:25.116+0200 [initandlisten] shutdown: waiting for fs preallocator...
2021-09-03T09:52:25.116+0200 [initandlisten] shutdown: closing all files...
2021-09-03T09:52:25.116+0200 [initandlisten] closeAllFiles() finished
2021-09-03T09:52:25.116+0200 [initandlisten] shutdown: removing fs lock...
2021-09-03T09:52:25.116+0200 [initandlisten] dbexit: really exiting now

I had already fixed previously buster 32 bits issues as per here https://docs.linuxserver.io/faq installing libseccomp2.4.4 (another docker containers such as radarr run fine).

Any ideas how to fix it (at least to get a proper backup and then updating my raspberry to 64).

[github-actions]

Thanks for opening your first issue here! Be sure to follow the bug or feature issue templates!

[thespad]

At a guess your DB has grown too large and it now can't allocate enough virtual address space to mmap it all.

I'm afraid I don't know of a solution other than upgrading the kernel or the full OS to 64-bit, but you should just be able to shut the container down and do a flat file backup of the /config mount to port it to a new install.

The only alternative is to nuke the DB and create a new install importing an existing config backup, but obviously you'll lose all your stats and historical data.

[jrhbcn]

Thanks @thespad, I am going to try just nuking the DB loosing the stats to see if it boots.

[jrhbcn]

I have managed to recover my installation thanks to @thespad suggestions. Unfortunately, the latest automatic backup from the controller does not work as it already triggers the out of memory problem in mongodb. Fortunately a previous version backup (with only config not historical data) worked fine. Closing the bug now.