Skip to content

Container initialization hangs at "Setting permissions..." on large scale deployments #368

@DevlTz

Description

@DevlTz

Hi @matteius! Upgraded to v0.32.4 today and hit a very interesting edge case regarding large-scale deployments and container initialization.

The Issue

After updating, the main lightnvr container hangs during startup. It stops right after printing [INFO] Setting permissions... and the web server never boots. Because the internal server doesn't start, my reverse proxy (Nginx) throws a 502 Bad Gateway.

Analysis

I am running 44 cameras continuously, meaning my /var/lib/lightnvr/data/recordings directory contains hundreds of thousands of small video chunks.

Looking at the processes, the docker-entrypoint.sh script is running a recursive permission change (chown -R or chmod -R) across the entire data directory. Because of the sheer volume of files and the IO bottleneck of standard mechanical HDDs, this permission sweep can take hours to complete.

Logs

1. Docker Compose Logs:
The initialization stops completely here:

lightnvr  | [INFO] Removing old go2rtc configuration to regenerate fresh...
lightnvr  | [INFO] Creating default go2rtc configuration...
lightnvr  | [INFO] Default go2rtc configuration created at /etc/lightnvr/go2rtc/go2rtc.yaml
lightnvr  | [INFO] No models to copy, skipping model initialization
lightnvr  | [INFO] Setting permissions...
# -> Hangs here indefinitely.

2. Host Process (htop):
Checking the host, the CPU usage isn't high, but the entrypoint process is stuck sleeping (S state), waiting for disk I/O to sweep the massive folder:

PID       USER   PRI  NI  VIRT   RES   SHR S  CPU% MEM%   TIME+  Command
1602911   root    20   0  2680  1892  1748 S   0.0  0.0  0:00.05 /bin/sh /usr/local/bin/docker-entrypoint.sh /bin/start.sh

Proposed Solutions

To fix this for large-scale users, I have a couple of architectural ideas:

Targeted Permissions: Modify the entrypoint script so it only recursively applies permissions to config/database folders, but specifically skips the recordings/ and mp4/ subdirectories.

Environment Variable: Add a toggle like SKIP_PERMISSIONS_CHECK=true in the docker-compose.yml so advanced users can manage host directory permissions manually and bypass the script.

  • Just wanted to flag this as the project scales to handle heavier loads! Let me know if you implement a fix, and I'll gladly test the PR on my 44-camera setup.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions