netbox user uid 101 Permission denied: '/etc/netbox/config/configuration.py'

[jkldgoefgkljefogeg]

7942e9e broke volume permission. I had to drop user: '101' to work around this.

Why use an arbitrary UID instead of a pre-created user?

-->

Current Behavior

PermissionError: [Errno 13] Permission denied: '/etc/netbox/config/configuration.py'

Expected Behavior

no filesystem permission error

Debug Information

~/git/netbox-docker$ docker version
Client: Docker Engine - Community
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        afacb8b7f0
 Built:             Wed Mar 11 01:25:46 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       afacb8b7f0
  Built:            Wed Mar 11 01:24:19 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
user@fl:~/git/netbox-docker$ git rev-parse HEAD
93a3784295fc3e070c76f699387eccd5bb820f4a

user@fl:~/git/netbox-docker$ docker inspect netboxcommunity/netbox:latest --format "{{json .Config.Labels}}"
{"BUILD_REASON":"netbox-docker","NETBOX_GIT_BRANCH":"HEAD","NETBOX_GIT_REF":"86755029ef07ba37ea5ee691934cc7b92fd226e9","NETBOX_GIT_URL":"https://github.com/netbox-community/netbox.git","ORIGINAL_TAG":"docker.io/netboxcommunity/netbox:v2.8.4","org.label-schema.build-date":"2020-05-16T16:56+00:00","org.label-schema.description":"A container based distribution of Netbox, the free and open IPAM and DCIM solution.","org.label-schema.name":"Netbox Docker","org.label-schema.schema-version":"1.0","org.label-schema.url":"https://github.com/netbox-community/netbox-docker","org.label-schema.usage":"https://github.com/netbox-community/netbox-docker/wiki","org.label-schema.vcs-ref":"93a3784295fc3e070c76f699387eccd5bb820f4a","org.label-schema.vcs-url":"https://github.com/netbox-community/netbox-docker.git","org.label-schema.vendor":"The netbox-docker contributors.","org.label-schema.version":"0.24.0","org.opencontainers.image.authors":"The netbox-docker contributors.","org.opencontainers.image.created":"2020-05-16T16:56+00:00","org.opencontainers.image.description":"A container based distribution of Netbox, the free and open IPAM and DCIM solution.","org.opencontainers.image.documentation":"https://github.com/netbox-community/netbox-docker/wiki","org.opencontainers.image.licenses":"Apache-2.0","org.opencontainers.image.revision":"93a3784295fc3e070c76f699387eccd5bb820f4a","org.opencontainers.image.source":"https://github.com/netbox-community/netbox-docker.git","org.opencontainers.image.title":"Netbox Docker","org.opencontainers.image.url":"https://github.com/netbox-community/netbox-docker","org.opencontainers.image.vendor":"The netbox-docker contributors.","org.opencontainers.image.version":"0.24.0"}

The output of docker-compose logs netbox:

$ docker-compose logs
Attaching to netbox-docker_nginx_1, netbox-docker_netbox_1, netbox-docker_netbox-worker_1, netbox-docker_redis-cache_1, netbox-docker_postgres_1, netbox-docker_redis_1
netbox-worker_1  | Traceback (most recent call last):
netbox-worker_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 224, in fetch_command
netbox-worker_1  |     app_name = commands[subcommand]
netbox-worker_1  | KeyError: 'rqworker'
netbox-worker_1  | 
netbox-worker_1  | During handling of the above exception, another exception occurred:
netbox-worker_1  | 
netbox-worker_1  | Traceback (most recent call last):
netbox-worker_1  |   File "/opt/netbox/netbox/netbox/configuration.py", line 7, in <module>
netbox-worker_1  |     spec.loader.exec_module(module)
netbox-worker_1  |   File "<frozen importlib._bootstrap_external>", line 724, in exec_module
netbox-worker_1  |   File "<frozen importlib._bootstrap_external>", line 859, in get_code
netbox-worker_1  |   File "<frozen importlib._bootstrap_external>", line 916, in get_data
netbox-worker_1  | PermissionError: [Errno 13] Permission denied: '/etc/netbox/config/configuration.py'
netbox-worker_1  | 
netbox-worker_1  | During handling of the above exception, another exception occurred:
netbox-worker_1  | 
netbox-worker_1  | Traceback (most recent call last):
netbox-worker_1  |   File "/opt/netbox/netbox/netbox/settings.py", line 40, in <module>
netbox-worker_1  |     from netbox import configuration
netbox-worker_1  |   File "/opt/netbox/netbox/netbox/configuration.py", line 10, in <module>
netbox-worker_1  |     raise ImportError('')
netbox-worker_1  | ImportError
netbox-worker_1  | 
netbox-worker_1  | During handling of the above exception, another exception occurred:
netbox-worker_1  | 
netbox-worker_1  | Traceback (most recent call last):
netbox-worker_1  |   File "/opt/netbox/netbox/manage.py", line 10, in <module>
netbox-worker_1  |     execute_from_command_line(sys.argv)
netbox-worker_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
netbox-worker_1  |     utility.execute()
netbox-worker_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 395, in execute
netbox-worker_1  |     self.fetch_command(subcommand).run_from_argv(self.argv)
netbox-worker_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 231, in fetch_command
netbox-worker_1  |     settings.INSTALLED_APPS
netbox-worker_1  |   File "/usr/local/lib/python3.7/site-packages/django/conf/__init__.py", line 76, in __getattr__
netbox-worker_1  |     self._setup(name)
netbox-worker_1  |   File "/usr/local/lib/python3.7/site-packages/django/conf/__init__.py", line 63, in _setup
netbox-worker_1  |     self._wrapped = Settings(settings_module)
netbox-worker_1  |   File "/usr/local/lib/python3.7/site-packages/django/conf/__init__.py", line 142, in __init__
netbox-worker_1  |     mod = importlib.import_module(self.SETTINGS_MODULE)
netbox-worker_1  |   File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
netbox-worker_1  |     return _bootstrap._gcd_import(name[level:], package, level)
netbox-worker_1  |   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
netbox-worker_1  |   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
netbox-worker_1  |   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
netbox-worker_1  |   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
netbox-worker_1  |   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
netbox-worker_1  |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
netbox-worker_1  |   File "/opt/netbox/netbox/netbox/settings.py", line 43, in <module>
netbox-worker_1  |     "Configuration file is not present. Please define netbox/netbox/configuration.py per the documentation."
netbox-worker_1  | django.core.exceptions.ImproperlyConfigured: Configuration file is not present. Please define netbox/netbox/configuration.py per the documentation.
netbox-worker_1  | Traceback (most recent call last):
netbox-worker_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 224, in fetch_command
netbox-worker_1  |     app_name = commands[subcommand]
netbox-worker_1  | KeyError: 'rqworker'
netbox-worker_1  | 
netbox-worker_1  | During handling of the above exception, another exception occurred:
netbox-worker_1  | 
netbox-worker_1  | Traceback (most recent call last):
netbox-worker_1  |   File "/opt/netbox/netbox/netbox/configuration.py", line 7, in <module>
netbox-worker_1  |     spec.loader.exec_module(module)
netbox-worker_1  |   File "<frozen importlib._bootstrap_external>", line 724, in exec_module
netbox-worker_1  |   File "<frozen importlib._bootstrap_external>", line 859, in get_code
netbox-worker_1  |   File "<frozen importlib._bootstrap_external>", line 916, in get_data
netbox-worker_1  | PermissionError: [Errno 13] Permission denied: '/etc/netbox/config/configuration.py'
netbox-worker_1  | 
netbox-worker_1  | During handling of the above exception, another exception occurred:
netbox-worker_1  | 
netbox-worker_1  | Traceback (most recent call last):
netbox-worker_1  |   File "/opt/netbox/netbox/netbox/settings.py", line 40, in <module>
netbox-worker_1  |     from netbox import configuration
netbox-worker_1  |   File "/opt/netbox/netbox/netbox/configuration.py", line 10, in <module>
netbox-worker_1  |     raise ImportError('')
netbox-worker_1  | ImportError
netbox-worker_1  | 
netbox-worker_1  | During handling of the above exception, another exception occurred:
netbox-worker_1  | 
netbox-worker_1  | Traceback (most recent call last):
netbox-worker_1  |   File "/opt/netbox/netbox/manage.py", line 10, in <module>
netbox-worker_1  |     execute_from_command_line(sys.argv)
netbox-worker_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
netbox-worker_1  |     utility.execute()
netbox-worker_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 395, in execute
netbox-worker_1  |     self.fetch_command(subcommand).run_from_argv(self.argv)
netbox-worker_1  |   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 231, in fetch_command
netbox-worker_1  |     settings.INSTALLED_APPS
netbox-worker_1  |   File "/usr/local/lib/python3.7/site-packages/django/conf/__init__.py", line 76, in __getattr__
netbox-worker_1  |     self._setup(name)
netbox-worker_1  |   File "/usr/local/lib/python3.7/site-packages/django/conf/__init__.py", line 63, in _setup
netbox-worker_1  |     self._wrapped = Settings(settings_module)
netbox-worker_1  |   File "/usr/local/lib/python3.7/site-packages/django/conf/__init__.py", line 142, in __init__
netbox-worker_1  |     mod = importlib.import_module(self.SETTINGS_MODULE)
netbox-worker_1  |   File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
netbox-worker_1  |     return _bootstrap._gcd_import(name[level:], package, level)
netbox-worker_1  |   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
netbox-worker_1  |   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
netbox-worker_1  |   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
netbox-worker_1  |   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
netbox-worker_1  |   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
netbox-worker_1  |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
netbox-worker_1  |   File "/opt/netbox/netbox/netbox/settings.py", line 43, in <module>
netbox-worker_1  |     "Configuration file is not present. Please define netbox/netbox/configuration.py per the documentation."
netbox-worker_1  | django.core.exceptions.ImproperlyConfigured: Configuration file is not present. Please define netbox/netbox/configuration.py per the documentation.
netbox_1         | Traceback (most recent call last):
netbox_1         |   File "/opt/netbox/netbox/netbox/configuration.py", line 7, in <module>
netbox_1         |     spec.loader.exec_module(module)
netbox_1         |   File "<frozen importlib._bootstrap_external>", line 724, in exec_module
netbox_1         |   File "<frozen importlib._bootstrap_external>", line 859, in get_code
netbox_1         |   File "<frozen importlib._bootstrap_external>", line 916, in get_data
netbox_1         | PermissionError: [Errno 13] Permission denied: '/etc/netbox/config/configuration.py'
netbox_1         | 
netbox_1         | During handling of the above exception, another exception occurred:
netbox_1         | 
netbox_1         | Traceback (most recent call last):
netbox_1         |   File "/opt/netbox/netbox/netbox/settings.py", line 40, in <module>
netbox_1         |     from netbox import configuration
netbox_1         |   File "/opt/netbox/netbox/netbox/configuration.py", line 10, in <module>
netbox_1         |     raise ImportError('')
netbox_1         | ImportError
netbox_1         | 
netbox_1         | During handling of the above exception, another exception occurred:
netbox_1         | 
netbox_1         | Traceback (most recent call last):
netbox_1         |   File "./manage.py", line 10, in <module>
netbox_1         |     execute_from_command_line(sys.argv)
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
netbox_1         |     utility.execute()
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 395, in execute
netbox_1         |     self.fetch_command(subcommand).run_from_argv(self.argv)
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 328, in run_from_argv
netbox_1         |     self.execute(*args, **cmd_options)
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 366, in execute
netbox_1         |     self.check()
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 395, in check
netbox_1         |     include_deployment_checks=include_deployment_checks,
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 63, in _run_checks
netbox_1         |     issues = run_checks(tags=[Tags.database])
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/core/checks/registry.py", line 72, in run_checks
netbox_1         |     new_errors = check(app_configs=app_configs)
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/core/checks/database.py", line 9, in check_database_backends
netbox_1         |     for conn in connections.all():
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 222, in all
netbox_1         |     return [self[alias] for alias in self]
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 219, in __iter__
netbox_1         |     return iter(self.databases)
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/utils/functional.py", line 48, in __get__
netbox_1         |     res = instance.__dict__[self.name] = self.func(instance)
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 153, in databases
netbox_1         |     self._databases = settings.DATABASES
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/conf/__init__.py", line 76, in __getattr__
netbox_1         |     self._setup(name)
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/conf/__init__.py", line 63, in _setup
netbox_1         |     self._wrapped = Settings(settings_module)
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/conf/__init__.py", line 142, in __init__
netbox_1         |     mod = importlib.import_module(self.SETTINGS_MODULE)
netbox_1         |   File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
netbox_1         |     return _bootstrap._gcd_import(name[level:], package, level)
netbox_1         |   File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
netbox_1         |   File "<frozen importlib._bootstrap>", line 983, in _find_and_load
netbox_1         |   File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
netbox_1         |   File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
netbox_1         |   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
netbox_1         |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
netbox_1         |   File "/opt/netbox/netbox/netbox/settings.py", line 43, in <module>
netbox_1         |     "Configuration file is not present. Please define netbox/netbox/configuration.py per the documentation."
netbox_1         | django.core.exceptions.ImproperlyConfigured: Configuration file is not present. Please define netbox/netbox/configuration.py per the documentation.
netbox_1         | ⏳ Waiting on DB... (0s / 30s)
netbox_1         | Traceback (most recent call last):
netbox_1         |   File "/opt/netbox/netbox/netbox/configuration.py", line 7, in <module>
netbox_1         |     spec.loader.exec_module(module)
netbox_1         |   File "<frozen importlib._bootstrap_external>", line 724, in exec_module
netbox_1         |   File "<frozen importlib._bootstrap_external>", line 859, in get_code
netbox_1         |   File "<frozen importlib._bootstrap_external>", line 916, in get_data
netbox_1         | PermissionError: [Errno 13] Permission denied: '/etc/netbox/config/configuration.py'
netbox_1         | 
netbox_1         | During handling of the above exception, another exception occurred:
netbox_1         | 
netbox_1         | Traceback (most recent call last):
netbox_1         |   File "/opt/netbox/netbox/netbox/settings.py", line 40, in <module>
netbox_1         |     from netbox import configuration
netbox_1         |   File "/opt/netbox/netbox/netbox/configuration.py", line 10, in <module>
netbox_1         |     raise ImportError('')
netbox_1         | ImportError
netbox_1         | 
netbox_1         | During handling of the above exception, another exception occurred:
netbox_1         | 
netbox_1         | Traceback (most recent call last):
netbox_1         |   File "./manage.py", line 10, in <module>
netbox_1         |     execute_from_command_line(sys.argv)
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
netbox_1         |     utility.execute()
netbox_1         |   File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 395, in execute
nginx_1          | 2020/05/17 02:09:02 [notice] 1#1: using the "epoll" event method
nginx_1          | 2020/05/17 02:09:02 [notice] 1#1: nginx/1.17.10
nginx_1          | 2020/05/17 02:09:02 [notice] 1#1: built by gcc 9.2.0 (Alpine 9.2.0) 
nginx_1          | 2020/05/17 02:09:02 [notice] 1#1: OS: Linux 4.15.0-96-generic
nginx_1          | 2020/05/17 02:09:02 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
nginx_1          | 2020/05/17 02:09:02 [notice] 1#1: start worker processes
nginx_1          | 2020/05/17 02:09:02 [notice] 1#1: start worker process 6
nginx_1          | 2020/05/17 02:16:57 [notice] 1#1: signal 15 (SIGTERM) received, exiting
nginx_1          | 2020/05/17 02:16:57 [notice] 6#6: exiting
nginx_1          | 2020/05/17 02:16:57 [notice] 6#6: exit
nginx_1          | 2020/05/17 02:16:57 [notice] 1#1: signal 17 (SIGCHLD) received from 6
nginx_1          | 2020/05/17 02:16:57 [notice] 1#1: worker process 6 exited with code 0
nginx_1          | 2020/05/17 02:16:57 [notice] 1#1: exit

this was also reported in https://networktocode.slack.com/archives/CD23LP8BC/p1589066827266200

[cimnine]

Did you upgrade from a previous version or was this a vanilla installation?

Why use an arbitrary UID instead of a pre-created user?

Because it does not matter, does it? As far as I understand Linux the username is a convenience for us humans to work with the system. On the system level it works with user id's. You can set file permissions for any arbitrary (valid) user id. /etc/passwd is "just" a convenience to add a name to a user id (and some other convenient properties like the start shell, the home directory, etc.).

We've chosen this approach because it has the least impact and is the most flexible solution. For example OpenShift assigns your container a "random" user id anyway (which is 1000 most of the time, but afaik in theory it could be random). So a pre-created user scenario would not work in that case.

[jkldgoefgkljefogeg]

This was on a fresh install (multiple users ran into this issue and mitigated it by dropping uid in docker-compose.yml)

[weisdd]

@jkldgoefgkljefogeg I've just tried to use a freshly installed netbox (sha256:0b27442b40c38415523580aff3a65f2da4cee77d21d7065e1f37812873a5c22e, which corresponds to the latest tag at the moment) and didn't see any issues with the file permissions.

As for the UID, +1 to @cimnine 's reply - whenever possible, software should be able to work with an arbitrary user id. That's true for both OpenShift and K8s with Pod Security Policies enabled.
The difference here comes with the fact that a non-existent user (=not described in /etc/passwd) gets automatically assigned to gid 0, that's why it becomes important to set "chmod -R g=u" for the files and folders, which will store new data. If you go a bit further and change default umask, then uid change while still having write permissions will be possible.
Also, the non-existent user's home dir would be "/". - So, better not to rely on HOME and explicitly specify all the locations the software needs to take into account (e.g. NLTK_DATA - not for netbox, but anyway).
To summarize, from my experience of "fighting" OpenShift, the best practice would be the following:

1. Add a user and pass the USER instruction, so the process doesn't run as root by default, and you know from the very beginning that it actually works;
2. Add the user to root group and respectively change the file permissions, so the behaviour and the permissions are the same despite the actual user id.

[mpfaff]

I absolutely hate seeing docker containers that require root or being started with a specific user. Docker provides facilities for changing the user, and there's no reason a docker container needs to bind to root-only ports; the user can forward them to the real port.

[cimnine]

I'm closing this issue as it seems not to be a problem for most, respectively that the current solution is even the preferred solution for those that cared to comment.

[jkldgoefgkljefogeg]

I guess this is why I got permission error with uid 101, not sure how it got into such a state

$ docker exec -it netbox-docker_netbox_1 ls -l /etc/ | grep netbox
drwxr-xr-x    1 root     root          4096 Feb 22 07:29 netbox

$ docker exec -it netbox-docker_netbox_1 ls -l /etc/netbox
total 12
drwx------    3 1000     1000          4096 Feb 22 06:53 config
drwx------    2 1000     1000          4096 May 17  2020 reports
drwx------    2 1000     1000          4096 May 17  2020 scripts

[jkldgoefgkljefogeg]

Found the workaround in #447
the default file permission does not work