SECRET_KEY - autogeneration broken

[cherusk]

From reading the source, intended behaviour is, if SECRET_KEY is not in netbox_config parameter, it will be generated. But

      - include_role:
          name: "netbox"
        vars:
          netbox_stable: true
          netbox_database_host: localhost
          netbox_database_user: "netbox"
          netbox_database_password: "netbox"
          netbox_superuser_password: "netbox"
          netbox_socket: "0.0.0.0:9000"
          netbox_load_initial_data: false
          netbox_application_log: "file:{{ netbox_shared_path }}/application.log"
          netbox_requests_log: "file:{{ netbox_shared_path }}/requests.log"
          netbox_scripts: []
          netbox_config:
             ALLOWED_HOSTS:
               - 0.0.0.0
             MEDIA_ROOT: "{{ netbox_shared_path }}/media"
             REPORTS_ROOT: "{{ netbox_shared_path }}/reports"
             SCRIPTS_ROOT: "{{ netbox_shared_path }}/scripts"
          #   SECRET_KEY: "" # auto-generated

Is ailing with:

    qemu: TASK [_netbox : Run database migrations for NetBox] ****************************
    qemu: fatal: [localhost]: FAILED! => {"changed": false, "cmd": "./manage.py migrate --noinput", "msg": "\n:stderr: Traceback (most recent call last):\n  File \"./manage.py\", line 10, in <module>\n    execute_from_command_line(sys.argv)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/management/__init__.py\", line 381, in execute_from_command_line\n    utility.execute()\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/management/__init__.py\", line 375, in execute\n    self.fetch_command(subcommand).run_from_argv(self.argv)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/management/base.py\", line 323, in run_from_argv\n    self.execute(*args, **cmd_options)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/management/base.py\", line 361, in execute\n    self.check()\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/management/base.py\", line 390, in check\n    include_deployment_checks=include_deployment_checks,\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/management/commands/migrate.py\", line 64, in _run_checks\n    issues = run_checks(tags=[Tags.database])\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/checks/registry.py\", line 72, in run_checks\n    new_errors = check(app_configs=app_configs)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/core/checks/database.py\", line 9, in check_database_backends\n    for conn in connections.all():\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/db/utils.py\", line 216, in all\n    return [self[alias] for alias in self]\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/db/utils.py\", line 213, in __iter__\n    return iter(self.databases)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/utils/functional.py\", line 80, in __get__\n    res = instance.__dict__[self.name] = self.func(instance)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/db/utils.py\", line 147, in databases\n    self._databases = settings.DATABASES\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/conf/__init__.py\", line 79, in __getattr__\n    self._setup(name)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/conf/__init__.py\", line 66, in _setup\n    self._wrapped = Settings(settings_module)\n  File \"/srv/netbox/current/venv-py3/lib/python3.7/site-packages/django/conf/__init__.py\", line 157, in __init__\n    mod = importlib.import_module(self.SETTINGS_MODULE)\n  File \"/usr/lib/python3.7/importlib/__init__.py\", line 127, in import_module\n    return _bootstrap._gcd_import(name[level:], package, level)\n  File \"<frozen importlib._bootstrap>\", line 1006, in _gcd_import\n  File \"<frozen importlib._bootstrap>\", line 983, in _find_and_load\n  File \"<frozen importlib._bootstrap>\", line 967, in _find_and_load_unlocked\n  File \"<frozen importlib._bootstrap>\", line 677, in _load_unlocked\n  File \"<frozen importlib._bootstrap_external>\", line 728, in exec_module\n  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n  File \"/srv/netbox/releases/netbox-2.6.7/netbox/netbox/settings.py\", line 46, in <module>\n    \"Required parameter {} is missing from configuration.py.\".format(parameter)\ndjango.core.exceptions.ImproperlyConfigured: Required parameter SECRET_KEY is missing from configuration.py.\n", "path": "/srv/netbox/current/venv-py3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "state": "absent", "syspath": ["/tmp/ansible_django_manage_payload_79_dlb_l/ansible_django_manage_payload.zip", "/usr/lib/python37.zip", "/usr/lib/python3.7", "/usr/lib/python3.7/lib-dynload", "/usr/local/lib/python3.7/dist-packages", "/usr/lib/python3/dist-packages"]}

Fancied the feature, but have no time to cure myself. Should be light one though.

[lae]

I did some debugging and the secret key is properly generated and slurped, but netbox_config is not updated with it. Is set_fact not supposed to work if you're using include_role? I'm not sure how one should fix this. The role works fine if you use the roles attribute in a playbook definition (hence why this hasn't been caught by any of the tests).

[lae]

Looks like there's an upstream Ansible issue opened about this, and it seems to be a stale/forgotten issue: ansible/ansible#32713

After some testing it looks like the netbox_config variable on the include_role task takes precedence over facts. If I remove it from the include_role task/move it to the playbook vars, the role completes fine. It also works fine if I use an import_role task and keep the netbox_config var on the task like you have. I think this might be intended behaviour for Ansible (set_fact documentation also states: "Per the standard Ansible variable precedence rules, many other types of variables have a higher priority, so this value may be overridden."), so I'm closing this as wontfix.

[cherusk]

Looks like there's an upstream Ansible issue opened about this, and it seems to be a stale/forgotten issue: ansible/ansible#32713 <ansible/ansible#32713> After some testing it looks like the |netbox_config| variable on the |include_role| task takes precedence over facts. If I remove it from the |include_role| task/move it to the playbook vars, the role completes fine. It also works fine if I use an |import_role| task and keep the |netbox_config| var on the task like you have. I think this might be intended behaviour for Ansible (|set_fact| documentation also states: "Per the standard Ansible variable precedence rules, many other types of variables have a higher priority, so this value may be overridden."), so I'm closing this as wontfix.

Thanks so much Musee for swift reaction. Intriguing and non-intuitive behavior by ansible if you asked me.

…

-- BR Matthias Tafelmeier

[beddari]

As more people are starting to use the include_role pattern they will step into this, I just did 🥇

Would you be open to an implementation that did the same for netbox_config as you do in tasks/load_variables.yaml for other values? If we keep _netbox_config private the combine can work without overwriting with set_fact ...

What do you think @lae ?

[beddari]

I had a go at fixing this so that the way of using the role that @cherusk posted here also works. Not many changes are needed, so I think I'll open a PR soon and discuss this there.