django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY environment variable

[timdonovanuk]

Hi,

I am getting:

2021-06-03T22:57:45+01:00 docker web[22644]: django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY environment variable
2021-06-03T22:57:47+01:00 docker web[22644]: /usr/local/lib/python3.9/site-packages/environ/environ.py:637: UserWarning: Error reading back/.env - if you're not configuring your environment separately, check this.

and the same in the celery container. I'm sure its something dumb I'm doing but I cannot figure it out!

I know my .env file is being read by the compose service as I am setting COMPOSE_PROJECT_NAME=chiefonboarding_footest (and the resulting stack is called 'chiefonboarding_footest').

Also please kindly consider creating a demo docker example that just works - it allows people to try your cool looking app much easier! What I mean by this is applying some KISS:

• remove the caddy dependency (if someone is trying this, there is a good chance they already have a reverse proxy running themselves)
• remove the depenency on .env files (which are currently not even supported in Portainer 2.5 so nobody using Portainer can deploy your app)
• redis and celery? What volumes of people are being onboarded here exactly!? :)
• it's welcome to have the database seperate, so this is great
• organization name from last steps could easily be an env variable
• 'Last steps' - a common method is to detect a first run (by connecting to the database and seeing no tables) and create and set up test data as part of the database initlisation

Basically look at any linuxserver docker stack example on how to make an app accessible for the masses.

Thanks!

[timdonovanuk]

I said it would be something dumb - in an attempt to get this working in Portainer I had removed the env_file reference in the compose file! Can I rename this issue to "please make docker demo simplier?" :)

[GDay]

I am glad you got it sorted! Thanks for writing this all out, I appreciate it and sorry about the confusion. I am not familiar with portainer, so I can't really comment on that.

remove the caddy dependency (if someone is trying this, there is a good chance they already have a reverse proxy running themselves)

I added that for people that install this on a fresh server. Caddy also automatically adds the SSL certificates needed for https.

remove the depenency on .env files (which are currently not even supported in Portainer 2.5 so nobody using Portainer can deploy your app)

The only reason why I set up the .env files config is because it needs to have the same values in both the celery part as well as the web app. Duplicating values felt a little odd. I am not experienced enough with Docker to actually tell what the best approach is in this case (with having both celery and the app).

redis and celery? What volumes of people are being onboarded here exactly!? :)

Haha! Since the human population seems to be increasing every day... better be safe than sorry, I guess? lol. You are right, celery is probably indeed a bit overkill here. It needs scheduled tasks to basically do two things: update new hires with tasks/messages for that day and connect new Slack users to accounts (the latter will be migrated to a webhook soon).

Hmm.. thinking about this, I can probably set up Django Q with the Postgres database as the message broker. That would get rid of both Celery and Redis. I will keep this issue open until I (or someone else) fixed that. This would also resolve the .env dependency issue. Thanks for the tip!

it's welcome to have the database seperate, so this is great

Great! Thanks.

organization name from last steps could easily be an env variable
'Last steps' - a common method is to detect a first run (by connecting to the database and seeing no tables) and create and set up test data as part of the database initlisation

Agreed. That's really easy to add and will fix that soon! Thanks!

Can I rename this issue to "please make docker demo simplier?" :)

Haha, sure!

[timdonovanuk]

Haha thanks, all of the above is appreciated!

• You are right, Caddy does make sense (and luckily I know enough to know I can remove it and use my existing nginx + letsencrypt services).
• Portainer is a super nice UI for manging docker containers and swarms. It's extremely popular (although less so with the rise of k8s, but it still has a huge audience). .env file support is actually being released in the next major version but not quite there yet.
• "Haha! Since the human population seems to be increasing every day." :D

I am still struggling with this, I get an error when running the final command:

user@docker:/opt/chiefonboarding# docker-compose run web python3 manage.py loaddata welcome_message.json
Starting chiefonboarding_foo_postgres_1 ... done
/usr/local/lib/python3.9/site-packages/environ/environ.py:637: UserWarning: Error reading back/.env - if you're not con                                                                                                                                                                                                                                                                                                                    figuring your environment separately, check this.
  warnings.warn(
/usr/local/lib/python3.9/site-packages/fuzzywuzzy/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Inst                                                                                                                                                                                                                                                                                                                    all python-Levenshtein to remove this warning
  warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/apps/registry.py", line 156, in get_app_config
    return self.app_configs[app_label]
KeyError: 'organizatio'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/core/serializers/python.py", line 153, in _get_model
    return apps.get_model(model_identifier)
  File "/usr/local/lib/python3.9/site-packages/django/apps/registry.py", line 206, in get_model
    app_config = self.get_app_config(app_label)
  File "/usr/local/lib/python3.9/site-packages/django/apps/registry.py", line 163, in get_app_config
    raise LookupError(message)
LookupError: No installed app with label 'organizatio'.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/manage.py", line 21, in <module>
    main()
  File "/app/manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_l                                                                                                                                                                                                                                                                                                                    ine
    utility.execute()
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 330, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 371, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/commands/loaddata.py", line 72, in handle
    self.loaddata(fixture_labels)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/commands/loaddata.py", line 114, in loaddata
    self.load_label(fixture_label)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/commands/loaddata.py", line 172, in load_label
    for obj in objects:
  File "/usr/local/lib/python3.9/site-packages/django/core/serializers/json.py", line 70, in Deserializer
    yield from PythonDeserializer(objects, **options)
  File "/usr/local/lib/python3.9/site-packages/django/core/serializers/python.py", line 91, in Deserializer
    Model = _get_model(d["model"])
  File "/usr/local/lib/python3.9/site-packages/django/core/serializers/python.py", line 155, in _get_model
    raise base.DeserializationError("Invalid model identifier: '%s'" % model_identifier)
django.core.serializers.base.DeserializationError: Problem installing fixture '/app/fixtures/welcome_message.json': Invalid model identifier: 'organizatio.welcomemessage'                                                                                                                                                                                                                                                                                                                

Is organizatio is missing an 'n' somewhere? I tried logging in anyway but get bounced back to the login screen immeditely, with the errors (from chrome dev console):

:9912/api/org/detail:1 Failed to load resource: the server responded with a status of 403 (Forbidden)
5aebfe6.js:1 Uncaught (in promise) TypeError: Cannot read property 'data' of undefined
    at 5aebfe6.js:1
(anonymous) @ 5aebfe6.js:1
VM132:1 GET http://docker.home:9912/api/org/detail 403 (Forbidden)
5aebfe6.js:1 Uncaught (in promise) TypeError: Cannot read property 'data' of undefined
    at 5aebfe6.js:1

[timdonovanuk]

I think it might be a typo here:

ChiefOnboarding/back/fixtures/welcome_message.json

Line 120 in 409857e

"model": "organizatio.welcomemessage",

?

[timdonovanuk]

I fixed this in the running container and ran the import command and it loaded the data okay now (btw docker-compose run spawns a new container and leaves it running, I'm not sure this is necessary - docker exec -it <container name> <command> would be more appropraite here).

I'm still getting bounced back to the login screen though:

VM331:1 GET http://docker.home:9912/api/org/detail 403 (Forbidden)
5aebfe6.js:1 Uncaught (in promise) TypeError: Cannot read property 'data' of undefined
at 5aebfe6.js:1

[timdonovanuk]

Possibly the allowed hosts var (again IMO more uncessary complexity!)? I changed it to * in the env file but now I'm wondering if I need to rebuild the stack from scratch (and go through all of the above) as it possibly is not re-reading the env file between start/stops.

[timdonovanuk]

Rebuilt the stack with allowed_hosts *. Still getting 403. I'm bowing out for now - my 3 hours journey is a good example of why a KISS 'just works' docker-compose file would be great :) Thanks anyway, will try this again at a later date.

[GDay]

Is organizatio is missing an 'n' somewhere?

Yup, sorry, this is a known bug, the current PR that's open fixes that, I haven't had time yet to fully merge that yet. Will get that done asap.

btw docker-compose run spawns a new container and leaves it running

Oh, I had no clue. Will get that replaced in the docs.

GET http://docker.home:9912/api/org/detail 403 (Forbidden)

ChiefOnboarding doesn't really support HTTP. HTTPS is technically a must. I think these settings are blocking you:

SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE=True

I'm bowing out for now - my 3 hours journey is a good example of why a KISS 'just works' docker-compose file would be great :)

Yeah, I see what you mean. The current docker-compose works just fine on a clean vps linked directly to a domain (except for the fixture thing), but it won't work well when it needs run along with other docker containers. I should have some time tomorrow to take a better look at this and make things simpler to run anywhere. I am sorry it didn't work out for you right now.

Thanks for giving it a shot though!

[GDay]

Alright, what I have done to make this all easier (this has been pushed in the latest release):

• Celery and Redis are now replaced by Django Q with a Django cache table.
• Environment files are gone now, everything is defined in the compose file.
• There is a new variable called HTTP_INSECURE. When set to True, it will use unsigned cookies (which should allow you to login without issues over an insecure connection).
• Fixtures are fixed.
• You can now specify a default password and email address for the first admin account (you can add more admin accounts in the dashboard and also remove the first one). If those credentials aren't specified, it will generate them which can then be found in the logs.
• It will auto apply all fixtures, so the dashboard has mock data in it. Obviously, this can all be removed as well.
• Supervisor runs both Django and Django Q in the same container, so only the database container and the web container are necessary to run this.

Hopefully, this makes things much easier. And it should be possible to run this in Portainer as well (though, I have not tested that).

Feel free to give it another go... or don't, that's cool too. :) Thanks again for all the feedback! I really appreciate it.

[timdonovanuk]

Hey thanks for all this - super appreciated :) However now the web container literally does nothing...no logs, nothing! :/

Both caddy and the db do print logs, but web..ziltch.

version: '3'

services:
  db:
    image: postgres:latest
    restart: always
    expose:
      - "5432"
    volumes:
      - /opt/chiefonboarding/pgdata:/var/lib/postgresql/data/
    environment:
      - POSTGRES_DB=chiefonboarding
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres


  web:
    image: chiefonboarding/chiefonboarding:latest
    restart: always
    ports:
      - "9082:8000"
    environment:
      - SECRET_KEY=somethingsupersecret
      - BASE_URL=https://test.chiefonboarding.com
      - DATABASE_URL=postgres://postgres:postgres@db:5432/chiefonboarding
      - ALLOWED_HOST=test.chiefonboarding.com
      - DEFAULT_FROM_EMAIL=hello@chiefonboarding.com
    depends_on:
      - db


  caddy:
    image: caddy:2.3.0-alpine
    restart: unless-stopped
    ports:
      - "9080:80"
      - "9081:443"
    volumes:
      - /opt/chiefonboarding/caddy/Caddyfile:/etc/caddy/Caddyfile
      - /opt/chiefonboarding/caddy/site:/srv
      - /opt/chiefonboarding/caddy/data:/data
      - /opt/chiefonboarding/caddy/onfig:/config

[GDay]

Thanks for getting back to me. That's very weird, somehow I can't reproduce it. I just created a fresh VPS (with docker and docker-compose preinstalled) and used the script above to run this app. It took some time to see some results in the terminal (3-5 minutes maybe). Then I got some Caddy errors, but that was obvious because I hadn't set up the folders correctly for the config that you have. Then I ran just the web one (with docker-compose up web) and it showed me this:

web_1    | [2021-06-14 15:51:57 +0000] [8] [INFO] Starting gunicorn 20.0.4
web_1    | [2021-06-14 15:51:57 +0000] [8] [INFO] Listening at: http://0.0.0.0:8000 (8)

Indicating that it is running nicely. So I am really not sure why it didn't start correctly for you. Perhaps try to start just that web container first, stop it and then try everything?

[timdonovanuk]

Hi there :) I tried again, this time I got as far as "Caddy works!"

If you don't want to have a secure connecting and want to connect over http (not secure, and you will have to change the Caddy file below), then add HTTP_INSECURE=True to your environment variables.

Change the Caddy file below to what? Because insecure or not, the web service still exports port 8000. So Caddy is still expecting this.

Thank you!

[GDay]

Okay, cool. In the Caddy file, the example says this:

test.chiefonboarding.com {
  reverse_proxy web:8000
}

For insecure access (http), you will have to prefix it with http://, otherwise it will try to get a certificate for your domain. So it would be like this:

http://test.chiefonboarding.com {
  reverse_proxy web:8000
}

Obviously, change the domain name to something that works for you. I hope that works for you.