Fixed #20081 -- Minimized risk of SECRET_KEY leak.

[aj-may]

By default set the SECRET_KEY value to be loaded from env vars. If not
set and DEBUG==True generate a random SECRET_KEY. Also adds the
generatesecret management command, which prints a random secret key to
stdout.

[PaulMcMillan]

https://code.djangoproject.com/ticket/20081

As a general principle, I'm opposed to this approach. Moving the secret key into an environment variable instead of a file doesn't really reduce the odds of it leaking (it may increase them - env variables have historically been leaky, and still must be stored somewhere on-disk in a user-accessible file in order to persist across reboots).

The root of the patch (using os.getenv()) is trivial to implement if a user prefers to inject the secret key that way.

Your patch generates a new secret key whenever debug=True. This means that sessions no longer work across dev server restarts, meaning a developer has to re-login to their app every time a file changes - not a good first user experience.

Furthermore, your patch doesn't address backwards compatibility and documentation.

If you reimplement this storing the secret key in a file rather than as an environment variable, and address the issues raised above, I think this will be a good addition.

[aj-may]

@PaulMcMillan
First off I really appreciate the feedback. I did not yet address backwards compatibility and documentation because I wanted to get feedback on this approach first.

I agree that env variables can be leaked just as easily as a file if set up poorly. However, I also see that there are many tools and deployment options available now that help the user do this correctly. The biggest issue with the SECRET_KEY leaking is that everyone is committing their settings.py file with the SECRET_KEY to their repos. Obviously this is a very bad practice.

A google search for "where should I keep my Django secret key" turns up many results pushing this same approach. ENV vars for configuration is also heavily pushed by Heroku and is outlined in The 12 Factor App - http://12factor.net/

Also, so I fully understand your ideal implementation, where would this file be located, what would it be called, what format would it be in, how would it be read in, and what would the server do if the file could not be found?

I am in complete agreement that not keeping sessions across dev server restarts would be really annoying, and should be modified to store the secret key at least semi-permanently. I will work on coming up with a new solution for that.

[aj-may]

In addition: http://stackoverflow.com/questions/12461484/is-it-secure-to-store-passwords-as-environment-variables-rather-than-as-plain-t

The first two answers make some great points.

@PaulMcMillan

[PaulMcMillan]

While it's true that you can't commit a password which is stored only in an environment variable, you can commit the file that's storing that variable so that it gets restored when you reboot the server, which boils down to the same problem.

Environment variables work well with Heroku's model because the container your application runs within is explicitly not multi-tenant - you can't see anyone else's processes. We can't make that assertion generally about Django deployments.

I don't have strong feelings about where the file is located, and what it is called. It should be created with 0700 permissions so that other users can't read it. Part of the reason I haven't patched this myself is that it's not immediately obvious where the file should live - in proper deployments, the webserver only has the capability to write to a few selected directories. It may not be possible to generally know that in advance. This is one problem Horizon's code has - it makes it easy for a new user to get started, but is manifestly inappropriate for many deployment scenarious.

I'd suggest getting input from other developers, since I'm not sure.

[PaulMcMillan]

Perhaps the solution is to try to write the auto-generated secret key into a file the same directory as the settings file. If that's not writable, fail with an easy-to-debug error.

It might make sense to discuss including some default VCS ignore files for that directory explicitly preventing the secret key from being committed.

[aj-may]

@PaulMcMillan
That was going to be my next question.. If its just a file, It will be just as easy to commit to a repo.

It might make sense to discuss including some default VCS ignore files for that directory explicitly preventing the secret key from being committed.

I think I can work with that.

[PaulMcMillan]

If its just a file, It will be just as easy to commit to a repo.

Yeah, and there's nothing we can do to prevent that. Any deployment scenario using environment variables also must also store them in some file (which can be easily committed to a repo).

[aj-may]

I deploy a lot of sites to heroku/dokku. I store all config in env vars and none of them are stored in a file. They are only stored on the application servers and on heroku compiled into the application "slug".

I agree that most use cases a secret key file would be a better option. I am working on an update th the pull request.

[aj-may]

@PaulMcMillan

How does this look?

[PaulMcMillan]

I still don't want to allow an empty secret key in DEBUG mode. For better or worse, people deploy real sites in debug mode, and if they use the pickle cookie deserializer with signed cookies, that would be a very bad thing.

[aj-may]

That makes sense. My idea was that a developer running the dev server shouldn't have to do anything to get working, but generating a secret key file if one doesn't exist achieves the same result.

[PaulMcMillan]

I left a few comments, please argue with me about them ;)

[aj-may]

@PaulMcMillan
updated

[PaulMcMillan]

can I get @dstufft or @alex to weigh in as a second voice on the security implications of this?

Other than my latest comments, it looks pretty good to me.

[alex]

The file should really be created with the correct mode, so it isn't readable in between.

[dstufft]

Even that's not enough. Doing this in Python is annoying.

import os

def _secure_open_write(filename, fmode):
    # We only want to write to this file, so open it in write only mode
    flags = os.O_WRONLY

    # os.O_CREAT | os.O_EXCL will fail if the file already exists, so we only
    # will open *new* files.
    # We specify this because we want to ensure that the mode we pass is the
    # mode of the file.
    flags |= os.O_CREAT | os.O_EXCL

    # Do not follow symlinks to prevent someone from making a symlink that
    # we follow and insecurely open a file.
    if hasattr(os, "O_NOFOLLOW"):
        flags |= os.O_NOFOLLOW

    # On Windows we'll mark this file as binary
    if hasattr(os, "O_BINARY"):
        flags |= os.O_BINARY

    # Before we open our file, we want to delete any existing file that is
    # there
    try:
        os.remove(filename)
    except (IOError, OSError):
        # The file must not exist already, so we can just skip ahead to opening
        pass

    # Open our file, the use of os.O_CREAT | os.O_EXCL will ensure that if a
    # race condition happens between the os.remove and this line, that an
    # error will be raised.
    fd = os.open(filename, flags, fmode)
    try:
        return os.fdopen(fd, "wb")
    except:
        # An error occurred wrapping our FD in a file object
        os.close(fd)
        raise

That snippet is something I wrote in another project, but it gives the basic idea. You can use that the same way you'd use open() (e.g. in a context manager or not). I'm not sure if we'd want the binary mode on Windows for this.. probably not.

[alex]

This definitely needs docs before this is landable, this is introducing a totally new convention.

[dstufft]

This should refuse to a secret key file that doesn't have safe permissions IMO. Much like how ssh won't use a public or private key with unsafe permissions.

[dstufft]

I'm going to follow up on the ticket for discussion about the idea in general.

[aj-may]

@alex @dstufft @PaulMcMillan

Updated. Still no docs. I want to flesh out these ideas first. I appreciate the feedback.

[Vanuan]

But if you deploy to Heroku, you'll still need to check in this file, right? This way you'll have a separate codebase in production, which isn't great.

[aj-may]

@Vanuan Heroku users will have to continue to follow Heroku's directions on using environment variables. The SECRET_KEY setting still exists as to not force the use of a secret key file.

Maybe to make it easier to switch between the two I can add a commented out line in the settings file using env vars. What do you think?

[Vanuan]

Yeah, it's a way better to have this as a default

SECRET_KEY = os.environ['SECRET_KEY']

What do you think about Rails approach: secret key is defined in a separate file (example: config/initializers/secret_token.rb). This way you can get advantages of both approaches, i.e. you can use a separate file, which you can check in or not. If you prefer not checking it in, you can use a command to generate it. If you want to have it in a repo, you can set ENV['SECRET_TOKEN'] there.

[aj-may]

@Vanuan My preference is also to use environment variables, but for many Django installations that could be insecure.

This is why I chose to make the default a file, but leave the user the option to load in a secret key however they choose.

[dbrgn]

Would a .strip() make sense here?

[aj-may]

I'm not sure. My initial thought is that the entire file, whitespace and all should be set as the SECRET_KEY. However, I do see how the current implementation could lead to unexpected results. I'd like to see if anyone else has thoughts on this.

[carljm]

I think the value should be stripped. This will still respect "internal" whitespace, but will remove leading and trailing whitespace. The most important reason for this, IMO, is that if you have SECRET_KEY = 'foo' and you move it to a file, and your editor automatically adds a final newline (which many editors do), that should not result in a subtly different secret key.

[aj-may]

@carljm good point. I'm convinced. I'll add a .strip()

[aaugustin]

This exception message is inaccurate if SECRET_KEY_FILE is set but points to an empty file.

[timgraham]

I think it would be helpful to summarize the discussion about this on the django-developers mailing list to ensure everyone is okay with the approach.

[aj-may]

Sorry for the delay. I will try and update this PR this week.

[timgraham]

Closing due to lack of activity. Please send a new PR if you get around to updating it, thanks!