Datasette secret mechanism - initially for signed cookies

[simonw]

See comment in #784 (comment)

Datasette needs to be able to set signed cookies - which means it needs a mechanism for safely handling a signing secret.

Since Datasette is a long-running process the default behaviour here can be to create a random secret on startup. This means that if the server restarts any signed cookies will be invalidated.

If the user wants a persistent secret they'll have to generate it themselves - maybe by setting an environment variable?

[simonw]

I previously solved this for the datasette-auth-existing-cookies plugin as described in this issue: simonw/datasette-auth-existing-cookies#1

Concrete plan: you have to pass a secret to the class constructor. The Datasette plugin (the code in __init__.py) uses the following in order of preference (first things are most preferred):

• A plugin configuration option called cookie_secret - which can be protected by this mechanism: https://datasette.readthedocs.io/en/stable/plugins.html#secret-configuration-values
• A JSON configuration file in the user_state_dir file, if it exists
• If that does not exist, a secret is generated and written to that JSON file

I originally planned to have separate support for an environment variable, but the existence of the secret configuration values mechanism means this is already handled.

[simonw]

That user_state_dir solution may have been more trouble than it was worth though - I seem to remember it causing issues on some hosting providers.

[simonw]

Maybe Datasette should have a --secrets=path/to/secrets.json command-line option for storing these?

[simonw]

I'm going to use https://github.com/pallets/itsdangerous for this.

Annoyingly they're very close to release v2.0 which adds support for key rotation... but it's not quite out of pre-release yet. I'll go with 1.1.0 for the moment and upgrade to 2.0 as soon as that is out.

[simonw]

First version of cookie signing will use a secret that is either pulled from DATASETTE_SECRET environment variable or generated every time the server starts. I'll add a non-environment-variable based secret later.

[simonw]

... actually no I'll do it using a CLI option that can also be in an environment variable:

https://click.palletsprojects.com/en/7.x/options/#values-from-environment-variables

@click.command()
@click.option('--secret', envvar='DATASETTE_SECRET')
def greet(secret):
    ...

[simonw]

I'll add two utility methods to the Datasette class:

• datasette.sign(value, "namespace") - returns signed string
• datasette.unsign(signed, "namespace") - returns value OR raises BadSignature

[simonw]

Documentation for those new methods: https://github.com/simonw/datasette/blob/e28207e76ec3b26b2c396370fd3fb325a60bfd49/docs/internals.rst#signvalue-namespacedefault

[simonw]

This is nearly ready to close. I'm going to add documentation for --secret and the DATASETTE_SECRET environment variable.

[simonw]

I'll add a section about secrets to this page: https://datasette.readthedocs.io/en/latest/config.html

[simonw]

That documentation: https://github.com/simonw/datasette/blob/c818de88a9c2683437875f788e325d911c8b767b/docs/config.rst#configuring-the-secret