Provide options for storing secrets outside the Nix store #24288

basvandijk · 2017-03-24T13:43:55Z

Introduction

Dear module authors and maintainers,

We currently have many modules that force users to store their secrets in the world-readble Nix store. This is bad for security. We should give users the option of specifying their secrets in individual files which can be stored outside the Nix store with suitable ownership and permissions. Users could then also use nixops to manage their secret files.

There's still the convenient but unsafe option of storing the secret file in the Nix store using pkgs.writeTextFile. If NixOS/nix#8 gets resolved these files can be encrypted / made private. Also see: NixOS/rfcs#5.

Proposal

The list below contains all the options that force a secret being stored in the Nix store. I propose the following:

Each option should get a warning in the documentation of the form: "Warning: this secret is stored in the world-readable Nix store!"
Each option should get an alternative passwordFile option.
For backwards compatibility the passwordFile option should get a default based on the password option:

{ config = {
    passwordFile = mkDefault (toString (pkgs.writeTextFile {
      name = "password-file";
      text = cfg.password;
    }));
  };
}

Some upstream programs don't support setting a password using a file. In that case an issue should be created in the upstream issue-tracker asking for that feature. (See Support specifying rpcpassword by file namecoin/namecoin-core#148 for example). A URL to the issue should be placed in the list below and in the documentation of the password option so that it's easier to track when it gets resolved.
If after some time (lets use September 2017 for now) the upstream developers have not provided the feature to specify the password by file, the NixOS module should be changed such that the config file that contains the password is written to /run before the service starts up. So something like the following:

{
  systemd.services.my-service = {  
    preStart = ''
      cat > /run/my-service/config << EOF
      ...
      password = $(cat "${cfg.passwordFile}")
      ...
      EOF
    '';
    script = "${pkgs.myService}/bin/my-service --config=/run/my-service/config";
  };
}

Lets use this issue for planning and to track progress. Please mention in the comments if you have provided a passwordFile option for one of the options below. Then I check the box to indicate it has been resolved. See PR wordpress: replace the dbPassword option with dbPasswordFile #24146 for reference.
If we make sure the new options are backwards compatible we could consider cherry-picking them onto release-17.03 making sure users get these security fixes ASAP.

Secret options

This list was compiled by running the following in <nixpkgs> and manually inspecting and processing the result:

find . -type f -exec grep --color -nH -i -E -e '(secret|pass|key)' {} +`

The text was updated successfully, but these errors were encountered:

ip1981 · 2017-03-24T13:50:16Z

FYI, check out how I handle secrets throughout all the apps here: https://github.com/ip1981/nixsap/tree/master/modules/apps

Especially, with Jenkins :)

teh · 2017-03-24T13:50:31Z

I suspect @fpletz is now the maintainer for gitlab (sorry, I see you'r name is in this list a lot!)

edolstra · 2017-03-24T14:08:31Z

Wow, I didn't know the situation was this bad. All of these options should be removed really. It's almost as if people don't realize that the Nix store is world readable...

jml · 2017-03-24T14:13:58Z

@basvandijk — any plan for preventing future changes adding secrets to the Nix store, beyond eternal vigilance?

Ekleog · 2017-03-24T14:28:17Z

Hmm, I didn't re-read NixOS/nix#8 completely, but it seemed to me last time that @edolstra 's solution using encryption in the store and decryption at startup time was working and there was mostly bikeshedding about encryption vs ACLs?

This would be much easier than trying to patch every single upstream program that does not accept password files, especially given that some may not be willing to do it as it adds quite a bit of complexity. Wouldn't it?

rnhmjoj · 2017-03-24T14:32:51Z

Besides being quite inconvenient, storing passwords/keys in a file with restricted access outside the store, may not solve the problem: they could end up in a systemd environment file or a unit file if you need to pass those as an command line argument.
Also expecting everyone upstream to comply seems incredibly too optimistic for me.

basvandijk · 2017-03-24T14:38:19Z

any plan for preventing future changes adding secrets to the Nix store, beyond eternal vigilance?

@jml we could add something to the PULL_REQUEST_TEMPLATE.md instructing contributors to use passwordFile instead of password options.

basvandijk · 2017-03-24T14:43:22Z

All of these options should be removed really...

@edolstra we could do that eventually but to ease the transition we should first provide a backwards compatible passwordFile option, then in a next release we could start throwing warnings when users use the password option and finally in a third release we can remove the password options.

basvandijk · 2017-03-24T14:48:58Z

@Ekleog regarding NixOS/nix#8, even if we have the ability to encrypt files in the Nix store I think it would be best to only encrypt files that should be encrypted. Currently we have big config files that somewhere contain a password. It would nicer if the config file remains unencrypted because then it can be shared and it makes debugging easier. Only the password needs to be encrypted. So having passwords in individual files would still be desirable.

edolstra · 2017-03-24T14:54:39Z

@basvandijk The encryption stuff allows you to encrypt only the "secret" parts of a configuration file. See edolstra@4c82120#diff-6c3fcb531890fdce200531b9ac69e4f8R14 for an example.

rnhmjoj · 2017-03-24T14:58:15Z

@basvandijk Sometimes even the password stored in the configuration file needs to be readable. dnschain, for example, parses namecoin.conf to connect to the rpc server.

basvandijk · 2017-03-24T15:55:37Z

@rnhmjoj lets see how upstream reacts to a request for a rpcpasswordFile parameter...

basvandijk · 2017-03-24T15:58:32Z

The encryption stuff allows you to encrypt only the "secret" parts of a configuration file.

@edolstra that's great! What needs to be done to get this merged into Nix?

edolstra · 2017-03-24T16:21:55Z

Well, it's not clear whether this is the way to go. @kevincox listed the issues here: NixOS/nix#8 (comment)

basvandijk · 2017-03-24T17:14:07Z

@rnhmjoj regarding namecoind, would cookie based authentication work?

basvandijk · 2017-03-24T17:57:04Z

Besides being quite inconvenient, storing passwords/keys in a file with restricted access outside the store, may not solve the problem: they could end up in a systemd environment file or a unit file if you need to pass those as an command line argument.

@rnhmjoj regarding secrets in systemd unit files, we can always create a wrapper script that cats the password file and passes that on to the original script. I do something similar here.

mbrgm · 2017-03-24T23:37:07Z

@basvandijk Regarding upstream changes for password file options: I think some cat/sed trickery, maybe creating a config file in /run from a template should also do the job for most cases where upstream doesn't have or doesn't want to add a password file, shouldn't it?

basvandijk · 2017-03-25T01:38:49Z

@mbrgm sure and we should do that in case upstream doesn't provide a password file option.

rasendubi · 2017-03-25T12:10:07Z

@basvandijk I would not tick the package until the PR is merged.

rnhmjoj · 2017-03-25T14:59:36Z

@basvandijk That seems a valid alternative for authenticating to namecoind however dnschain does not support it, so it would break the service. Anyway, thank you for opening the issue.

basvandijk · 2017-03-25T18:13:02Z

I would not tick the package until the PR is merged.

@rasendubi makes sense. I've unticked the wordpress checkbox.

edwtjo · 2017-03-25T20:29:16Z

Well for aiccu; SixXS is closing down its IPv6 tunnel in June so it doesn't seem worth the effort to create a patch for aiccu to support password files. Lets just remove the service in 0606.

zimbatm · 2017-03-26T14:59:52Z

Nix encryption is going to take a while to get there given nix's release history speed. I don't think it should be a blocker for trying alternative implementations.

On the nixpkgs side, the protection would be based on unix file ACL. How about treating secrets like any other state? We could introduce a "mkState" interface that defines any kind of state reference on the filesystem.

let
  postgresState = mkState {
    type = "directory";
    name = "postgres";
    owner = "postgres";
    group = "postgres";
    mode = "0700";
    mustExist = false;
    # run a command if it's missing
    onMissing = "pg_init";
  };
  assert (toString postgresState) == "/var/lib/postgres";
  # mkSecret is a specialization of mkState with a default dir to /run/keys/${name}, mode = 0700 and mustExist = true  
  nginxSecret = mkSecret {
    owner = "nginx";
  };

This would translate to (1) some activation script actions like creating the state dir (2) systemd service to initialize and/or check the secret, which can then be used as a dependency for other services.

I know it's still pretty vague but hopefully enough to convey the idea.

domenkozar · 2021-12-16T10:52:30Z

There are now docs for how to handle this for new modules at #142282

kevincox · 2021-12-16T14:07:05Z

These docs handle the easy part of the problem. They are about how use use secrets outside of the store, not how to manage the secrets themselves. Ideally we wouldn't need to push this complexity onto the user.

ncfavier · 2021-12-16T16:21:15Z

We can probably tick the networking.wireless.networks.*.psk box now that #134618 was merged.

nixos-discourse · 2022-04-03T11:47:35Z

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/agenix-encrypted-plaintext-passwords-and-builtins-readfile/18425/8

When using the declarative shared folder configuration for resilio sync it is now possible to pass a path from which to read the secret should be read at runtime. The path will not be added to the nix store. The 'secret' parameter to specify the secret directly is still supported. This option will still store the secret in the nix store. This commit follows the pattern described in this issue, for upstream programs that do not provide support for setting a password using a file: #24288

When using the declarative shared folder configuration for resilio sync it is now possible to pass a path from which to read the secret should be read at runtime. The path will not be added to the nix store. The 'secret' parameter to specify the secret directly is still supported. This option will still store the secret in the nix store. This commit follows the pattern described in this issue, for upstream programs that do not provide support for setting a password using a file: NixOS#24288

…ore: At this point this is basically a full rewrite of this module, which is a breaking change and was necessary to properly expose the useful parts of hostapd's config. The notable changes are: - `hostapd` is now started with additional systemd sandbox/hardening options - A single-daemon can now manage multiple distinct interfaces, which is why all configuration had to be moved into `hostapd.interfaces` - By default WPA3-SAE will be used, but WPA2 and WPA3-SAE-TRANSITION are supported, too - Added passwordFile-like options for wpa and sae - Add new relevant options for MAC ACL, WiFi5, WiFi6 and WiFi7 configuration - Removes `with lib;` and thus makes this module conformant with [RFC42](https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md), so it can be marked as done in NixOS#144575. - Related to NixOS#49171. Closes NixOS#24288.

At this point this is basically a full rewrite of this module, which is a breaking change and was necessary to properly expose the useful parts of hostapd's config. The notable changes are: - `hostapd` is now started with additional systemd sandbox/hardening options - A single-daemon can now manage multiple distinct interfaces, which is why all configuration had to be moved into `hostapd.interfaces` - By default WPA3-SAE will be used, but WPA2 and WPA3-SAE-TRANSITION are supported, too - Added passwordFile-like options for wpa and sae - Add new relevant options for MAC ACL, WiFi5, WiFi6 and WiFi7 configuration - Removes `with lib;` and thus makes this module conformant with [RFC42](https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md), so it can be marked as done in NixOS#144575. - Related to NixOS#49171. Closes NixOS#24288.

ppom0 · 2023-11-23T10:31:12Z

Some boxes can be checked (✅ done, 🙅 module removed, ❓ option doesn't seem to exist anymore):

basicAuth ✅
services.almir.director_password 🙅
services.btsync.httpPass 🙅
services.buildbot-worker.workerPass ✅
services.cassandra.keyStorePassword ❓
services.cassandra.trustStorePassword ❓
services.cfdyndns.apikey ✅
services.coturn.cli-password ✅ (can be stored as a hash)
services.coturn.static-auth-secret ✅
services.crowd.openidPassword ✅
services.dd-agent.api_key ✅ (now services.datadog-agent.apiKeyFile)
services.ddclient.password ✅
services.frab.secretKeyBas 🙅
services.grafana.database.password ✅ (grafana-specific alternative pointed in option's doc)
services.grafana.security.adminPassword ✅ (grafana-specific alternative pointed in option's doc)
services.grafana.security.secretKey ✅ (grafana-specific alternative pointed in option's doc)
services.httpd.extraSubservices.* ❓ (maybe I'm missing something)
services.panamax.secretKey 🙅
services.prometheus.*.consul_sd_config.password ✅ (now services.prometheus.scrapeConfigs.*.consul_sd_configs.*.basic_auth.password_file)
services.prometheus.*.scrape_config.basic_auth.password ✅ (now services.prometheus.scrapeConfigs.*.consul_sd_configs.*.basic_auth.password_file)
services.redis.requirePass ✅ (now services.redis.servers.<name>.requirePassFile)
services.tor.torsocks.socks5Password ✅ (although it should maybe first be better documented that using systemd.services.tor.serviceConfig.EnvironmentFile is possible)
services.tt-rss.database.password ✅
services.wakeonlan.interfaces.*.password 🙅

Maybe some unsecure options appeared in new modules since then!

CMCDragonkai · 2023-11-24T00:06:40Z

Is there a conclusion to this issue that specifies exactly what behaviour is?

ppom0 · 2023-12-12T11:16:29Z

Can someone edit the issue? @basvandijk maybe :)

joachifm added 1.severity: security 6.topic: nixos labels Mar 24, 2017

basvandijk changed the title ~~Provide options of storing secrets outside the Nix store~~ Provide options for storing secrets outside the Nix store Mar 24, 2017

basvandijk mentioned this issue Mar 25, 2017

ssmtp: use the authPassFile option instead of authPass #24331

Merged

basvandijk mentioned this issue Mar 26, 2017

cadviser: add storageDriverPasswordFile option #24341

Merged

TLATER mentioned this issue Jan 25, 2022

Allow specifying HTTP basic auth credentials from a file tsl0922/ttyd#872

Open

aanderse mentioned this issue Mar 27, 2022

nixos/openconnect: Add NixOS module for declarative OpenConnect VPNs #165883

Merged

13 tasks

roberth mentioned this issue Jun 12, 2022

treewide: markdown option docs #176146

Merged

13 tasks

Artturin modified the milestones: 21.05, 23.05 Dec 31, 2022

cyounkins mentioned this issue Feb 2, 2023

NixOS power.ups module does not generate all needed config files #91681

Closed

oddlama mentioned this issue Mar 22, 2023

nixos/hostapd: rewrite to support multi-AP, WiFi 6, password from file, and more #222536

Merged

12 tasks

RaitoBezarius modified the milestones: 23.05, 23.11 May 31, 2023

RaitoBezarius closed this as completed in #222536 Jul 8, 2023

jtojnar reopened this Jul 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Provide options for storing secrets outside the Nix store #24288

Provide options for storing secrets outside the Nix store #24288

basvandijk commented Mar 24, 2017 •

edited by samueldr

ip1981 commented Mar 24, 2017

teh commented Mar 24, 2017

edolstra commented Mar 24, 2017

jml commented Mar 24, 2017

Ekleog commented Mar 24, 2017

rnhmjoj commented Mar 24, 2017

basvandijk commented Mar 24, 2017

basvandijk commented Mar 24, 2017

basvandijk commented Mar 24, 2017

edolstra commented Mar 24, 2017

rnhmjoj commented Mar 24, 2017

basvandijk commented Mar 24, 2017

basvandijk commented Mar 24, 2017 •

edited

edolstra commented Mar 24, 2017

basvandijk commented Mar 24, 2017

basvandijk commented Mar 24, 2017

mbrgm commented Mar 24, 2017

basvandijk commented Mar 25, 2017

rasendubi commented Mar 25, 2017

rnhmjoj commented Mar 25, 2017

basvandijk commented Mar 25, 2017

edwtjo commented Mar 25, 2017

zimbatm commented Mar 26, 2017 •

edited

domenkozar commented Dec 16, 2021

kevincox commented Dec 16, 2021

ncfavier commented Dec 16, 2021

nixos-discourse commented Apr 3, 2022

ppom0 commented Nov 23, 2023 •

edited

CMCDragonkai commented Nov 24, 2023

ppom0 commented Dec 12, 2023

Provide options for storing secrets outside the Nix store #24288

Provide options for storing secrets outside the Nix store #24288

Comments

basvandijk commented Mar 24, 2017 • edited by samueldr

Introduction

Proposal

Secret options

ip1981 commented Mar 24, 2017

teh commented Mar 24, 2017

edolstra commented Mar 24, 2017

jml commented Mar 24, 2017

Ekleog commented Mar 24, 2017

rnhmjoj commented Mar 24, 2017

basvandijk commented Mar 24, 2017

basvandijk commented Mar 24, 2017

basvandijk commented Mar 24, 2017

edolstra commented Mar 24, 2017

rnhmjoj commented Mar 24, 2017

basvandijk commented Mar 24, 2017

basvandijk commented Mar 24, 2017 • edited

edolstra commented Mar 24, 2017

basvandijk commented Mar 24, 2017

basvandijk commented Mar 24, 2017

mbrgm commented Mar 24, 2017

basvandijk commented Mar 25, 2017

rasendubi commented Mar 25, 2017

rnhmjoj commented Mar 25, 2017

basvandijk commented Mar 25, 2017

edwtjo commented Mar 25, 2017

zimbatm commented Mar 26, 2017 • edited

domenkozar commented Dec 16, 2021

kevincox commented Dec 16, 2021

ncfavier commented Dec 16, 2021

nixos-discourse commented Apr 3, 2022

ppom0 commented Nov 23, 2023 • edited

CMCDragonkai commented Nov 24, 2023

ppom0 commented Dec 12, 2023

basvandijk commented Mar 24, 2017 •

edited by samueldr

basvandijk commented Mar 24, 2017 •

edited

zimbatm commented Mar 26, 2017 •

edited

ppom0 commented Nov 23, 2023 •

edited