Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC 0005] Nix encryption #5

Closed
wants to merge 6 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
273 changes: 273 additions & 0 deletions rfcs/0005-nix-encryption.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,273 @@
---
feature: nix-encryption
start-date: 2016-03-28
author: Eelco Dolstra
co-authors:
related-issues: https://github.com/NixOS/nix/issues/8, https://github.com/NixOS/nix/pull/329
---

# Summary
[summary]: #summary

We currently lack a way to store secret information in the Nix
store. The proposal is to add a builtin function to Nix to allow
secrets to be encrypted at evaluation time with a key. At runtime,
files containing encrypted data can be decrypted using the same key.

# Motivation
[motivation]: #motivation

It is often necessary to store secrets such as passwords or SSL
certificates in NixOS system closures. Unfortunately, the Nix store is
world-readable, so this is currently not possible in a secure way;
anybody who gains access to the filesystem of a NixOS machine can read
any file in its Nix store. Nevertheless, lots of NixOS modules *do*
store secrets in the Nix store (see
https://github.com/NixOS/nixpkgs/issues/24288), many of them lacking
warnings about the danger of doing so.

# Detailed design
[design]: #detailed-design

The proposal consists of the following components:

* A builtin function `encryptString :: Path -> String -> String` that
encrypts a string using a symmetric key stored in the specified
path. The encryption is done using libsodium's authenticated
encryption function `crypto_secretbox_easy()`. The resulting string
has the format `<{|nixcrypt:base64-encoded-data|}>`, where
`base64-encoded-data` is the base-64-encoded output of
`crypto_secretbox_easy()`, along with the nonce used.

* A command `nix decrypt` to decrypt files containing encrypted
strings produced by `encryptString`. This command searches for
`<{|nixcrypt:...|}>` fragments and decrypts them using a decryption
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has <{|nixcrypt: been selected to make it easier to do find-and-replace at the decryption phase or are there other considerations? I'm wondering if nixcrypt::<base64-salt> could be enough. It's a bit of a detail though.

key specified on the command line. Any other data is copied
verbatim.

* A command `nix generate-key` to generate a key in the format
expected by `encryptString` and `nix decrypt`.

A typical usage in a module NixOS would look like this:

```
let
configFile = pkgs.writeText "foo.conf"
''
# Store the password of the foo service in encrypted form.
password=${builtins.encryptString <nixos-store-key> cfg.password}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that cfg.password is cleartext, but it's not clear to me where it is defined and/or how the user will provide this.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm aware of two solutions to providing the password:

  • Approach 1: The user does not supply a secret at all and the derivation auto-generates the secret at startup. The secret is generated so that only root can access it and the user has to ssh into the machine as root to obtain the credential if they need it to log in.
  • Approach 2: The user logs into the machine after deploy and deposits the secret in a preconfigured location by the user after deploy and the program/service waits or fails until the secret is available.

'';
in {

systemd.services.foo = {
preStart =
''
# Decrypt the configuration file.
${config.nix.package}/bin/nix-store \
--decrypt ${toString <nixos-store-key>} \
Copy link
Member

@michalrus michalrus May 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If somebody has access to the file system (before this solution), they also have access to /proc and can read this argv, stealing the key…

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because this passes the path to the key, not the key itself.

${configFile} > /run/secret/foo.conf
Copy link
Member

@zimbatm zimbatm Mar 28, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the configuration has to be generated at runtime, what difference does it make to get the secret from the nix store or another source like hashicorp vault, /run/keys/* or an environment variable?

Copy link
Member

@arianvp arianvp Mar 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly my thought. Currently, nixops supports delivering keys out of band a well and saves them in /run/keys . I think the actual hard part is not solved by this RFC as it gives the same level of flexibility as what nixops currently already delivers! The real hard part is that we need to rewrite dozens of NixOS modules that directly store secrets such that they can retrieve their secrets from disk. And we should discourage or maybe totally deprecate any kind of other usage ...

Perhaps a larger part of the proposal isn't actually the encryption builtin, but actually rewriting modules in such way that they support this paradigm (and the nixops / vault paradigm).

Actually, I am in favor to make the nixops feature more easy to use instead of introducing an encryption primitive. I think it clashes too much with reproducibility to store nonced encrypted blobs in the nix store.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two things here I think: (1) an interface to declare that the secret should exist (or other kind of states actually). Potentially it could also have an initialization script if the secret can be dynamically generated. (2) a way to compose existing nix derivations with the state.

For (2) it would be cool if it integrated with the language. Potentially we could have a subset of nix that can be evaluated at runtime.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will FUSE wrapper on encrypted files solve this problem of requiring rewrite of modules?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zimbatm @arianvp: My understanding is that the main benefit of this approach is that some programs might not be equipped to read secrets from a file (for example, a configuration file that only accepts a secret as an inline string field). I don't know any such programs off the top of my head that are like this, though, but I guess it could happen. Are there any concrete examples where this is the case?

However, I that most programs can accept secrets from protected files outside the /nix/store and we could easily fix most NixOS services to use this approach today without any changes to Nix.

'';
# Run the service using the decrypted configuration file/
serviceConfig.Exec =
"${pkgs.foo}/bin/foo -c /run/secret/foo.conf";
};
};
```
(See below for a discussion of simplifications to this example.)

At evaluation time, this causes the following string to be emitted by
the Nix evaluator:
```
password=<{|nixcrypt:base64-encoded-data|}>
```

Note that we don't necessarily encrypt entire files (though that's
possible); generally, we'll encrypt only the sensitive parts of
files. This allows users to read non-sensitive parts of configuration
files. It *may* also enable graceful degradation if secrets cannot be
decrypted, but this is risky (see below under "Decryption failure").

The assumption is that `<nixos-store-key>` resolves to something like
`/var/lib/nixos/store-key`, which should contain a key generated by
`nix generate-key` and should obviously be readable by root only.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's say I have 3 different servers with different configs and a Hydra that pushes their config to a binary cache. Wouldn't one compromised server be able to read all the server's config from the binary cache?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zimbatm: I believe this is safe in the presence of both binary caches and distributed builds.

For binary caches, the actual cached configuration only stores encrypted secrets, so even if you mirror the configuration to other /nix/stores they won't be able to decrypt these secrets unless they have access the same key.

For distributed builds, files on the NIX_PATH are not copied to distributed build slaves, so if the <nixos-store-key> path is missing on the build slave machine(s) then the build will just fail.


# Prototype

* Nix implementation: https://github.com/edolstra/nix/commit/6b70036
* Nixpkgs example: https://github.com/edolstra/nixpkgs/commit/4c8212069429bf9fb959e00ce8d9345ac7cb7ff0

# Drawbacks
[drawbacks]: #drawbacks

* The main downside of this approach is that encrypted files need to
be decrypted before they can be used. Thus, for example, a daemon
configuration file cannot be used directly from the Nix store by
passing something like `--config /nix/store/.../foo.conf` to the
daemon. Instead, we have to decrypt the configuration file to some
suitably secure temporary location in a pre-start script, and then
pass that location (e.g. `--config /tmp/.../foo.conf`).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The biggest drawback here is if another file depends on that file.

{
nginxConf = writeFile "nginx.conf" "include ${vserverConf};";
vserverConfg = writeFile "vserver.conf" " my secret is here ";
}

Would it be possible to taint the derivation output with an "encrypted" attribute and then have a mechanism that would prevent referencing it? (unless if it's to decrypt it) The tainting could also be used as a signal to not push it to public stores.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. We often have chains of config files where changing a leaf bubbles up correctly all the way to the top thanks to nix. I'm not sure how many of our files contain secrets though. So this might actually be an OK price to pay in practice.


In NixOS, we can provide some helper functions to reduce boilerplate
code dealing with encrypted files in systemd services. For example,
we can add an option to allow modules to declare files that need
decryption:

```
security.encryptedFiles = [ wpaSupplicantConf ];
```

which would be decrypted by the activation script to a well-known
location such as `/var/secrets/<storepath>`. This way, services can
refer to the decrypted path easily:

```
systemd.services.wpa_supplicant.serviceConfig.Exec =
"wpa_supplicant -c /var/secrets/${wpaSupplicantConf}";
# i.e. ... -c /var/secrets/nix/store/.../wpa_supplicant.conf
```

The above can also be extended to support access by non-root users,
e.g.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be a critically important component, so individual services don't need (poorly) re-implement this behavior.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, and ideally this would need some way to get a path for each configuration file. For security reasons systemd services can switch users, and we would want these files to be read only by these users.

I would prefer to avoid FUSE as the first iteration, as this would imply trusting more code than necessary. (we are talking about the manipulation of secrets)

I think we could have something like:

{ security.encryptedFiles.file = {
    user = "foo"; /* uses users.users.<name>.uid or assert if missing */
    file = ...;
  };
  systemd.services.wpa_supplicant.serviceConfig.Exec =
     "wpa_supplicant -c ${security.encryptedFiles.file}";
}

The problem of this scheme is that the author of the module would have to think ahead that the file has to be secure.

For example, simple configuration can provide a default configuration based on pam, but leave complex option which could be provided via extraConfig to set password in the configuration files. In such case we might want as a user to retrofit security in an expression which did not expected it first.

To handle such cases, I think we should make a submodule which is global to NixOS to let people register secure files anywhere, by adding an option type which would be a submodule for declaring files. This submodule would then be in charge of finding the proper renaming of the files path, and these entries could be listed in the security.encryptedFiles option to let the activation script decipher these files as root, before changing the ownership. Thus a service would look like:

{ lib, config, ... }:
{ options.services.foo = {
    enable = ...;
    extraConfig = ...;
    configFile = lib.mkOption { type = lib.types.file; };
  };
  config = with config.services.foo; lib.mkIf enable {
    security.secureFiles = [ configFile ]; # Can be added by another module
    systemd.services.foo.serviceConfig.Exec =
       "command --conf ${configFile}";
    services.foo.configFile.content = extraConfig;
  };
}

Doing it that way, ensure that the security.maybeSecureFiles can be registered by anybody who needs to set the services.foo.configFile.secure = true;. Thus, the submodule lib.types.file will contain the logic to translate any file from a store path to a /var or /tmp path if the file has the secure flag. The decoding would be handled by the security.secureFiles option, which would decipher the files as root, and change the ownership as part of the activation script of NixOS.

Thus, instead of forcing something special for secure files only, we would force something special for files in general, which would make it less special, and easier to extend in a modular fashion.


```
security.encryptedFiles = [
{ file = httpdConf;
owner = "httpd";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe group (string, default null) and groupReadable (boolean, default false) as options too.

}
];
```

An alternative is to write a little FUSE filesystem that
transparently decrypts files on the underlying filesystem. For
example, the filesystem mounted on /var/secrets (accessible only to
root) would automatically decrypt files from /nix/store. But that's
probably over-engineering it.

* This approach only allows secrets to be stored at evaluation time
and used at runtime. Secrets cannot be created or used at build time
(except by copying encrypted secrets verbatim), since no keys are
available at build time.

* The fact that encrypted secrets are world-readable allows attackers
to mount (offline) brute-force attacks. Depending of how we handle
nonces (see below), it may also be possible for attackers to observe
that secret values have changed, that identical secrets are used in
multiple places, or that secrets have changed back to a previous
value.

# Alternatives
[alternatives]: #alternatives

## Access control

The main alternative is to add some notion of access control
(i.e. ownership and permissions) to Nix store paths. (See
e.g. https://github.com/NixOS/nix/pull/329.) This has the advantage
that secret files can be used "directly" by a sufficiently privileged
user. However, adding access control raises many issues:

* Nix has always assumed a world-readable store, so we'd have to
carefully audit that nothing in the daemon can leak the contents of
store paths.

* Users should be allowed to export closures that they built, even
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps instead of checking the ACL's of the store paths, use the evaluator to determine whether the user can copy the closure?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which would require tracking the original expressions or .drv's which we do not do.

when those closures include paths not readable by them. This is
essential for tools such as NixOps, which perform a `nix-build`
followed by `nix-copy-closure`, generally running under a
unprivileged user account. However, this requires Nix to keep track
of which users instantiated or built which paths, which is an
unfortunate amount of statefulness. It also prevents an evaluation
optimization: currently, if we're adding a derivation X that depends
on derivation Y to the store, we don't have to check Y if we see
that X is already valid.

* If we want to allow the creation of files with arbitrary uids/gids,
how does a builder create them? The builder runs as a non-privileged
`nixbld` user, so it does not have the ability to chown files.

Also, the desired users and groups may not exist at build time. (On
NixOS, they are created at activation time, so *after* the system
has been built.)

Finally, allowing builders to create files with arbitrary ownership
(in particular when combined with permissions) creates obvious
security issues.

* The NAR format needs to be extended to store ownership and
permission data. A NAR can no longer necessarily be unpacked in a
lossless way if the required uids/gids don't exist.

* How should binary caches deal with ownership and permissions?

Encryption support, on the other hand, is a very localised change,
requiring only the addition of an encryptString primop and a
decryption tool.

## NixOps keys mechanism

NixOps has a somewhat ad hoc mechanism for dealing with
secrets. Secrets can be defined in the NixOps machine specification
via the option `deployment.keys`:

```
deployment.keys.my-password.text = "fnord";
```

These secrets do not end up in the system closure; rather, they are
copied by the command `nixops send-keys` to `/run/keys/<key-id>`.

This mechanism could be generalised beyond NixOps into a Nix feature.

# Unresolved questions
[unresolved]: #unresolved-questions

## Reproducibility and nonces

An important goal of NixOS is reproducibility. Nowadays this goal also
includes bitwise-identical reproducibility (see
https://reproducible-builds.org/); that is, building a NixOS system
twice from the same Nix expressions should yield an identical result.

However, libsodium's authenticated encryption requires supplying a
nonce which (by definition) should be used only once. From
https://download.libsodium.org/doc/secret-key_cryptography/authenticated_encryption.html:
"The nonce doesn't have to be confidential, but it should never ever
be reused with the same key."

Unfortunately, the uniqueness of nonces is at odds with the
requirement of bitwise reproducibility: if multiple calls to
`encryptString` with the same inputs return different results, then
logically identical NixOS system closures will not be bitwise
identical.

We could replace the nonce with a deterministic value, but it's not
entirely clear what the cryptographic implications are. At the very
least, it allows attackers to obverse that a secret has changed, or
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: s/obverse/observe/g

that it has changed back to a previously observed value.

## Decryption failure

What should `nix decrypt` do if it cannot decrypt an encrypted string?
Keeping the encrypted string or replacing it by an empty string is
dangerous because it can cause e.g. a predictable password to be
emitted. Probably the best solution is to fail by default (i.e. not
emit a file at all), or optionally replace the encrypted string by a
cryptographically secure random string.

## Public-key cryptography

We could have `encryptString` use a public key, and `nix decrypt` use
the corresponding private key. This is kind of nice because Nix
already has a command to generate public/private key pairs using
libsodium for binary cache signing.

However, I couldn't think of a plausible use case for this. It would
allow the user who generates/builds the configuration not to have the
decryption key, but that doesn't seem very useful.

## Key generation

Currently, we assume that `<nixos-store-key>` exists beforehand. It
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would this work in a multi-user environment? Could each user feasibly have their own key? And how would this integrate with something like NixOps, where one might want to have a separate key for NixOps deployments, and possibly share that key with others managing the deployment, without exposing their local key?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This RFC only adds builtins.encryptString and a nix-store --decrypt helper, so as far as I understand this issue is left to future work.

I believe it's best to first get this in as soon as we can all agree on what's wanted, and then try thinking more about how to generate keys and how to use them, once the primitives will be in place (and when we will have moved a bit further in the process of getting nix encryption in production, making steps one by one so we don't just stumble on agreeing on a RFC for three months like what is currently happening). Don't you think? :)

might be nice if Nix can generate keys on the fly. For example,
`<nixos-store-key>` could resolve to `$HOME/.config/nix/key` and be
generated on first use.