Provide secrets via file descriptor

[Alan-R]

Probably the most secure way for one application to give information is through a file which was created, opened, then removed. Here's a suggestion about how it might be implemented on UNIX-like systems:

• Create a tmp file readable only by owner (mode 0600), and opened for read-write
• unlink (remove) the temporary file
• Write the name=value contents into it
• seek to the beginning
• pass the file descriptor number to the client program

The client program then reads the file, gets the secrets it wants, then closes the file. No other process on the system can at any time see the contents unless they open it before its been removed. Of course, someone with access to the block devices under the OS could read it. But then you're screwed anyway ;-).

[dustinmm80]

Thanks for opening this issue Alan! It's been on my mind, for sure.

I'm looking at the container landscape now and it seems to be coalescing on mounting (something like) a tmpfs volume with the secrets in it into containers. This could work really well with summon actually, because we already have a way to clear up folders/files (see !tmp:file in secrets.yml).

That said, I'll get to this as I get time, but PR welcome :)

[Alan-R]

Didn't mean to press the close button. I didn't view this as high-priority, but for those cases where something a little more sophisticated was consuming the secrets, it seems like a reasonable thought. FWIW - I heard your talk at DevOpsDaysRox when you talked about summon there.

[dustinmm80]

those cases where something a little more sophisticated was consuming the secrets

Can you explain this a bit more? Would be really valuable to know what you're talking about here.

[Alan-R]

I meant something more sophisticated than a Chef recipe for example. What I had in mind was a Python program, or C program, or Java program, or something written by a software person rather than a system administrator who was in a hurry and not very comfortable writing code.

Your current method (environment variables) is really easy to use. That's a good thing. But it's also easy to see using /proc, or a child process, etc. For Linux, you have to be the same uid to see it, but not so for all *NIX systems. You could also change the values of all the credentials you inherited to keep it from leaking to child processes - but that's a good bit of work. You could also unexport them (depending on the language).

The file descriptor could still be leaked, of course, but if the consumer closes the file when they've gotten what they want out of it, then it would be harder to leak accidentally. Closing the file descriptor is really easy to so, and very effective at making it hard for anyone else to accidentally acquire.

[dustinmm80]

Thank you, I'll consider this over the weekend and will have a response early next week. Really appreciate you spelling the use case(s) out, helps this project a ton!

[dividedmind]

That is an excellent idea! At least on Linux, you can even pass it to programs that don't understand it, by using pseudo-symlinks in /dev/fd (or /proc/self/fd, which that is a symlink to) directory. Digging further, there is a Linux-specific flag to open(2), O_TMPFILE, which skips the part where the file is linked to a directory, and also memfd_create(2) syscall, which skips touching the filesystem altogether.

What I'm not sure about any of these solutions is if it really buys any extra security (on Linux) -- any process with the correct permissions can access those fds and their contents over /proc anyway! But they're definitely more elegant than using tmpfs (and the basic create-unlink is also more portable).

[Alan-R]

There is a window when anyone running as the same user can access it, but once it's closed - it's gone. This is long before the process exits - so the window would normally be much shorter. It's not an urgent need, but what you can to make it a bit less likely to be leaked accidentally or otherwise is good.

Paranoia is a good thing when thinking about handling secrets.

[dividedmind]

Paranoia is a good thing when thinking about handling secrets.

Well said! Fully agreed.

I just want to point out, that the window can be equally short in both cases: either the child unlinks (and closes) the tempfile, or it uses the fd directly and closes it. Closing the access window requires its cooperation either way; unless I'm missing something, without cooperation it stays open. Ideally, on Linux, keyrings(7) would be used to propagate secrets; this doesn't provide any window at all. But this would also require childs' cooperation.

The strength of the present approach is that it's transparent to the child, so it doesn't have to know about any of this stuff as long as it knows how to read environment and files. That said, providing a way to strengthen security given the child process cooperation is certainly a lofty goal! However, after further thought, given the above, perhaps it doesn't make that much sense to implement it this way -- I can't think of any examples of executables which take data in fds (except 0/1/2 ofc) out-of-the-box. And if the executable needs to be modified anyway, it's better to use keyrings and provide an actual security boost.

(This ignores the question of portability. For non-Linux systems, passing via fds can be more secure -- given that the system doesn't provide a way to access foreign fds like Linux does with /proc/*/fd. I'm not familiar enough with other POSIX systems to say how common this is.)

[kgilpin]

I think that expanding the use of the keyring would be great. Not many people know about it and it provides a nice security profile. Keys are not even readable by root when they are in the keyring.

[ryanprior]

Based on the conversation and my knowledge of how software is configured, this a likely "won't fix" for Summon. I am strongly interested in counter-arguments if there are good ones yet to be made. But here's how I see the issue:

1. configuration using the environment is a great and popular pattern. It's recommended in the 12 Factor App guidelines and the 10 Factor CI guidelines and has support throughout the entire server stack.
2. configuration using a file descriptor is niche. adding support for this to Summon is not going to immediately benefit many operators
3. people who do use fds to configure their software can compose Summon with a simple utility that takes certain environment variables, makes them available via a file descriptor, and unsets them before executing a subprocess.
4. people who are looking for the most secure way to pass secrets between processes on Linux should use kernel keyrings anyway which are precisely the right tool for the job. in light of this, to the extent we can popularize through Summon a novel technology for securely configuring software, we should promote Linux keyrings and not file descriptors.

Again, please make comments or open a new issue if I'm off base or lacking information. Thanks again to everybody in this thread!

[Alan-R]

Sounds good. I wound up implementing my own API. The current solution works great for short duration processes, but is not appropriate for longer running applications because it prevents key rotation and if an adversary pops a shell from a long running process (web app for example), they have the secret in their shell. -- Alan Robertson alanr@unix.sh

On Wed, Sep 20, 2017, at 04:25 PM, Ryan Prior wrote: Based on the conversation and my knowledge of how software is configured, this a likely "won't fix" for Summon. I am strongly interested in counter-arguments if there are good ones yet to be made. But here's how I see the issue: 1. configuration using the environment is a great and popular pattern. It's recommended in the 12 Factor App guidelines and the 10 Factor CI guidelines and has support throughout the entire server stack. 2. configuration using a file descriptor is niche. adding support for this to Summon is not going to immediately benefit many operators 3. people who do use fds to configure their software can compose Summon with a simple utility that takes certain environment variables, makes them available via a file descriptor, and unsets them before executing a subprocess. 4. people who are looking for the most secure way to pass secrets between processes on Linux should use kernel keyrings anyway which are precisely the right tool for the job. in light of this, to the extent we can popularize through Summon a novel technology for securely configuring software, we should promote Linux keyrings and not file descriptors.> Again, please make comments or open a new issue if I'm off base or lacking information. Thanks again to everybody in this thread!> — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub[1], or mute the thread[2].>

Links: 1. #42 (comment) 2. https://github.com/notifications/unsubscribe-auth/ABM0o_wF6mLMmYqpmnVtZukWd8hG6CePks5skZDlgaJpZM4N8s0n