TCP support in the Nix daemon #5265
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Great!! I was hoping we'd do these things. |
Ericson2314
reviewed
Sep 20, 2021
edolstra
changed the title
Like UDSRemoteStore, but over a TCP connection. Example usage: $ nix path-info --store tcp://example.org:1234 ...
In conjunction with systemd socket activation, this allows the daemon to listen to multiple sockets, e.g. [Socket] ListenStream=/nix/var/nix/daemon-socket/socket ListenStream=1234
Co-authored-by: John Ericson <git@JohnEricson.me>
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/tweag-nix-dev-update-18/15300/1 |
|
I wonder if we really need that in Nix itself. if it is purely about testing locally wouldn't socat on the current unix socket work just fine? |
|
But for building something like nixbuild.net, this can be extremely helpful. |
|
To me this is good enough to merge I don't care about the daemon being able to open sockets of any sort because I basically believe one should always use socket activation. I assume @edolstra is mainly waiting to get 2.4 our before returning to new features. That is also fine with me. |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/distributing-the-nix-store-with-cvmfs-nix/15706/2 |
Can you elaborate on this? What could an adversary with access to the TCP socket do, except for DoS (by consuming compute resources, bandwidth and disk space)? Is there a way for an adversary to compromise other users that use the same Nix daemon over TCP? My potential use case: Sharing a long-living Nix daemon across short-living (and mutually distrusting) CI jobs. |
|
I marked this as stale due to inactivity. → More info |
7 tasks
Ericson2314
added a commit
to obsidiansystems/nix
that referenced
this pull request
Feb 2, 2023
These settings are not needed for libstore at all, they are just used by the nix daemon *command* for authorization on unix domain sockets. My moving them to a new configuration struct just in that file, we avoid them leaking anywhere else. Also, it is good to break up the mammoth `Settings` struct in general. Issue NixOS#5638 tracks this. The message is not changed because I do not want to regress in convenience to the user. Just saying "this connection is not trusted" doesn't tell them out to fix the issue. The ideal thing to do would be to somehow parameterize `processCommand` on how the error should be displayed, so different sorts of connections can display different information to the user based on how authentication is performed for the connection in question. This, however, is a good bit more work, so it is left for the future. This came up with me thinking about the tcp:// store (NixOS#5265). The larger project is not TCP *per se*, but the idea that it should be possible for something else to manage access control to services like the Nix Daemon, and those services simply trust or trust the incoming connection as they are told. This is a more capability-oriented way of thinking about trust than "every server implements its own auth separately" as we are used to today. Its very great that libstore itself already implements just this model, and so via this refactor I basically want to "enshrine" that so it continues to be the case.
7 tasks
Ericson2314
added a commit
to obsidiansystems/nix
that referenced
this pull request
Feb 2, 2023
These settings are not needed for libstore at all, they are just used by the nix daemon *command* for authorization on unix domain sockets. My moving them to a new configuration struct just in that file, we avoid them leaking anywhere else. Also, it is good to break up the mammoth `Settings` struct in general. Issue NixOS#5638 tracks this. The message is not changed because I do not want to regress in convenience to the user. Just saying "this connection is not trusted" doesn't tell them out to fix the issue. The ideal thing to do would be to somehow parameterize `processCommand` on how the error should be displayed, so different sorts of connections can display different information to the user based on how authentication is performed for the connection in question. This, however, is a good bit more work, so it is left for the future. This came up with me thinking about the tcp:// store (NixOS#5265). The larger project is not TCP *per se*, but the idea that it should be possible for something else to manage access control to services like the Nix Daemon, and those services simply trust or trust the incoming connection as they are told. This is a more capability-oriented way of thinking about trust than "every server implements its own auth separately" as we are used to today. Its very great that libstore itself already implements just this model, and so via this refactor I basically want to "enshrine" that so it continues to be the case.
|
I have fixed the comments in this, but before I go too far proposing it I figured it would be good to start with something like #7739 |
|
#6312 merges master into this. |
Ericson2314
added a commit
to obsidiansystems/nix
that referenced
this pull request
Feb 23, 2023
We now support: 1. Multiple sockets, per the systemd socket activation spec 2. The sockets not having pid/uid/gid peer info because they might not be Unix domain sockets. The changes are by @edolstra, taken from NixOS#5265. This is just that PR *without* the TCP parts, which I gathered are the controversial parts. Hopefully this remainder is not so controversial.
7 tasks
Ericson2314
added a commit
to obsidiansystems/nix
that referenced
this pull request
Mar 16, 2023
We now support: 1. Multiple sockets, per the systemd socket activation spec 2. The sockets not having pid/uid/gid peer info because they might not be Unix domain sockets. The changes are by @edolstra, taken from NixOS#5265. This is just that PR *without* the TCP parts, which I gathered are the controversial parts. Hopefully this remainder is not so controversial.
Ericson2314
added a commit
to obsidiansystems/nix
that referenced
this pull request
Apr 7, 2023
We now support: 1. Multiple sockets, per the systemd socket activation spec 2. The sockets not having pid/uid/gid peer info because they might not be Unix domain sockets. 3. `--stdio` on the new `nix daemon`, not just the old `nix-daemon`. In addition, `PeerInfo` is made more type safe with `std::optional`. These changes are by @edolstra, taken from NixOS#5265. This is just that PR *without* the TCP parts, which I gathered are the controversial parts. Hopefully this remainder is not so controversial.
Ericson2314
added a commit
to obsidiansystems/nix
that referenced
this pull request
Apr 8, 2023
We now support: 1. Multiple sockets, per the systemd socket activation spec 2. The sockets not having pid/uid/gid peer info because they might not be Unix domain sockets. 3. `--stdio` on the new `nix daemon`, not just the old `nix-daemon`. In addition, `PeerInfo` is made more type safe with `std::optional`. These changes are by @edolstra, taken from NixOS#5265. This is just that PR *without* the TCP parts, which I gathered are the controversial parts. Hopefully this remainder is not so controversial.
Ericson2314
added a commit
to obsidiansystems/nix
that referenced
this pull request
May 10, 2023
We now support: 1. Multiple sockets, per the systemd socket activation spec 2. The sockets not having pid/uid/gid peer info because they might not be Unix domain sockets. 3. `--stdio` on the new `nix daemon`, not just the old `nix-daemon`. In addition, `PeerInfo` is made more type safe with `std::optional`. These changes are by @edolstra, taken from NixOS#5265. This is just that PR *without* the TCP parts, which I gathered are the controversial parts. Hopefully this remainder is not so controversial.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR makes it possible to connect to a remote Nix daemon via TCP, and for the Nix daemon to listen on multiple sockets (including TCP) via systemd.
Lacking encryption/authentication, this is primarily useful to testing, e.g. when locally simulating high-latency SSH store connections via qdisc/netem.
Example usage: