Start integrating SPDM into sled-agent #407
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This code only represents the negotiation phase of the protocol. A follow up commit will contain the responder identification (digests + cert retrieval) code, as well as challenge response, which is the first part of the protocol used to begin securing a SPDM channel. These are the parts of the SPDM implementation that exist so far. More will be added over time. It's expected that `run` for each of a requester and responder will be used by a caller establishing a secure channel. That usage will come in a follow up commit. It's likely that when the SPDM protocol is complete a secure channel will be returned for writing to. Until then, callers will pretend that run sets up a fully secure channel, and then use the existing TCP stream when run returns.
andrewjstone
commented
Nov 19, 2021
smklein
reviewed
Nov 23, 2021
Collaborator
smklein
left a comment
smklein
left a comment
There was a problem hiding this comment.
This looks like a good start - maybe we can talk about where this would integrate in the bootstrap code today
Co-authored-by: Sean Klein <sean@oxide.computer>
smklein
approved these changes
Nov 29, 2021
* Wrap requester and responder data in a `Ctx` * Bump `spdm` crate so that slices instead of sizes are returned * Wrap `Framed<TcpStream, LengthDelimitedCodec>` to allow sending slices, and converting internally to `Bytes` so the callers. don't have to do this.
andrewjstone
added a commit
that referenced
this pull request
Dec 6, 2021
In order to allow for encrypted storage on individual sleds without the need for a user to type a password at bootup, we utilize secret sharing across sleds, where a threshold number of sleds need to communicate in order to generate a `rack secret`. This rack secret can then be used to derive local encryption keys from individual sleds. We therefore provide the ability to prevent an attacker from stealing a subset of sleds or storage devices and obtaining any data. In fact, the control plane software does not even boot until the rack secret is reconstructed and the protected storage unlocked. There are quite a few moving parts required in order to implement a trust quorum, some of which involve the service processor and hardware root of trust. This commit only implements the part of the trust quorum responsible for retrieiving existing key shares over an unfinished SPDM channel. It runs entirely on the host machine as part of the sled-agent. The code builds upon the multicast discovery code in #404, the SPDM negotiation code in #407 and the secret sharing code in #429. In the "normal" lifetime of an Oxide rack, a rack secret will be generated upon initialization of the new rack by the customer. The shares will then be destributed over SPDM channels to individual sleds such that they can be retrieved and combined at a later time when an individual sled or the entire rack reboots. The initial generation and distribution of shares is *not* part of this commit. We fake rack initialization through the completely insecure use of a configuration file provided as part of the `omicron-package` install that contains all key shares. The configuration file disables the trust quorum by default, so that the sled-agent continues to run on a single node. When enabled, share retrieval attempts will begin and when a quorum of shares are received, the rack secret will be reconstructed, and the rest of the control plane will begin to boot. In order for this to work, the user also has to edit the config file to ensure that a different `sled_index` (which points to a given unique share) exists in each config file, and then the sled-agent must be restarted with `svcadm restart sled-agent`. The included config file only includes shares for 2 sleds, but a new one can be generated with the provided `gen_trust_quorum_config` program. Lastly, the location of the config file is given in the sled-agent smf file and passed through as `rack_secret_dir` in the `BootstrapConfig` struct. The SPDM protocol is run over a 2-byte size header framed transport operating over a TCP stream. We generate a client and server to initialize this transport, perform SPDM negotiation, and then begin share retrieval. As noted in #407, only the negotiation phase of the SPDM protocol is currently implemented, and so we simply return the TCP based transport when negotiation completes, and pretend for now that we are operating over a secure channel. This allows us to test out the end-to-end behavior before we have a production ready SPDM implementation integrated. This commit also makes a small change to the SPDM transport to provide for timeouts on `send` and `recv` operations, and no longer requires passing a logger to each call of `recv`.
andrewjstone
added a commit
that referenced
this pull request
Dec 6, 2021
In order to allow for encrypted storage on individual sleds without the need for a user to type a password at bootup, we utilize secret sharing across sleds, where a threshold number of sleds need to communicate in order to generate a `rack secret`. This rack secret can then be used to derive local encryption keys from individual sleds. We therefore provide the ability to prevent an attacker from stealing a subset of sleds or storage devices and obtaining any data. In fact, the control plane software does not even boot until the rack secret is reconstructed and the protected storage unlocked. There are quite a few moving parts required in order to implement a trust quorum, some of which involve the service processor and hardware root of trust. This commit only implements the part of the trust quorum responsible for retrieiving existing key shares over an unfinished SPDM channel. It runs entirely on the host machine as part of the sled-agent. The code builds upon the multicast discovery code in #404, the SPDM negotiation code in #407 and the secret sharing code in #429. In the "normal" lifetime of an Oxide rack, a rack secret will be generated upon initialization of the new rack by the customer. The shares will then be destributed over SPDM channels to individual sleds such that they can be retrieved and combined at a later time when an individual sled or the entire rack reboots. The initial generation and distribution of shares is *not* part of this commit. We fake rack initialization through the completely insecure use of a configuration file provided as part of the `omicron-package` install that contains all key shares. The configuration file disables the trust quorum by default, so that the sled-agent continues to run on a single node. When enabled, share retrieval attempts will begin and when a quorum of shares are received, the rack secret will be reconstructed, and the rest of the control plane will begin to boot. In order for this to work, the user also has to edit the config file to ensure that a different `sled_index` (which points to a given unique share) exists in each config file, and then the sled-agent must be restarted with `svcadm restart sled-agent`. The included config file only includes shares for 2 sleds, but a new one can be generated with the provided `gen_trust_quorum_config` program. Lastly, the location of the config file is given in the sled-agent smf file and passed through as `rack_secret_dir` in the `BootstrapConfig` struct. The SPDM protocol is run over a 2-byte size header framed transport operating over a TCP stream. We generate a client and server to initialize this transport, perform SPDM negotiation, and then begin share retrieval. As noted in #407, only the negotiation phase of the SPDM protocol is currently implemented, and so we simply return the TCP based transport when negotiation completes, and pretend for now that we are operating over a secure channel. This allows us to test out the end-to-end behavior before we have a production ready SPDM implementation integrated. This commit also makes a small change to the SPDM transport to provide for timeouts on `send` and `recv` operations, and no longer requires passing a logger to each call of `recv`.
andrewjstone
added a commit
that referenced
this pull request
Dec 18, 2021
In order to allow for encrypted storage on individual sleds without the need for a user to type a password at boot, we utilize secret sharing across sleds, where a threshold number of sleds need to communicate in order to generate a `rack secret`. This rack secret can then be used to derive local encryption keys for individual sleds. We therefore provide the ability to prevent an attacker from stealing a subset of sleds or storage devices and obtaining any data. In fact, the control plane software does not even boot until the rack secret is reconstructed and the protected storage unlocked. There are quite a few moving parts required in order to implement a trust quorum, some of which involve the service processor and hardware root of trust. This commit only implements the part of the trust quorum responsible for retrieving existing key shares over an unfinished SPDM channel. It runs entirely on the host machine as part of the sled-agent. The code builds upon the multicast discovery code in #404, the SPDM negotiation code in #407 and the secret sharing code in #429. In the "normal" lifetime of an Oxide rack, a rack secret will be generated upon initialization of the new rack by the customer. The shares will then be distributed over SPDM channels to individual sleds such that they can be retrieved and combined at a later time when an individual sled or the entire rack reboots. The initial generation and distribution of shares is *not* part of this commit. Instead shares are individually distributed along with metadata as a `ShareDistribution` stored in a `share.json` file in the `sled_agent/pkg` directory under the install directory configured for `omicron-package install`. Share generation must be done manually now, but a follow up commit is coming for a deployment system that will generate the rack secret and distribute the shares along with the install of omicron. If the `share.json` file is not present, the server operates in single-node mode, and does not try to form a a trust quorum. This is behavior required for current development backwards compatibility and will eventually be removed. The SPDM protocol is run over a 2-byte size header framed transport operating over a TCP stream. We generate a client and server to initialize this transport, perform SPDM negotiation, and then begin share retrieval. As noted in #407, only the negotiation phase of the SPDM protocol is currently implemented, and so we simply return the TCP based transport when negotiation completes, and pretend for now that we are operating over a secure channel. This allows us to test out the end-to-end behavior before we have a production ready SPDM implementation integrated. This commit also makes a small change to the SPDM transport to provide for timeouts on `send` and `recv` operations, and no longer requires passing a logger to each call of `recv`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This code only represents the negotiation phase of the protocol.
A follow up commit will contain the responder identification (digests + cert
retrieval) code, as well as challenge response, which is the first part of the
protocol used to begin securing a SPDM channel. These are the parts of the SPDM
implementation that exist so far. More will be added over time.
It's expected that
runfor each of a requester and responder will be used by acaller establishing a secure channel. That usage will come in a follow up
commit. It's likely that when the SPDM protocol is complete a secure channel
will be returned for writing to. Until then, callers will pretend that run sets
up a fully secure channel, and then use the existing TCP stream when run
returns.