Skip to content

Omicron integration - #31

Draft
plotnick wants to merge 38 commits into
mainfrom
omicron-integration
Draft

Omicron integration#31
plotnick wants to merge 38 commits into
mainfrom
omicron-integration

Conversation

@plotnick

@plotnick plotnick commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

This is the sush half of the Omicron integration; see RFD 620 §§ 4.5.1-2. It pairs with oxidecomputer/omicron#11034, which embeds the server in sled-agent and adds a proxy-only service to the switch zones. The early commits make the server embeddable; the rest allow running jobs on the rack from outside it.

  • Make the server embeddable.
  • Gossip jobs and sessions between sleds over sprockets.
  • Add the job target grammar.
  • Rewrite the proxy to route by target, and resolve targets through the proxy.
  • Gossip identities as evidence rather than assertions.
  • Serve the proxy over TLS with an ephemeral key whose certificate the RoT signs once at switch zone startup.
  • Follow the rack's cubby map from MGS.
  • Snapshot wire formats as binary files, use nextest to run tests.
  • Bump dropshot to 0.17 with the matching progenitor and reqwest.

🤖 Disclaimer: written with LLM assistance.

@plotnick
plotnick force-pushed the omicron-integration branch 3 times, most recently from a97ba54 to f345c26 Compare August 10, 2026 05:25
Base automatically changed from public-readiness to main August 10, 2026 05:55
plotnick and others added 24 commits August 10, 2026 21:28
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The proxy terminates TLS with the sled's platform identity, so the
client verifies against baked-in platform PKI roots: no TOFU, no
per-rack cert distribution, and identity survives reboot. The switch
zone can't sign every handshake with the RoT, and delegating the IPCC
device into the zone may not be acceptable, so sled-agent mints an
ephemeral key and has the RoT sign its certificate once at startup;
the proxy then terminates TLS with that key locally. The client's
verifier accepts a chain that is the platform identity itself, or an
ephemeral leaf the platform identity signed in the RoT's convention:
Ed25519 over the SHA3-256 digest of the TBS certificate.
Job requests may name their targets by cubby, and each server checks
signed targets against its cubby map, which until now was always
empty. The job manager now takes a watch channel carrying the map,
which sled-agent will feed from MGS. The map is a rack fact rather
than universe state, so it survives gossip migrations.
The client ships the same idcerts the sled OS does, and verifies a
proxy's TLS certificate against them by default. The --proxy-root
flag replaces the baked-in roots, e.g., for a test PKI.
illumos has no O_CLOEXEC for posix_openpt, so rustix offers no
OpenptFlags::CLOEXEC there. Set the flag with fcntl right after
open on every platform.
sprockets-tls now gates IPCC behind a default-on feature. Nothing in
this workspace uses it, so opt out: the client binary no longer needs
libipcc.so.1, which only exists on Oxide sleds. The embedded server
gets the feature back through sled-agent's own sprockets dependency.
Advancing sprockets pulls a newer dice-util, whose requirements
cascade far enough that the lock is regenerated wholesale.
Feature unification turns it on for every schemars user in a
workspace that embeds sush, which reorders the properties of every
generated schema and breaks omicron's schema snapshot tests. Nothing
here depends on property order.
The same race attribution had: job B could start before the cancel
landed on a loaded machine, and a dependency refresh made the window
easy to hit locally. Queue B behind a chain hole.
@plotnick plotnick mentioned this pull request Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants