Skip to content

runsc: serialize RDMA and PCI sysfs data at boot#13114

Draft
atoniolo76 wants to merge 1 commit intogoogle:masterfrom
modal-labs:alessio/serialize-rdma-sysfs-boot
Draft

runsc: serialize RDMA and PCI sysfs data at boot#13114
atoniolo76 wants to merge 1 commit intogoogle:masterfrom
modal-labs:alessio/serialize-rdma-sysfs-boot

Conversation

@atoniolo76
Copy link
Copy Markdown

Snapshot host RDMA/InfiniBand and PCI device sysfs attributes before pivot_root (while the host sysfs is still accessible), serialize them as JSON, and reconstruct them as virtual kernfs entries inside the sentry. This gives NCCL and libibverbs the topology information they need for device discovery and PCI distance computation without granting the sandbox access to the real host sysfs.

Key components:

  • pkg/sentry/fsimpl/sys/rdma.go: RDMA device data collection and virtual /sys/class/infiniband{,_verbs}/ + /sys/class/net/ entries
  • pkg/sentry/fsimpl/sys/pci_devices.go: PCI topology collection and virtual /sys/devices/pci*/ hierarchy with symlinks for NCCL
  • runsc/cmd/chroot.go: collection before pivot_root
  • runsc/cmd/boot.go: --rdmaproxy flag and data deserialization
  • runsc/boot/vfs.go: wire sysfs InternalData for RDMA/PCI
  • runsc/config: --rdmaproxy configuration flag

@google-cla
Copy link
Copy Markdown

google-cla Bot commented May 7, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@atoniolo76 atoniolo76 force-pushed the alessio/serialize-rdma-sysfs-boot branch 3 times, most recently from 2bae3ba to 7f9b289 Compare May 8, 2026 18:23
@atoniolo76 atoniolo76 force-pushed the alessio/serialize-rdma-sysfs-boot branch 2 times, most recently from dad6dcd to 44bb161 Compare May 8, 2026 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant