What's happening in Shadow 2021-04

[sporksmith]

This is part of a series of periodic updates of development in Shadow. This work is sponsored by the NSF. Previous update: 2020-12

We're approaching release of a new version of Shadow with our new process-based architecture! With the new architecture, Shadow is now capable of running Tor simulations using unmodified tor binaries, faster and using less RAM than the latest production release! Although our work on the new architecture is available in our dev branch, we don't recommend using it quite yet; in particular, we plan to rework some of the command-line and configuration options before an official release in order to make Shadow more pleasant to use. We are targeting an official release within the next 1-2 months.

In the meantime, here are some of the major changes we have made in the last few months:

Functionality:

• We now support enough thread primitives to run Tor simulations, and plan to support more (Implement futex operations that currently generate warnings in tor sims #1262, Implement get_robust_list and set_robust_list syscalls #1260) before release.
• To avoid potential confusion or data loss, Shadow will no longer automatically delete old data directories, nor use a template directory that wasn't explicitly specified (Remove the default value for '--data-template' and stop auto-deleting the data directory #1210).
• We've implemented more syscalls in the new architecture (Add support for socketpair syscall #1058, Add support for the dup() syscall #1071, Miscellaneous syscalls to support running Tor #1075, Fix descriptor bugs and support dup() on regular files for Python #1257). We've now implemented the full set of those we need to run Tor simulations!
• We've implemented syscalls used by Nginx (Small ioctl and sendto bug fixes, and implement FIONBIO for Nginx #1179) and Python (Fix descriptor bugs and support dup() on regular files for Python #1257). In Shadow's new architecture, we're now able to run these without having to modify or recompile them.

Performance:

• We've implemented CPU-pinning (Pinning main #1079, Adds CPU pinning to the development branch. #1084), which significantly improves performance by reducing cache-trashing CPU migrations.
• We've moved some of Shadow's state into shared memory, allowing us to implement some hot-path syscalls in the LD_PRELOADd shim, saving a context switch to Shadow and back (Handle hot-path time syscalls in the shim #1224).
• We discovered that waitpid in Linux is unexpectedly O(n) instead of O(1) in Linux, leading to quadratic overhead in Shadow simulations (waitpid is O(# children), resulting in O((# sim processes)^2) in thread_ptrace #1134). We've submitted a patch to the Linux maintainers to make waitpid O(1).
• We've implemented a new scheduler (Decouple # of worker threads from max concurrency #1238) that works around the O(n) waitpid in current/legacy kernels. It also implements a new way of balancing load across CPU cores that has less overhead than the previous work-stealing scheduler, and doesn't require ptrace detaching and reattaching.
• We've removed a "book keeping round" that was previously done between every scheduler round (Remove inter-round collect info step in scheduler #1258).

Code health:

• We've continued expanding our test suite, primarily in Rust.
• We now build and publish the developer documentation for our Rust code to https://shadow.github.io/docs
• We've added support in Rust for descriptors, and for writing syscall handlers (Add support for Rust syscalls/descriptors, and implement pipe() in Rust #1068).
• We've added support for new socket types to Shadow's network interface, enabling future support for implementing socket descriptors in Rust (Support multiple socket types in the network interface #1144).
• We now run a Tor simulation as part of our integration tests in the dev branch (Run small Tor network as part of CI tests #1184) using all supported versions of tor (we were already doing so in main).

Happy simulating!

The Shadow team