v2.4.0

Summary

In this release, we continue our transition from C to Rust. Most of the changes included in the release are backend changes that support our continued Rust migration. However, we also fixed many bugs and made some other changes to improve the experience for users as described below.

We intend additional work following this release to focus on changes to some of Shadow's core networking components, including the TCP stack and other facilities for forwarding packets between nodes. This is somewhat higher risk work that could result in bugs that affect Shadow's network performance and stability. We are issuing this v2.4.0 release now to ensure that users have a stable version of Shadow that they can use while we work on the high risk networking code.

Primary user-facing changes since v2.3.0

• Fixed an uncommon memory leak in epoll_ctl. #2586
• Tests that use shadow and tgen now use the binaries from $PATH and not
  ~/.local/bin. #2572
• Shadow now forces the use of a specific Rust version using a
  rust-toolchain.toml file. #2614
• Added official support for Fedora 37. #2687
• Fixed a bug that could leak closed UDP sockets. #2594
• Emulate sched_{get,set}affinity syscalls. #2602
• Emulate reading from /sys/devices/system/cpu/possible and
  /sys/devices/system/cpu/online. #2602
• Fixed the TCP header sizes in pcap files. #2620
• Various minor improvements to the experimental strace logger (improved
  formatting of strings, buffers, and socket addresses, added logging of
  vdso-handled syscalls, etc).
• Added etcd and wget2 examples to the examples/ directory. #2637, #2659
• Improved the line styles in plotting script. #2638
• Support higher-level host-specific log levels. #2645
• Fixed a bug where a socket can receive packets that were intended for a
  different socket. #2593

All Merged Pull Requests

• Build shadow once for extra tests by @sporksmith in #2572
• Remove Transport by @stevenengler in #2578
• Delete unused hostc_setup by @sporksmith in #2582
• SelfContainedMutex: move from shadow-shim-helper-rs to shadow_shmem by @sporksmith in #2567
• Host::add_application: take Rust objects and remove unsafe by @sporksmith in #2583
• Move process list to Rust Host and delete HostCInternal by @sporksmith in #2584
• Remove "Tor Tests" ci badge from readme by @stevenengler in #2585
• Fix epoll memory leak by @stevenengler in #2586
• Add SyscallHandler::legacy_syscall helper function by @stevenengler in #2588
• Add InetSocket enum and placeholder TcpSocket struct by @stevenengler in #2589
• Reset changelog by @stevenengler in #2591
• Extra Tests CI: Improve Rust caching by @sporksmith in #2575
• Make clippy happy by @trinity-1686a in #2587
• Bump nix from 0.25.0 to 0.26.1 in /src by @dependabot in #2580
• Update network interface association/disassociation by @stevenengler in #2594
• Add initial version of the TCP wrapper by @stevenengler in #2595
• Remove rust code that is no longer used by @stevenengler in #2598
• Support getsockname/getpeername() for tcp wrapper by @stevenengler in #2599
• Support ioctl() for tcp wrapper by @stevenengler in #2600
• Add support for rust sockets in the network interface by @stevenengler in #2603
• Don't refcount Process by @sporksmith in #2601
• Remove Host::setup() by @stevenengler in #2606
• Add examples for dynamic shadow config generation by @sporksmith in #2605
• Changed systemctl TasksMax Parameter by @Ti-ger in #2609
• Move host networking to new NetworkNamespace object by @stevenengler in #2607
• emulate sched_{get,set}affinity and sysconf(_SC_NPROCESSORS_*) by @trinity-1686a in #2602
• TypedPluginPtr: quote code in doc comment by @sporksmith in #2613
• regular_file.c: don't treat 0 as a bad file descriptor by @sporksmith in #2616
• Bump memoffset from 0.7.1 to 0.8.0 in /src by @dependabot in #2615
• maintainer playbook: Handle project-issues by @sporksmith in #2610
• Enforce clippy by @sporksmith in #2614
• Support bind() for tcp wrapper by @stevenengler in #2611
• Make the Rust Process own the C Process by @sporksmith in #2612
• Rename borrowing accessors by @sporksmith in #2623
• Changed two debug logs to trace by @stevenengler in #2627
• Add rust TcpSocket to network interface as LegacySocket by @stevenengler in #2626
• Simplify syscall logging by @stevenengler in #2624
• Remove unused router code by @robgjansen in #2628
• Use 0 as last refill time for new token buckets by @robgjansen in #2629
• Fix an issue with the _test_implicit_bind test by @stevenengler in #2630
• TGen speed test threshold by @robgjansen in #2633
• Issue 2619; fixes TCP header sizes in pcap files by @rwails in #2620
• Bump once_cell from 1.16.0 to 1.17.0 in /src by @dependabot in #2635
• updating tgen getting started notes to better address the ../../.. prefix by @rwails in #2634
• Add example (and test) for etcd by @stevenengler in #2637
• Improve line styles in plotting script by @stevenengler in #2638
• Small code improvements by @stevenengler in #2641
• Check for specific socket association before wildcard by @stevenengler in #2640
• Show nanoseconds in strace log by @stevenengler in #2642
• Bump lzma-rs from 0.2.0 to 0.3.0 in /src by @dependabot in #2644
• Migrate most of Process from C to Rust by @sporksmith in #2631
• Update dependencies by @stevenengler in #2647
• Log syscalls that are handled in the shim by @stevenengler in #2646
• Simplify strace logging in Process by @stevenengler in #2649
• Support displaying more-complex syscall arguments by @stevenengler in #2648
• Process: move thread list to Rust by @sporksmith in #2650
• Bump Rust to 1.66.1 by @sporksmith in #2655
• Log socket addresses in the syscall logger by @stevenengler in #2652
• Allow higher-level host log levels by @stevenengler in #2645
• Fix sched_{get,set}affinity bug by @stevenengler in #2657
• Added example for Wget2 by @stevenengler in #2659
• Convert some Process code to Rust by @sporksmith in #2651
• bind-shadow test: extend timeout by @sporksmith in #2663
• Remove unneeded workaround in syscallhandler_epoll_ctl by @stevenengler in #2660
• Bump clap from 4.0.32 to 4.1.0 in /src by @dependabot in #2666
• Don't stall when already delivered packets are queued by @robgjansen in #2667
• Convert remaining Process code to Rust by @sporksmith in #2661
• Change log msg level from info to debug by @stevenengler in #2672
• Improve strace string logging by @stevenengler in #2669
• Remove C Process by @sporksmith in #2665
• Warn if running with root privileges by @stevenengler in #2670
• Support InetSocket in the tracker by @stevenengler in #2671
• Multithread death by @sporksmith in #2680
• Bump which from 4.3.0 to 4.4.0 in /src by @dependabot in #2682
• ci scripts: use rust-toolchain.toml when installing Rust by @sporksmith in #2688
• Add support for Fedora 37 in the CI by @stevenengler in #2687
• Update CHANGELOG with missing entries by @stevenengler in #2694

New Contributors

• @trinity-1686a made their first contribution in #2587
• @Ti-ger made their first contribution in #2609

Full Changelog: v2.3.0...v2.4.0