FYI: serde hot-path bottleneck in russh-sftp (with fix) — upstream is stalled

[Yaminyam]

Heads-up for fellow russh users who also depend on russh-sftp: while profiling bssh-server (russh-based SSH server) for SFTP upload throughput, I traced a large performance bottleneck into russh-sftp, filed a fix upstream, and want to leave a breadcrumb here since russh-sftp is unrelated to this repo but sits in the same de-facto stack.

TL;DR

• protocol::Write and protocol::Data in russh-sftp use #[derive(Deserialize)] on data: Vec<u8>, which dispatches to serde's deserialize_seq and walks the payload one byte at a time through VecVisitor. For a 1 GiB upload that's ~10⁹ visitor dispatches.
• perf on the server shows ~42% CPU in VecVisitor::visit_seq. AEAD (aws-lc-rs) is only ~6% combined — crypto is not the limit.
• Fix: annotate the two fields with #[serde(with = "serde_bytes")] so Vec<u8> routes through deserialize_byte_buf, which the crate's own Deserializer already implements as a single bulk try_get_bytes. The serializer side gets a matching serialize_bytes implementation with the same u32 len + bytes framing, so the wire format is unchanged.
• Measured on a CPU-bound host (Xeon Silver 4214): 1 GiB SFTP upload 74.8 → 96.4 MiB/s (+29%). OpenSSH sftp-server is ~101 MiB/s on the same host, so the gap shrinks from ~26% to ~5%. Network-bound hosts see no change, as expected.
• Wire format verified compatible with stock OpenSSH sftp client end-to-end.

Upstream status

• PR: AspectUnk/russh-sftp#83
• AspectUnk/russh-sftp has had no commits since the v2.1.1 bump on 2025-04-18, and related reports #55 / #70 sit without a root-cause analysis. PR may or may not land.

Why post this here

russh-sftp is a separate repo under a different maintainer, so strictly speaking this isn't a russh bug. But it's the default SFTP subsystem for almost every russh-based server I can find on GitHub (~340 hits), and the bottleneck is invisible to anyone who hasn't profiled it — the serde derive looks innocuous. Leaving the note here so russh users stumbling over slow SFTP have something to grep for, and in case the russh project ever wants to consider a closer relationship with (or fork of) russh-sftp given its stalled maintenance.

No ask on the russh side — russh itself has been great, and the recent #653 mlock perf fix even compounds nicely with this one.