SDK: add smol-dvpn to provide per app user space tunnels - #6953
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe PR adds three Rust workspace crates: ChangesUserspace dVPN SDK
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
jstuczyn
force-pushed
the
feature/nym-sdk-dvpn
branch
from
July 22, 2026 11:59
c9b24e9 to
3440ae7
Compare
mfahampshire
left a comment
mfahampshire
left a comment
Contributor
There was a problem hiding this comment.
smol-* changes lgtm
jstuczyn
force-pushed
the
feature/nym-sdk-dvpn
branch
from
July 22, 2026 12:46
3440ae7 to
2e81ea4
Compare
jstuczyn
force-pushed
the
feature/nym-sdk-dvpn
branch
from
July 22, 2026 12:58
2e81ea4 to
dc7b20a
Compare
jstuczyn
requested changes
Jul 22, 2026
jstuczyn
left a comment
jstuczyn
left a comment
Contributor
There was a problem hiding this comment.
One big issue that I let the A1 friends format in a more readable language, but tl;dr; we don't have a mechanism for automatic top-ups which I think will cause a lot of friction with library consumers:
Bandwidth top-up isn't automatic - and can't self-sustain even when wired up
The docs (design §10) read as though a live tunnel keeps its own bandwidth topped up. In the code it doesn't, and I think that's a gap against what library users will reasonably expect: bring up a tunnel, have it stay alive without writing plumbing.
Three issues:
-
Off by default, opt-in only.
TunnelBuilder.topupisNoneunless the caller explicitly callsbandwidth_topup(config, source)(tunnel.rs:147). Without it, the tunnel just stops passing traffic once the registered bandwidth is spent - no warning, no refill. -
Caller has to wire two crates together themselves. The
BandwidthCredentialSourcetrait lives innym-smol-dvpn, andSessiondoesn't implement it (dependency direction forbids it). So integrators must hand-roll an adapter overSession::obtain_wireguard_credential- and we don't ship
one. Thesmol-dvpn-topupexample is one-shot; nothing demonstrates the background task. -
Even when enabled, it can't reach back to the chain.
obtain_wireguard_credentialonly spends an already-stored ticket and errors with"no stored ticket available for top-up"(session.rs:222) if the store is dry - it never callsfetch_ticketbook. So a long-lived tunnel silently dies
once the pre-stocked tickets run out; the mnemonic is never used to re-issue mid-session.
Suggestions:
- Make top-up the default for a long-lived tunnel.
- Provide a built-in
Session-backedBandwidthCredentialSource(or an adapter constructor) so callers don't implement anything. - Have live top-up fall back to
fetch_ticketbook(mnemonic re-issuance) when the store runs low.
At minimum, the design doc should be corrected to state that top-up is manual and store-limited.
furthermore:
Datapath does several needless heap allocations per packet
The per-packet hot path allocates (and in places zero-fills) far more than needed, across four files. For a single outbound two-hop packet this adds up to well over 100 KiB allocated + memset, and inbound is worse:
engine.rs—encap/decap/timer/handshake_initeach dolet mut out = vec![0u8; 65535]per call (plus a fresh 64 KiB buffer per iteration of the decap drain loop), then.to_vec()the result. WG packets are ~1.4 KB, so these buffers are ~46x oversized and the zero-init is pure waste. A two-hop send isencap(exit)+encap(entry)= two 64 KiB alloc+zero.transport.rs—WgReceiver::recvallocates and zeroes a 65535-byte buffer per received packet, fills ~1.4 KB, then truncates.bridge.rs—senddoesBytes::copy_from_slice(packet)andrecvdoesframe.to_vec(), a copy each per packet.framing.rs—build_ipv4_udpallocates aVecper call andparse_ipv4_udpdoes.to_vec()on the payload.
None of these are correctness bugs, but the datapath is exactly where allocator churn shows up under load. Since the engine runs single-task (no locking), it can own reusable scratch buffers and hand &mut slices to boringtun; the transport/bridge halves can reuse a buffer sized to a realistic WG max (MTU + overhead) instead of the UDP max. Worth one focused "reduce datapath allocations" pass rather than fixing piecemeal.
| } | ||
|
|
||
| /// Set the assigned IPv6 address. | ||
| pub fn with_ipv6(mut self, ipv6: Ipv6Addr) -> Self { |
Contributor
There was a problem hiding this comment.
given lack of dual-stack support (for now) we should mark this method with a warning, because it will not do anything
Contributor
Author
There was a problem hiding this comment.
Added a warning doc to with_ipv6: dual-stack isn't wired end-to-end yet (tokio-smoltcp 0.5 binds a single interface address), so it currently has no routing effect.
| let mut addrs = Vec::new(); | ||
| // A and AAAA are independent queries; collect whatever resolves. | ||
| for rtype in [RecordType::A, RecordType::AAAA] { | ||
| match query_one(net, cfg, &name, rtype).await { |
Contributor
There was a problem hiding this comment.
possible improvement: we could make the A and AAAA queries concurrently given most of the time is going to be spent waiting on IO
Contributor
Author
There was a problem hiding this comment.
Superseded: AAAA is now skipped entirely while the stack is IPv4-only, so there's only a single query. Left a note that join! applies once dual-stack lands.
| /// any sockets open at the moment of the change are reset. (A fully seamless | ||
| /// in-place interface resize is not supported by `tokio-smoltcp`, which fixes | ||
| /// the interface MTU at construction.) | ||
| pub fn set_mtu(&self, mtu: MtuConfig) -> Result<()> { |
Contributor
There was a problem hiding this comment.
set_mtu takes &self and does the swap send and the stack publish as two separate steps with no lock spanning both. Two concurrent calls can interleave so the datapath ends on call B's channels while self.stack ends on call A's stack - a permanent desync where callers write into a stack whose channels the datapath never reads. Either take &mut self, hold a dedicated swap mutex across both steps, or document it as single-caller.
Contributor
Author
There was a problem hiding this comment.
Fixed — set_mtu now serialises the whole swap behind a mutex, so two concurrent calls can't leave the published stack and the datapath's channels referring to different stacks.
| Some("http") => 80, | ||
| _ => 443, | ||
| }); | ||
| let stream = stack.tcp_connect_host(&host, port).await?; |
Contributor
There was a problem hiding this comment.
IP-literal hosts are sent through DNS and fail. tcp_connect_host unconditionally resolves its host argument (smol-core's resolve does Name::from_utf8(host) → DNS query, with no host.parse::<IpAddr>() short-circuit), so a URI whose host is already an IP becomes a bogus lookup:
http://10.0.0.1:50051 → resolve("10.0.0.1") → A-query for the name 10.0.0.1. → NXDOMAIN → DnsNoRecords → connection fails.
This is the exact example in this module's own doc comment (from_static("http://10.0.0.1:50051")), and dVPN targets are commonly raw IPs - so the documented snippet can't actually connect. Root cause is in smol-core's resolve/tcp_connect_host, but it surfaces here.
Fix: parse host as an IpAddr first and call stack.tcp_connect((ip, port).into()) directly, only falling through to tcp_connect_host for real hostnames. (While there, strip brackets from IPv6 literals — Uri::host() returns "[::1]" - or reject v6 with a clear error instead of a confusing DNS failure.)
Contributor
Author
There was a problem hiding this comment.
Fixed in both places — smol-core's resolve/tcp_connect_host short-circuit an IP-literal host (no DNS query), and TunnelConnector strips IPv6 brackets from Uri::host() and connects directly. The module's http://10.0.0.1:50051 doc example works now.
|
|
||
| /// Build a metadata HTTP client for the given endpoint URL. | ||
| pub(crate) fn metadata_client(url: &str) -> Result<MetadataHttpClient> { | ||
| MetadataHttpClient::new_url(url, None) |
Contributor
There was a problem hiding this comment.
Top-up traffic bypasses the tunnel — likely broken, and a privacy leak if not.
metadata_client builds a plain MetadataHttpClient::new_url(url, None) with the default connector, and run_topup is given no Stack/connector — so every available_bandwidth / topup_once request goes over the host network, not through the userspace tunnel. This contradicts the design (§10), which says the metadata endpoint is "reached through the tunnel ... just another destination reachable via smol-core's tcp_connect on the running tunnel."
Either way it's a problem:
- If the gateway serves metadata in-tunnel (private tunnel IP inside the entry peer's
allowed_ips, as §10 describes), a host-network client can't reach it → top-up always fails and the feature is dead. - If it's publicly reachable, top-up works but leaks the client's real IP to the gateway metadata endpoint, correlating the real IP with the tunnel session — a deanonymization hole in a privacy tool.
Fix: route the metadata client through the tunnel — thread the tunnel's TunnelConnector into run_topup and construct MetadataHttpClient over it (or have the caller pass a tunnel-backed HTTP client). As-is I'd treat this as a blocker: the feature probably can't work against an in-tunnel metadata endpoint, and where it does work it undermines the VPN.
Contributor
Author
There was a problem hiding this comment.
Fixed — the metadata client now dials through the tunnel (a hyper HTTP/1 client over the tunnel's own TunnelConnector / in-tunnel TcpStream), never the host network, so top-up can't leak the client's real IP. On the broader point: gateway-side top-up (spending already-stored tickets) is now on by default for session-built tunnels via a shipped ProviderCredentialSource; buying new ticketbooks (chain-side restock) is opt-in via with_automatic_topups because a ticketbook is ~37.5 GB; and a Tunnel::bandwidth_events() stream lets integrators react (e.g. prompt the user) even when auto-top-up is off. Design §10 corrected.
jstuczyn
marked this pull request as ready for review
July 23, 2026 09:13
jstuczyn
force-pushed
the
feature/nym-sdk-dvpn
branch
from
July 23, 2026 09:20
7f3c80c to
5d57eb6
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 12
🧹 Nitpick comments (3)
docs/design/sdk/smol-dvpn/design.md (1)
45-54: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winFix the shared Markdown lint issue.
docs/design/sdk/smol-dvpn/design.md#L45-L54: label the crate-family diagram fence.docs/design/sdk/smol-dvpn/design.md#L92-L124: label the data-path diagram fence.docs/design/sdk/smol-dvpn/design.md#L147-L158: label the pseudocode fence.docs/design/sdk/smol-dvpn/design.md#L194-L206: label the control/data-plane diagram fence.docs/design/sdk/smol-dvpn/design.md#L358-L364: label the lifecycle diagram fence.docs/design/sdk/smol-dvpn/README.md#L53-L63: label the architecture diagram fence.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/design/sdk/smol-dvpn/design.md` around lines 45 - 54, Markdown diagram and pseudocode fences are unlabeled, causing lint failures. In docs/design/sdk/smol-dvpn/design.md at lines 45-54, 92-124, 147-158, 194-206, and 358-364, and docs/design/sdk/smol-dvpn/README.md at lines 53-63, add an appropriate language or diagram fence label to each affected block while preserving its contents.Source: Linters/SAST tools
sdk/rust/smol-dvpn/src/engine.rs (1)
45-94: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winFresh 64KB zeroed buffer allocated per packet/timer tick.
encap,decap,timer, andhandshake_initeachvec on every call — and these run per application packet, per inbound datagram, and every 250ms via the timer pump intunnel.rs. This is avoidable allocation+zeroing churn on the hot datapath.Consider holding one or two reusable scratch buffers as part of
WgEngine's state (or a small buffer pool) and writing into them instead of allocating fresh each call.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@sdk/rust/smol-dvpn/src/engine.rs` around lines 45 - 94, Eliminate the per-call 64KB allocations in encap, decap, timer, and handshake_init by adding reusable scratch buffer state to WgEngine (or a small pool) and passing those buffers to boringtun, preserving each function’s current packet handling and returned owned data.sdk/rust/smol-dvpn/examples/quic-probe.rs (1)
122-171: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
quic-probealways exits 0, even when every probe fails.
probe_targetonly printsFAIL/OK; nothing propagates failure back tomain, which unconditionally callsstd::process::exit(0). This makes the tool unusable for scripted/CI reachability checks — a caller can't tell success from failure without parsing stdout.♻️ Track failures and exit non-zero
-async fn probe_target(name: &str, addr: &str, sni: &str, id_b64: &str) { +async fn probe_target(name: &str, addr: &str, sni: &str, id_b64: &str) -> bool { println!("\n== {name} =="); println!(" addr={addr} sni={sni}"); let sock: SocketAddr = match addr.parse() { Ok(s) => s, Err(e) => { println!(" invalid addr: {e}"); - return; + return false; } }; + let mut ok = true; // 1. Raw reachability (hq-29). match raw_handshake(sock, sni, b"hq-29").await { Ok(()) => {} - Err(e) => println!(" raw: FAIL — {e}"), + Err(e) => { println!(" raw: FAIL — {e}"); ok = false; } } ... match probe_bridge(¶ms, &CancellationToken::new()).await { Ok(()) => println!(" pinned: OK — bridge connect + open_bi succeeded"), - Err(e) => println!(" pinned: FAIL — {e}"), + Err(e) => { println!(" pinned: FAIL — {e}"); ok = false; } } + ok } #[tokio::main(flavor = "multi_thread", worker_threads = 2)] async fn main() { ... + let mut all_ok = true; if let (Some(addr), Some(sni), Some(id)) = (arg("--addr"), arg("--sni"), arg("--id")) { - probe_target("custom", &addr, &sni, &id).await; + all_ok = probe_target("custom", &addr, &sni, &id).await; } else { for t in DEFAULTS { - probe_target(t.name, t.addr, t.sni, t.id_b64).await; + all_ok &= probe_target(t.name, t.addr, t.sni, t.id_b64).await; } } println!("\ndone."); - std::process::exit(0); + std::process::exit(if all_ok { 0 } else { 1 }); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@sdk/rust/smol-dvpn/examples/quic-probe.rs` around lines 122 - 171, Update probe_target to return whether both reachability checks succeed, propagating failure instead of only printing it. Have main aggregate the results across the selected target(s) and exit with a non-zero status when any probe fails, while retaining exit code 0 only when all probes succeed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@common/smol-core/README.md`:
- Around line 19-32: Update the README Rust example around Stack::new and the
tcp_connect/udp_socket/resolve calls by placing it in an async main returning a
Result and executing it with the documented Tokio runtime approach, so its await
and ? expressions compile in context while preserving the existing example flow.
In `@common/smol-core/src/dns.rs`:
- Around line 102-119: Check the parsed DNS message’s truncation flag before
collecting or returning addresses in the response handling flow. When the TC bit
is set, reject the response using the existing DNS error path so callers can
retry over TCP; only collect and return A/AAAA records for complete responses.
In `@common/smol-core/src/stack.rs`:
- Around line 43-77: Make the configured MTU effective in Stack::new by using
StackConfig::mtu when creating the caller-owned device, or remove the mtu field
and with_mtu method if device construction cannot consume it. Ensure with_mtu no
longer silently has no effect and preserve the existing default MTU behavior
from StackConfig::new.
- Around line 157-161: Update tcp_connect_host to access the resolved addresses
with first() instead of indexing at 0, and map an empty result to the
established SmolCoreError before constructing SocketAddr. Preserve the existing
tcp_connect flow for a successfully resolved address.
In `@common/smol-core/tests/stack.rs`:
- Around line 119-124: Update the connection assertion around tcp_connect so the
test succeeds only when it returns Ok(Err(_)). Treat both successful connections
and timeout/outer Err results as failures, preserving the panic context for
unexpected outcomes.
In `@deny.toml`:
- Line 108: Remove the global "AGPL-3.0" allowance from deny.toml, while
preserving the existing crate-scoped exception for nym-bridges so AGPL approval
remains limited to that crate.
In
`@openspec/changes/archive/2026-07-11-nym-sdk-dvpn/specs/dvpn-quic-bridge/spec.md`:
- Around line 21-29: Align the QUIC bridge implementation with the contract by
removing the sdk/rust/smol-dvpn dependency on nym_bridges and replacing
bridge.rs delegation to nym_bridges::transport::quic with inline quinn logic
covering ALPN, certificate pinning, and length-prefixed bidirectional streams;
configure keep-alive, max idle timeout, and BBR for long-lived sessions. Update
the requirements in
openspec/changes/archive/2026-07-11-nym-sdk-dvpn/specs/dvpn-quic-bridge/spec.md
lines 21-29 and 62-66, design.md lines 76-84, and proposal.md lines 29-43 and
78-80 to consistently describe the implemented guarantees.
In `@sdk/rust/nym-sdk-session/src/dvpn.rs`:
- Around line 132-139: Update the QuicBridge construction path to reject entries
whose args.id_pubkey is empty or whitespace-only before returning
Some(QuicBridge), ensuring has_quic cannot select an unpinned bridge. Preferably
also validate that the identity pin is valid base64, while preserving the
existing empty-address rejection.
In `@sdk/rust/nym-sdk-session/src/session.rs`:
- Around line 265-276: Make registration non-cancellable once ticket spending
begins: update register_single_hop, register_two_hop, and register_two_hop_quic
and their _inner flows to honor cancellation only before the
register_dvpn/handshake_and_register_dvpn ticket-spend boundary, then let
issuance and registration complete and return the resulting Registration.
Preserve SessionError::Cancelled for cancellation occurring before that
boundary.
In `@sdk/rust/smol-dvpn/src/bridge.rs`:
- Around line 123-134: Update the connection setup around `conn.open_bi()` so it
is also wrapped in the existing `tokio::select!` cancellation boundary and
`tokio::time::timeout(CONNECT_TIMEOUT, ...)`. Return `DvpnError::Cancelled` when
`cancel` fires, map timeout failures to the established bridge timeout error,
and preserve the existing `open_bi` error context for transport failures.
In `@sdk/rust/smol-dvpn/tests/live_bringup.rs`:
- Around line 108-115: Remove the std::process::exit(0) call from probe_traffic
after the bounded tunnel.shutdown timeout so the test returns normally and
sibling live-bringup tests can complete. Preserve the existing timeout-based
teardown behavior and success message.
In `@smolmix/core/tests/public_api.rs`:
- Around line 13-30: Strengthen _api_lock by type-checking each referenced
Tunnel and TunnelBuilder API with realistic arguments or explicit
function-pointer signatures, rather than only assigning method names. Ensure
constructors, lifecycle methods, networking methods, and builder methods
validate their current parameter and return signatures while remaining
unreachable at runtime.
---
Nitpick comments:
In `@docs/design/sdk/smol-dvpn/design.md`:
- Around line 45-54: Markdown diagram and pseudocode fences are unlabeled,
causing lint failures. In docs/design/sdk/smol-dvpn/design.md at lines 45-54,
92-124, 147-158, 194-206, and 358-364, and docs/design/sdk/smol-dvpn/README.md
at lines 53-63, add an appropriate language or diagram fence label to each
affected block while preserving its contents.
In `@sdk/rust/smol-dvpn/examples/quic-probe.rs`:
- Around line 122-171: Update probe_target to return whether both reachability
checks succeed, propagating failure instead of only printing it. Have main
aggregate the results across the selected target(s) and exit with a non-zero
status when any probe fails, while retaining exit code 0 only when all probes
succeed.
In `@sdk/rust/smol-dvpn/src/engine.rs`:
- Around line 45-94: Eliminate the per-call 64KB allocations in encap, decap,
timer, and handshake_init by adding reusable scratch buffer state to WgEngine
(or a small pool) and passing those buffers to boringtun, preserving each
function’s current packet handling and returned owned data.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: b9216b71-d59d-4e03-9f93-497e0a6b8894
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (60)
Cargo.tomlcommon/smol-core/Cargo.tomlcommon/smol-core/README.mdcommon/smol-core/src/device.rscommon/smol-core/src/dns.rscommon/smol-core/src/error.rscommon/smol-core/src/lib.rscommon/smol-core/src/stack.rscommon/smol-core/tests/stack.rsdeny.tomldocs/design/sdk/smol-dvpn/README.mddocs/design/sdk/smol-dvpn/design.mdopenspec/changes/archive/2026-07-11-nym-sdk-dvpn/.openspec.yamlopenspec/changes/archive/2026-07-11-nym-sdk-dvpn/design.mdopenspec/changes/archive/2026-07-11-nym-sdk-dvpn/proposal.mdopenspec/changes/archive/2026-07-11-nym-sdk-dvpn/specs/dvpn-quic-bridge/spec.mdopenspec/changes/archive/2026-07-11-nym-sdk-dvpn/specs/dvpn-session/spec.mdopenspec/changes/archive/2026-07-11-nym-sdk-dvpn/specs/dvpn-tools/spec.mdopenspec/changes/archive/2026-07-11-nym-sdk-dvpn/specs/dvpn-tunnel/spec.mdopenspec/changes/archive/2026-07-11-nym-sdk-dvpn/specs/smol-core-stack/spec.mdopenspec/changes/archive/2026-07-11-nym-sdk-dvpn/tasks.mdopenspec/specs/dvpn-quic-bridge/spec.mdopenspec/specs/dvpn-session/spec.mdopenspec/specs/dvpn-tools/spec.mdopenspec/specs/dvpn-tunnel/spec.mdopenspec/specs/smol-core-stack/spec.mdsdk/rust/nym-sdk-session/Cargo.tomlsdk/rust/nym-sdk-session/README.mdsdk/rust/nym-sdk-session/src/dvpn.rssdk/rust/nym-sdk-session/src/error.rssdk/rust/nym-sdk-session/src/gateway.rssdk/rust/nym-sdk-session/src/lib.rssdk/rust/nym-sdk-session/src/session.rssdk/rust/smol-dvpn/.gitignoresdk/rust/smol-dvpn/Cargo.tomlsdk/rust/smol-dvpn/README.mdsdk/rust/smol-dvpn/examples/common/mod.rssdk/rust/smol-dvpn/examples/quic-probe.rssdk/rust/smol-dvpn/examples/smol-dvpn-config.rssdk/rust/smol-dvpn/examples/smol-dvpn-grpc.rssdk/rust/smol-dvpn/examples/smol-dvpn-topup.rssdk/rust/smol-dvpn/examples/two-hop-ip.rssdk/rust/smol-dvpn/examples/two-hop-quic.rssdk/rust/smol-dvpn/examples/zcash-sync.rssdk/rust/smol-dvpn/src/bridge.rssdk/rust/smol-dvpn/src/config.rssdk/rust/smol-dvpn/src/connectors.rssdk/rust/smol-dvpn/src/engine.rssdk/rust/smol-dvpn/src/error.rssdk/rust/smol-dvpn/src/framing.rssdk/rust/smol-dvpn/src/lib.rssdk/rust/smol-dvpn/src/topup.rssdk/rust/smol-dvpn/src/transport.rssdk/rust/smol-dvpn/src/tunnel.rssdk/rust/smol-dvpn/tests/live_bringup.rssmolmix/core/Cargo.tomlsmolmix/core/src/error.rssmolmix/core/src/lib.rssmolmix/core/src/tunnel.rssmolmix/core/tests/public_api.rs
💤 Files with no reviewable changes (1)
- smolmix/core/src/lib.rs
Comment on lines
+119
to
+124
| // Either the handshake was refused (Err) or it never completed (timeout). | ||
| // The one thing that must NOT happen is a successful connection. | ||
| if let Ok(Ok(_stream)) = result { | ||
| panic!("connect unexpectedly succeeded to a closed port") | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Require a connection error instead of accepting a timeout.
This test passes when tcp_connect never completes, despite the specified behavior requiring an I/O error for a refused connection. Fail the test on timeout and require Ok(Err(_)).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@common/smol-core/tests/stack.rs` around lines 119 - 124, Update the
connection assertion around tcp_connect so the test succeeds only when it
returns Ok(Err(_)). Treat both successful connections and timeout/outer Err
results as failures, preserving the panic context for unexpected outcomes.
Contributor
Author
There was a problem hiding this comment.
Intentional: the closed-port test accepts either a connection-refused error or a connect timeout so it's stable across platforms (some return RST, others just time out). The invariant it guards is "must not connect successfully," which it still enforces.
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
ast-grep outline common/smol-core/src/stack.rs --items all
printf '\n--- tcp_connect implementation ---\n'
rg -n -A35 -B8 'pub\s+async\s+fn\s+tcp_connect\b' common/smol-core/src/stack.rs
printf '\n--- closed-port test ---\n'
sed -n '108,130p' common/smol-core/tests/stack.rs
printf '\n--- specified TCP failure behavior ---\n'
rg -n -A12 -B4 'handshake failure|shutdown.*I/O error|tcp_connect' openspec/specs/smol-core-stack/spec.md openspec/changes/archive/2026-07-11-nym-sdk-dvpn/specs/smol-core-stack/spec.mdLength of output: 11285
@mmsinclair, I understand the intended invariant, but I don’t think the platform-stability rationale applies to this test: both peers are in-process smoltcp stacks connected by channels, rather than host TCP sockets.
More importantly, the outer tokio::time::timeout expiring indicates that the test harness gave up; it does not demonstrate tcp_connect returned the required I/O error. Stack::tcp_connect’s own documentation and the smol-core specification explicitly require failed/refused/timed-out handshakes to resolve as Err, rather than remain pending.
So this test should continue to use a bounded test timeout, but fail if that timeout elapses and only pass on Ok(Err(_)).
mmsinclair
added a commit
that referenced
this pull request
Jul 23, 2026
Datapath / correctness: - topup: dial the gateway metadata endpoint THROUGH the tunnel (hyper/1 over TunnelConnector) instead of the host network, closing the IP-leak/broken-topup blocker. - tunnel: fix the set_mtu stack-swap race (biased select! with the swap branch first, break on swap-channel closure) and serialise concurrent set_mtu behind a mutex; classify transport recv errors (fatal bridge close exits; transient UDP backs off) to end the busy-loop. - connectors/smol-core: connect IP-literal hosts directly instead of routing them through DNS (strip IPv6 brackets); short-circuit in resolve/tcp_connect_host. - bridge: prefer IPv4 candidate addresses for v4-only clients. smol-core DNS: random per-query id validated against the response id AND source, recv loop until match/timeout, distinct DnsServerFailure for SERVFAIL/REFUSED, and skip AAAA on a v4-only stack (without discarding resolved A records). nym-sdk-session: run the bandwidth controller (single writer) and spend via its request sender; scope acquisition to WireGuard ticket types; opt-in chain restock (with_automatic_topups), gateway-side top-up on by default via a shipped ProviderCredentialSource; external-provider escape hatch; hashed+zeroized client id (no mnemonic clone); single topology fetch for two-hop; drop the mixnet entry/exit role restriction for dVPN selection; split Nyxd/Api errors. Performance: engine owns one reusable encrypt/decrypt scratch buffer and the Direct receiver reuses its recv buffer (no per-packet 64 KiB alloc+zero). Also: BandwidthEvent stream (tunnel.bandwidth_events()), typed WG keys in PeerConfig, redacted Debug, target-locked mobile MTU, #[must_use] builders, and doc corrections (design §10 top-up model; nym-bridges now on crates.io).
mmsinclair
added a commit
that referenced
this pull request
Jul 24, 2026
Resolves the PR #6953 base conflicts. Resolution decisions: - Cargo.toml: took develop's workspace-wide 1.21.5-rc.1 bump; re-added this branch's members (common/smol-core, sdk/rust/nym-sdk-session, smoldvpn) and workspace deps (nym-registration-client, nym-sdk-session, smol-core) at the new version, in sorted positions. - common/smol-core/src/device.rs: kept the shared ChannelDevice as-is — explicit usize MTU (policy stays with callers) and the documented unbounded max_burst_size (capping at 1 serialized the datapath; smoldvpn throughput depends on this). Removed develop's smolmix-specific client_mtu() that rename-detection auto-merged into the shared file. - smolmix/core/src/tunnel.rs: kept this branch's smol-core Stack architecture and adopted develop's MTU negotiation — client_mtu() (SMOLMIX_MTU override > Android const > IPR-negotiated > fallback) ported to the call-site, feeding ChannelDevice::new an explicit MTU. - nym-sdk-session: adapted to develop's register_dvpn signature change by passing spend_time_skew: None at all three registration sites (no spend-time backdating; can be exposed via SessionConfig if skew tolerance is ever needed). Verified: smoldvpn, nym-sdk-session, smolmix, smol-core all compile; full test suites green; clippy clean incl. tests and examples.
mmsinclair
force-pushed
the
feature/nym-sdk-dvpn
branch
from
July 24, 2026 21:24
1d32986 to
9fb2268
Compare
jstuczyn
pushed a commit
that referenced
this pull request
Jul 25, 2026
Datapath / correctness: - topup: dial the gateway metadata endpoint THROUGH the tunnel (hyper/1 over TunnelConnector) instead of the host network, closing the IP-leak/broken-topup blocker. - tunnel: fix the set_mtu stack-swap race (biased select! with the swap branch first, break on swap-channel closure) and serialise concurrent set_mtu behind a mutex; classify transport recv errors (fatal bridge close exits; transient UDP backs off) to end the busy-loop. - connectors/smol-core: connect IP-literal hosts directly instead of routing them through DNS (strip IPv6 brackets); short-circuit in resolve/tcp_connect_host. - bridge: prefer IPv4 candidate addresses for v4-only clients. smol-core DNS: random per-query id validated against the response id AND source, recv loop until match/timeout, distinct DnsServerFailure for SERVFAIL/REFUSED, and skip AAAA on a v4-only stack (without discarding resolved A records). nym-sdk-session: run the bandwidth controller (single writer) and spend via its request sender; scope acquisition to WireGuard ticket types; opt-in chain restock (with_automatic_topups), gateway-side top-up on by default via a shipped ProviderCredentialSource; external-provider escape hatch; hashed+zeroized client id (no mnemonic clone); single topology fetch for two-hop; drop the mixnet entry/exit role restriction for dVPN selection; split Nyxd/Api errors. Performance: engine owns one reusable encrypt/decrypt scratch buffer and the Direct receiver reuses its recv buffer (no per-packet 64 KiB alloc+zero). Also: BandwidthEvent stream (tunnel.bandwidth_events()), typed WG keys in PeerConfig, redacted Debug, target-locked mobile MTU, #[must_use] builders, and doc corrections (design §10 top-up model; nym-bridges now on crates.io).
jstuczyn
pushed a commit
that referenced
this pull request
Jul 25, 2026
Correctness: - nym-sdk-session: registration is no longer cancellable once a WireGuard ticket is spent. Gateway selection (topology fetch) stays cancellable via fetch_topology_cancellable, but register_single_hop/register_two_hop/ register_two_hop_quic run the LP registration exchange to completion without racing the caller's cancel token — closing the lost-ticket window if a cancel fired after the gateway processed the spend. - smol-core dns: reject a truncated (TC-bit) response with a new DnsTruncated error instead of returning the partial answers it carries (RFC 1035). - smol-dvpn bridge: bound conn.open_bi() by the same CONNECT_TIMEOUT + cancellation as the QUIC handshake, so a stalled bridge can't hang connect(). Robustness / cleanup: - smol-core: remove the dead StackConfig::mtu field + with_mtu (Stack::new never read it; the caller-supplied device is the MTU source of truth). - nym-sdk-session: reject a blank id_pubkey so an unpinned bridge is never advertised as QUIC-capable. - live_bringup test: drop std::process::exit(0) (which could kill sibling tests); the tokio::time::timeout around shutdown() already bounds teardown. Docs: - smol-dvpn README: reword the kill-switch/no-leaks line to a scoped guarantee (only sockets opened through the tunnel are protected; ordinary sockets and host DNS bypass it). - design doc: clarify the bridge SNI must be an accepted cert alt-name (the "decoy" is how ordinary it looks to DPI, not a free-form hostname), and document the top-up headroom (top-up fires at a threshold above zero so the in-tunnel metadata request still fits). Adds a smol-core test asserting truncated responses are rejected.
jstuczyn
force-pushed
the
feature/nym-sdk-dvpn
branch
from
July 25, 2026 08:38
9fb2268 to
b018135
Compare
jstuczyn
force-pushed
the
feature/nym-sdk-dvpn
branch
from
July 25, 2026 14:47
b018135 to
e6eb1e9
Compare
jstuczyn
force-pushed
the
feature/nym-sdk-dvpn
branch
from
July 25, 2026 14:50
e6eb1e9 to
9b0a219
Compare
jstuczyn
force-pushed
the
feature/nym-sdk-dvpn
branch
from
July 25, 2026 15:01
9b0a219 to
11cddec
Compare
jstuczyn
pushed a commit
that referenced
this pull request
Jul 25, 2026
remove garbage data
post rebasing fixes
openspec: archive dvpn-registration-reuse after mainnet validation
Mainnet validation passed (2026-07-24): consecutive runs against the same
gateways reused the cached registrations with zero ticket spend, post-rename.
Deltas merged into live specs: dvpn-session gains 'Gateway registration
persistence and reuse', dvpn-tunnel gains 'Awaitable per-hop session
establishment'. The same run completes smoldvpn-rename task 4.1 (live run
under the new name), leaving that change fully implemented and ready to
archive whenever convenient.
smoldvpn: move to repo root and rename from nym-smol-dvpn
Implements openspec change smoldvpn-rename: the dVPN datapath crate is now
consistent with its sibling smolmix — sdk/rust/smol-dvpn -> smoldvpn/ (git
mv, history follows), package + lib renamed nym-smol-dvpn/nym_smol_dvpn ->
smoldvpn. Never published to crates.io, so the rename precedes first
publication (no legacy name to manage).
Rename mapping for archaeology:
nym-smol-dvpn -> smoldvpn (package, -p flags, prose)
nym_smol_dvpn -> smoldvpn (use paths, RUST_LOG targets)
smol-dvpn-* -> smoldvpn-* (example binaries + their data/ dirs)
- Examples smol-dvpn-{config,grpc,topup} renamed smoldvpn-*; the shared
examples (zcash-sync, two-hop-*, quic-probe) and their data dirs are
name-stable. README gains a migration note (RUST_LOG target change;
rename local data/smol-dvpn-* dirs to keep credentials).
- Workspace member updated (sorted next to smolmix); Cargo.lock follows.
- Prose updated: crate README, common/smol-core README, nym-sdk-session
README + doc comments; docs/design/sdk/smol-dvpn -> docs/design/smoldvpn.
- Live specs dvpn-tunnel/dvpn-tools/dvpn-quic-bridge updated in place; the
change's MODIFIED deltas carry identical text so archiving is idempotent.
Historical openspec change artifacts deliberately untouched.
Verified: straggler grep clean outside historical artifacts; full test
suites green for smoldvpn + nym-sdk-session; clippy clean incl. examples;
`cargo run -p smoldvpn --example zcash-sync -- --help` works; examples'
default log filter now spells smoldvpn=info,boringtun=info.
openspec: archive dvpn-signer-failure-tests + dvpn-signer-fault-http-harness
Both changes fully implemented and validated (the signer-tolerance change
including its live mainnet validation). Deltas merged into live specs:
dvpn-session gains the 'Tolerance to unresponsive ecash signers'
requirement; the new dvpn-signer-fault-harness capability spec is created
(harness, multi-API characterization, threshold probe).
dvpn-registration-reuse stays active pending its manual mainnet validation
(task 5.3). Also records the smoldvpn-rename D6 resolution: nym-smol-dvpn
was never published to crates.io, so the rename precedes first publication.
docs: ignore docs/scratch (working notes stay uncommitted)
nym-sdk-session + smol-dvpn: address code-quality review findings
Thermo-nuclear review of the branch's session work; behavior-preserving
restructurings, all suites green (47 tests) and clippy clean:
- session.rs had crossed the 1k-line boundary (747 -> 1145): split
SessionConfig + RestockPolicy into config.rs and the unit tests into
session_tests.rs (#[path] module) — session.rs is back to 792. In
passing this fixed a latent export bug: RestockPolicy was reachable only
as an unnameable public field type; it is now re-exported.
- The persist-then-assemble HopConfig tail was copy-pasted across all three
registration sites: extracted as Session::finalize_hop (testable
identity+GatewayInfo signature, same as cached_hop).
- The reg-cache lock/poison-recovery incantation was triplicated:
collapsed into Session::with_cache.
- datapath() had grown to 8 args behind a clippy::too_many_arguments allow
while two of them already formed the named StackChannels type: pass the
named pair, allow removed.
- clone_cfg's serde round-trip now documents WHY deriving Clone upstream is
wrong (PresharedKey is ZeroizeOnDrop secret material that deliberately
omits Clone), not just that the derive is missing.
Reviewed and left as-is (justified): TimeoutFetcher decorator shape,
opt-out-as-Option cache design, the examples' per-file self-containment
duplication, engine progress flags.
nym-sdk-session: HTTP-level signer-fault harness (tests only)
Implements openspec change dvpn-signer-fault-http-harness: drive the REAL
NyxdGlobalDataFetcher — real DKG discovery, real HTTP client, real
try-each-API loop — against local mock nym-apis whose ecash routes are
independently scriptable as healthy / slow / erroring / hanging (accept the
connection, never respond: the exact client-visible behavior observed
against mainnet). Dev-only; no common/ production code touched, no new
mock-server dependency (the Hang mode is trivial on a raw TcpListener).
- tests/support/http_harness.rs: MockNymApi with per-route FaultMode and
runtime re-scripting; teardown releases parked Hang connections.
- tests/support/fake_dkg.rs: single-method DkgQueryClient double answering
vk-share + threshold queries from fabricated ContractVKShares announcing
the mock URLs (owners must be real-format bech32 — discovery parses them
into cosmrs AccountIds). Fails loud on unexpected queries.
- tests/support: TestEcash gains vk-share bs58 accessors and annotated
expiration-date-signature fixtures (parseable by the real client).
- tests/signer_fault_http.rs (6 tests, ~2.6s): fixture sanity through real
discovery; characterize_* reproduction of the mainnet hang on the raw
stack; TimeoutFetcher guard measured end-to-end; multi-API bounded
retries; threshold probe both sides (K >= threshold: partials retrievable
during a full aggregated outage — the client-side aggregation go-signal;
K < threshold: insufficiency reported).
Findings recorded in the change's design.md, notably: the SDK's per-fetch
bound wraps the fetcher's whole try-each-API loop, so per-API fallback needs
a per-request bound inside query_random_apis_until_success
(common/bandwidth-fetcher) — the concrete item to hand the ecash owners.
cargo test -p nym-sdk-session: 40 tests green; clippy --tests clean.
nym-sdk-session + smol-dvpn: reuse gateway registrations across runs
Every registration used to generate fresh WireGuard keys and spend one
zk-nym ticket per hop, even against a gateway registered with moments
earlier — abandoning the previous peer's remaining ~477 MiB allowance on
the gateway (observed on mainnet: two identical runs = 4 tickets for
~500 MB used). Implements openspec change dvpn-registration-reuse; no
common/ production code touched.
- registration_cache (new): registrations.json in the session data dir —
client WG key + gateway-returned WireguardConfiguration keyed by
(network, gateway, role); atomic writes, 0600 on unix, 30-day hygiene
pruning (validity is proven by use, never assumed). Contains private
keys/PSKs: same sensitivity class as the adjacent creds.db.
- Session: register_single_hop/register_two_hop(_quic) consult the cache
after gateway selection — full hit returns with NO gateway exchange and
NO ticket spent ('reusing cached registration … — no ticket spent' at
info); partial two-hop hits register (and provision ticketbooks for)
only the missing hop. Fresh registrations persist immediately after
register_dvpn. New: invalidate_registration(gateway, role) for the
fallback path; SessionConfig::reuse_registrations (default on; opt-out
documents the peer-linkability trade-off and neither reads nor writes
the cache).
- smol-dvpn: Tunnel::await_established(timeout) — per-hop establishment
published from the engine's handshake tracking via a watch channel;
NotEstablished reports entry/exit status so callers invalidate exactly
the dead hop.
- Examples: common::connect() implements register → build →
await_established(15s) → invalidate-failed-hops → re-register-once;
adopted by zcash-sync/two-hop-ip/two-hop-quic (and smol-dvpn-grpc
inline), replacing the blind 10×3s ipinfo warmup loops with an
establishment gate + short display probe.
- Tests: 8 cache unit tests; 4 offline session-level reuse tests
(counting provider proves zero-spend cache hits, opt-out, invalidation,
restart survival); 4 loopback await_established tests with a real
boringtun responder (incl. exit-only failure attribution). The
await_established tests run on a manual runtime with shutdown_timeout —
the smol-core stack parks blocking-pool workers that outlive the
tunnel, which would hang #[tokio::test] forever in Runtime::drop (the
examples sidestep the same via std::process::exit(0)).
cargo test: 28 (session lib) + 7 (signer_failure) + 4 (await_established)
+ crate suites all green; clippy clean on lib/tests/examples for both.
Remaining: manual mainnet validation (task 5.3).
smol-dvpn: make WireGuard handshake progress observable at info level
Debugging a mainnet two-hop tunnel that registered fine but passed no data
(DNS-in-tunnel timeouts) showed the datapath is a black box between
'datapath starting' and the first application traffic: handshake success is
never logged at any level, and decapsulate errors (e.g. a rejected handshake
response) were silently swallowed.
- decap(): log TunnResult::Err at warn instead of dropping it silently.
- WgEngine: one-shot info markers — 'first datagram received from the entry
transport', 'entry-hop wireguard session established', 'exit-hop wireguard
session established' (single-hop: 'wireguard session established'), driven
off boringtun's stats().0 flipping to Some. A single default-log run now
localizes a dead tunnel: no inbound datagrams at all vs. entry handshake
never completing vs. exit handshake never completing vs. established but
no upstream data.
- 'dVPN tunnel datapath starting' now logs the entry/exit WG endpoints, so
the exact UDP destinations being dialed appear in the log.
- Fix the pre-existing clippy while-let warning in the decap drain loop.
smol-dvpn: close the session credential store cleanly in the examples
The example CLIs never called Session::shutdown(): error paths dropped the
session without awaiting the bandwidth controller's cleanup (surfacing
'SqlitePoolGuard dropped without explicit close()' for creds.db), and the
happy paths called std::process::exit(0), which skips destructors entirely —
leaving the sqlite WAL uncheckpointed (the multi-hundred-KB creds.db-wal
observed after mainnet runs).
Wrap each session-using flow in an async block and always await
session.shutdown() afterwards — success or failure — before printing the
final verdict / exiting. The shutdown awaits the controller task, which
closes the fetcher's pending-request store and the credential store
(WAL checkpoint); stored tickets are retained. Applies to all six
session-building examples: two-hop-ip, two-hop-quic, zcash-sync,
smol-dvpn-config, smol-dvpn-grpc, smol-dvpn-topup.
openspec: record successful mainnet validation of signer-failure tolerance
zcash-sync against mainnet (2026-07-23): provisioning completed in ~1m47s
instead of hanging forever; both SignerTimeout warnings fired at the 15s
bound; both wireguard ticketbooks (entry + exit, 50 tickets each) persisted
to data/zcash-sync/mainnet/creds.db. Ticks the last open task (5.4).
nym-sdk-session: tolerate unresponsive ecash signers (no hang, no lost funds)
Mainnet's aggregated-expiration-date-signatures endpoint accepts connections
and never responds (offline distributed signers — a permanent operating
condition). With no request timeout anywhere in the fetch stack, the
bandwidth controller's store_ticketbook wedged mid-persist on its run loop:
provisioning hung forever, the freshly issued (paid-for) ticketbook was never
stored, and every retry re-deposited NYM. SDK-confined fix per openspec
change dvpn-signer-failure-tests; no common/ production code touched.
- TimeoutFetcher decorator (new fetcher.rs): bounds the three read-only
global-signing-data fetches at 15s per call, surfacing a SignerTimeout
FetcherError naming unresponsive signers. fetch_ticketbooks (the deposit)
is deliberately NOT timed. Wired around NyxdCredentialFetcher in
Session::spawn_controller. A bounded failure is recoverable: the
controller's best-effort store path persists the ticketbook anyway and
readiness resolves from stored stock; missing signatures back-fill later.
- ensure_ticketbooks gains an overall 5-minute budget mapping to a new
SessionError::ProvisioningTimeout (defense in depth; cancellation race and
funds-safety unchanged). Note: SessionError is not #[non_exhaustive], so
exhaustive matchers gain one arm.
- Tier 1 tests (virtual clock, in fetcher.rs): hang→bounded error for all
three fetches, under/over-threshold boundary, error passthrough, and
issuance provably unbounded by the decorator.
- Tier 2 tests (tests/signer_failure.rs + tests/support): the real
BandwidthController over ephemeral storage with a FlakyFetcher reproducing
the mainnet outage, and a locally fabricated 2-of-3 threshold-signed
ticketbook (no chain, no network, no funds). Proves: the pre-fix wedge and
empty store (the exact observed creds.db state), post-fix persistence +
resolved readiness, zero re-purchase on retry, and fail-fast spend during
the outage.
- openspec: dvpn-signer-failure-tests artifacts (tasks ticked; only optional
manual mainnet validation remains) and the follow-up
dvpn-signer-fault-http-harness proposal (HTTP-level fault harness, may not
be implemented).
cargo test -p nym-sdk-session: 23 passed. clippy --tests: clean.
resolve .gitignore merge conflict
Keep both sides of the conflict: tmp/ + the operator-tools nodes.csv
(HEAD) and *storybook.log (NYM-1199).
smol-dvpn: log example output via tracing, restructure example data dirs
Wire a tracing subscriber into every example so the crate's logs (and the
examples' own narration) are visible out of the box, and convert the examples
from println! to tracing macros.
- Add tracing-subscriber (env-filter) dev-dep; each example installs a
stderr fmt subscriber honouring RUST_LOG. Default filter (RUST_LOG unset)
is the running example plus nym_smol_dvpn and boringtun at info; the
example crate root is derived from module_path! so its own info! shows.
- Convert progress/result output to info!, retries/probe failures to warn!,
and fatal main errors to error!. Logs go to stderr; stdout stays reserved
for genuine deliverables (the smol-dvpn-config WireGuard config and --help).
- Log the per-example credential store / data directory at startup.
- Restructure example data output to data/<example>/<network> (e.g.
data/zcash-sync/sandbox) instead of flat *-data dirs; create it if missing.
- Ignore /data/ in the crate and repo-root .gitignore.
- Update README developer notes for the logging behaviour and data layout.
use nym native types on 'PeerConfig' and remove Clone derivation
use concrete inner error for SessionError::Api
fix deps of nym-registration-client
ibid for 'nym-authenticator-client'
mark 'nym-registration-client' as publishable
clippy
openspec: sync specs + design doc with delivered dVPN design
Reconcile the OpenSpec deltas with what actually shipped and apply them to the live
specs (archiving fix-dvpn-review-6953 and fix-coderabbit-findings-6953):
- bandwidth-type-scoping (new capability): reflects the PR #6976 design — the
controller's `managed_ticket_types` config field is the sole scoping mechanism
(empty set = no proactive restock; post-spend restock gated on the set). The
earlier fetcher-level veto approach was dropped in review and is no longer
described.
- dvpn-session / dvpn-tunnel / smol-core-stack: add the delivered requirements
(running-controller provisioning, opt-in restock, registration-not-cancellable-
past-spend, non-empty QUIC pin, in-tunnel top-up + events, set_mtu survival,
transport error handling, IP-literal handling, IPv4 bridge preference, bounded QUIC
connect, validated/truncation-aware DNS).
Also update docs/design/sdk/smol-dvpn/design.md: replace the removed `auto_restock`
reference with the managed-ticket-types model, and clarify that a registration
ticket-spend runs to completion (a cancel can't drop it mid-spend).
nym-sdk-session: fix two issues found reviewing the CodeRabbit fixes
- dvpn.rs `into_quic`: iterate all `quic_plain` transports (find_map) instead of
only inspecting the first — a malformed earlier transport (no routable address or
no identity pin) no longer shadows a valid later one. Also store the `id_pubkey`
(and SNI host) trimmed, so a whitespace-padded directory value still base64-decodes
at connect time. Adds unit tests.
- session.rs: keep the ticket-free LP `perform_handshake` cancellable. The previous
fix made the whole registration exchange non-cancellable; only the ticket-spending
`register_dvpn`/`handshake_and_register_dvpn` calls need that, so a stalled/
black-holed entry gateway would otherwise ignore the cancel token until the OS TCP
timeout. The handshake now races the cancel token again; only the spend runs bare.
smol-dvpn: address CodeRabbit review findings (#6953)
Correctness:
- nym-sdk-session: registration is no longer cancellable once a WireGuard ticket is
spent. Gateway selection (topology fetch) stays cancellable via
fetch_topology_cancellable, but register_single_hop/register_two_hop/
register_two_hop_quic run the LP registration exchange to completion without racing
the caller's cancel token — closing the lost-ticket window if a cancel fired after
the gateway processed the spend.
- smol-core dns: reject a truncated (TC-bit) response with a new DnsTruncated error
instead of returning the partial answers it carries (RFC 1035).
- smol-dvpn bridge: bound conn.open_bi() by the same CONNECT_TIMEOUT + cancellation
as the QUIC handshake, so a stalled bridge can't hang connect().
Robustness / cleanup:
- smol-core: remove the dead StackConfig::mtu field + with_mtu (Stack::new never read
it; the caller-supplied device is the MTU source of truth).
- nym-sdk-session: reject a blank id_pubkey so an unpinned bridge is never advertised
as QUIC-capable.
- live_bringup test: drop std::process::exit(0) (which could kill sibling tests);
the tokio::time::timeout around shutdown() already bounds teardown.
Docs:
- smol-dvpn README: reword the kill-switch/no-leaks line to a scoped guarantee (only
sockets opened through the tunnel are protected; ordinary sockets and host DNS
bypass it).
- design doc: clarify the bridge SNI must be an accepted cert alt-name (the "decoy"
is how ordinary it looks to DPI, not a free-form hostname), and document the
top-up headroom (top-up fires at a threshold above zero so the in-tunnel metadata
request still fits).
Adds a smol-core test asserting truncated responses are rejected.
smol-dvpn: address PR #6953 review feedback
Datapath / correctness:
- topup: dial the gateway metadata endpoint THROUGH the tunnel (hyper/1 over
TunnelConnector) instead of the host network, closing the IP-leak/broken-topup
blocker.
- tunnel: fix the set_mtu stack-swap race (biased select! with the swap branch
first, break on swap-channel closure) and serialise concurrent set_mtu behind a
mutex; classify transport recv errors (fatal bridge close exits; transient UDP
backs off) to end the busy-loop.
- connectors/smol-core: connect IP-literal hosts directly instead of routing them
through DNS (strip IPv6 brackets); short-circuit in resolve/tcp_connect_host.
- bridge: prefer IPv4 candidate addresses for v4-only clients.
smol-core DNS: random per-query id validated against the response id AND source,
recv loop until match/timeout, distinct DnsServerFailure for SERVFAIL/REFUSED,
and skip AAAA on a v4-only stack (without discarding resolved A records).
nym-sdk-session: run the bandwidth controller (single writer) and spend via its
request sender; scope acquisition to WireGuard ticket types; opt-in chain restock
(with_automatic_topups), gateway-side top-up on by default via a shipped
ProviderCredentialSource; external-provider escape hatch; hashed+zeroized
client id (no mnemonic clone); single topology fetch for two-hop; drop the mixnet
entry/exit role restriction for dVPN selection; split Nyxd/Api errors.
Performance: engine owns one reusable encrypt/decrypt scratch buffer and the
Direct receiver reuses its recv buffer (no per-packet 64 KiB alloc+zero).
Also: BandwidthEvent stream (tunnel.bandwidth_events()), typed WG keys in
PeerConfig, redacted Debug, target-locked mobile MTU, #[must_use] builders, and
doc corrections (design §10 top-up model; nym-bridges now on crates.io).
update nym-bridges dep to use version published to crates.io
permit usage of AGPL-3.0 dependencies
relax doc comment compile-time check restrictions
update the sample to no_run
fix mnemonic parsing
ignore tests when env is not set
chore(smol-dvpn): remove conformance spikes, normalize license headers
- delete the throwaway conformance spikes (tools/internal/dvpn-spikes) and
drop the now-dead path pointer from the archived OpenSpec design doc
- replace the copyright header in the three new crates (smol-core,
nym-sdk-session, nym-smol-dvpn) with the 2026 notice + Apache-2.0 SPDX
identifier, in both source files and Cargo.toml headers
- add a License section to the smol-dvpn README (Apache-2.0)
docs(smol-dvpn): move design docs, archive OpenSpec change, sync specs
Move sdk/rust/docs/nym-sdk-dvpn -> docs/design/sdk/smol-dvpn and rework the
docs now that the implementation is done:
- drop the exploration framing (open-questions.md, "no implementation yet",
spike A/B, D-numbered decision jargon) from the design docs, crate READMEs,
and source doc-comments
- reconcile the QUIC bridge design with reality: it now depends on the
git-pinned nym-bridges client rather than a byte-match reimpl
- add a builder-focused "What can I use nym-smol-dvpn for?" section to the
crate README, with a pointer to nym-credential-proxy for gifting zk-nyms
- archive the completed nym-sdk-dvpn OpenSpec change and sync its five delta
specs into openspec/specs/ (dvpn-tunnel, dvpn-quic-bridge, dvpn-tools,
dvpn-session, smol-core-stack)
- link the synced capability specs from the READMEs
chore(nym-sdk-session): migrate off deprecated NymApiClient
`nym_validator_client::client::NymApiClient` is deprecated in favour of
`nym_http_api_client::Client`. Use the client directly and reach the paged
`get_all_described_nodes_v2` via the `NymApiClientExt` trait, matching the
pattern already used in `nym-sdk`. Behaviour is preserved: the session's only
API call bypassed the wrapper's bincode flag, so `Client::new(url, timeout)` is
an exact substitute. Adds `nym-http-api-client` as a direct dependency.
refactor(smol-dvpn): use the nym-bridges QUIC client instead of a byte-match reimpl
The inline QUIC bridge client duplicated the reference cert-pinning verifier,
ALPN and dialing logic and had to be kept byte-for-byte in sync with the server
by hand. Delegate the connection to the canonical `nym_bridges` crate
(`transport::quic::{ClientOptions, transport_conn}`) so the client can never
drift from the bridge:
- Drop the local `IdentityBasedVerifier`, cert SPKI extraction and quinn client
config (~150 LOC); keep only the datapath framing (one `open_bi()` stream,
2-byte big-endian length prefix per WireGuard packet) plus the cancellation +
connect-timeout wrapper.
- `probe` and `connect` keep their signatures. `BridgeParams` now carries the
directory's base64 `id_pubkey` string and passes it straight to
`nym_bridges::ClientOptions`, dropping the `[u8; 32]` field and the
hand-rolled base64 codec that only existed to feed the old local verifier.
- This picks up the reference client's IPv6-first dial and dual-stack `[::]:0`
bind, replacing the previous IPv4-only path.
`nym-bridges` is not published to crates.io, so it is pulled as a pinned git
dependency; `cargo publish` of this crate is therefore blocked until it is
released. Drop the now-unused `ed25519-dalek` and `x509-parser` deps; add `bs58`
and `base64` as dev-deps for the conformance test, which now exercises framing +
ed25519 pinning against a real `nym_bridges` QUIC server.
chore(smol-dvpn): lower zcash-sync default blocks to 10_000
Faster default test run; still overridable with --blocks <N>. Docs + OpenSpec
updated to match.
perf(smol-core): widen TCP window + uncap device burst (10x tunnel throughput)
Bulk transfers through the tunnel (e.g. zcash-sync) were ~10x slower than direct
because of two self-inflicted caps in the smoltcp stack:
1. tokio-smoltcp's default per-socket TCP buffer is 8 KiB, and the rx buffer IS
the TCP receive window, so throughput was capped at 8 KiB / RTT — crippling on
a higher-RTT two-hop path. StackConfig now defaults to a 512 KiB window
(DEFAULT_TCP_BUFFER, tunable via with_tcp_buffer) applied through
NetConfig.buffer_size; smoltcp window scaling handles >64 KiB.
2. ChannelDevice set max_burst_size = Some(1), making smoltcp process one packet
per poll and serializing the datapath. Left unbounded (None).
Measured (release, zcash-sync --blocks 1000, sandbox two-hop):
before: tunnel 32 blocks/s, 87x slower than direct (debug baseline)
after: tunnel 3937 blocks/s, 1.61x slower than direct
smol-core unit tests still pass.
feat(nym-sdk-session): two-hop selection never reuses the entry gateway as exit
gateway::select gains an `exclude` parameter; register_two_hop(_quic) passes the
chosen entry's identity when selecting the exit, so random/country selection can
never resolve both hops to the same gateway. An exit spec that can only match the
entry gateway now fails with the new SessionError::SameGatewaySelected rather than
building a degenerate one-gateway "two-hop" tunnel.
Unit test added (excluded_identity_is_rejected); dvpn-session spec + session
README + tasks updated.
feat(smol-dvpn): configurable zcash-sync block count; document release builds
- zcash-sync: add --blocks <N> (shared example CLI) to choose how many compact
blocks to sync; default raised 1000 -> 100_000. Verified live that the count
flows into the GetBlockRange query (250 -> 250 blocks synced direct).
- Document building the examples with --release across the READMEs and example
usage docs: boringtun's userspace crypto is much slower in debug, which
dominates the through-tunnel timing (acute at the new 100k default).
- Sync OpenSpec (tasks 7.5/7.6, dvpn-tools spec) with the configurable count.
docs(nym-sdk-dvpn): sync READMEs + OpenSpec with QUIC/metadata/examples
READMEs:
- nym-sdk-session: document SessionConfig.dvpn_directory_url, the dVPN directory
(gateway monikers + QUIC entry selection), register_two_hop_quic / QuicBridge /
NoQuicGateway, and the GatewayInfo metadata now on HopConfig.
- smol-dvpn: note the MTU is runtime-adjustable via Tunnel::set_mtu().
OpenSpec (validates --strict):
- proposal: list the full example set (grpc, two-hop-ip, two-hop-quic, zcash-sync
+ shared CLI) and the directory/QUIC session capability.
- dvpn-session spec: add requirements for optional dVPN directory metadata and
QUIC-bridge entry gateway selection.
- dvpn-tools spec: add requirements for the gRPC, IP-relocation, Zcash-sync
examples and the shared selection CLI.
- tasks: record 3.9/3.10 (directory + QUIC selection) and 7.3–7.6 (new examples
+ CLI), all implemented.
docs(smol-dvpn): configurable example CLI + detailed README
- Add a shared example CLI (examples/common): --one-hop/--two-hop,
--entry/--exit/--gateway <SPEC> (random | ISO country | base58 identity),
and --quic, with validation (QUIC is two-hop entry only) and --help. The
two-hop-ip, two-hop-quic, and zcash-sync examples now honour it via a common
register()/build_tunnel() dispatch (single- vs two-hop, Direct vs QUIC entry).
- Expand the README Examples section: per-example table, full option/SPEC
reference, invocation recipes, and a mermaid sequence diagram of the zcash-sync
flow (ticketbooks -> two-hop registration -> gRPC compact-block sync -> teardown).
- Add a Developers section documenting how to point the examples at sandbox
(source envs/sandbox.env + funded MNEMONIC; DVPN_DIRECTORY_URL / RUST_LOG).
Verified: cargo clippy --examples clean; --help/validation paths; and a live
default two-hop-ip run still relocates the public IP to the exit gateway.
feat(sdk): dVPN example programs + QUIC-bridge gateway selection
nym-sdk-session:
- Add a lightweight dVPN gateway-directory client (dvpn.rs). QUIC bridge params
live in the VPN/dvpn directory, not the nym-api described-nodes, so the session
fetches the directory (best-effort) to enrich gateway metadata and enable QUIC
entry selection. SessionConfig gains `dvpn_directory_url: Option<String>`.
- GatewayInfo gains `name` (directory moniker) and already carried node_id/country/ip;
HopConfig gains `bridge: Option<QuicBridge>` (set only for a QUIC entry hop).
- New `QuicBridge` (addresses/SNI/base64 id_pubkey) and
`register_two_hop_quic(entry, exit)`, which requires a QUIC-capable entry
(honoring the GatewaySpec) and fails with `NoQuicGateway` otherwise.
register_single_hop / register_two_hop are unchanged (bridge = None).
smol-dvpn examples:
- two-hop-ip: prove a two-hop tunnel relocates your public IP (ipinfo.io direct
vs. through the tunnel); `--quic` requires a QUIC entry.
- two-hop-quic: two-hop tunnel whose entry leg is carried over a QUIC bridge.
- zcash-sync: time syncing the last 1000 Zcash compact blocks from a public
lightwalletd (gRPC-over-TLS) directly vs. through the tunnel; `--quic` supported.
- Shared examples/common/ (session/tunnel setup, hop->PeerConfig, gateway
printing, HTTPS-over-any-stream ipinfo fetch, a TLS-wrapping tower connector
for gRPC-over-TLS through the tunnel, QuicBridge->BridgeParams mapping).
- Thread the new SessionConfig field through the existing examples + live test.
Verified against sandbox: two-hop-quic selects the designated QUIC entry gateway
from the directory, populates its moniker, extracts bridge params and dials the
correct IPv4 endpoint; the sandbox QUIC bridge endpoint is currently unreachable
(ICMP-alive host, QUIC/UDP:4443 times out) so the live handshake is infra-blocked,
but the full selection/param/datapath path is exercised. cargo build/clippy clean
across both crates (examples + tests).
feat(smol-dvpn): runtime-adjustable MTU via WG-preserving stack rebuild (task 4.7)
Add Tunnel::set_mtu(): rebuilds the smol-core interface at the new MTU while
keeping the WireGuard engine/session alive (no re-handshake), by swapping the
datapath's stack-side channels through a control channel. The stack is held
behind a swappable handle (Arc<RwLock<Arc<Stack>>>) so the tokio socket
surfaces and the tower connector keep working across the swap.
tokio-smoltcp 0.5 fixes the interface MTU at construction (BufferDevice caches
capabilities), so this rebuilds the interface rather than a fully seamless
in-place resize; open sockets at the instant of change are reset while the
tunnel stays connected.
Verified live against sandbox: resolve -> set_mtu(MOBILE) -> resolve still
flows through the same WireGuard session (live_bringup.rs).
Completes 42/42 tasks for nym-sdk-dvpn.
feat(smol-dvpn): tonic gRPC example through the tunnel (task 5.4)
Add examples/smol-dvpn-grpc.rs: brings up a single-hop tunnel, builds a tonic
Channel over tunnel.connector() (so gRPC/HTTP2 flows inside the tunnel), and
issues a gRPC Health.Check using tonic-health's ready-made client (no custom
proto/build.rs). Compiles; run against a live gRPC health service like the
other live examples.
Also annotate task 4.7 as partial: configurable MtuConfig + reference
DESKTOP/MOBILE defaults are implemented, but MTU resize-while-up is blocked by
tokio-smoltcp 0.5 (BufferDevice caches the interface MTU at construction).
41/42 tasks; 4.7 partial pending an upstream tokio-smoltcp capability.
feat(sdk): pure-Rust userspace dVPN datapath (smol-core, nym-sdk-session, smol-dvpn)
Implements the nym-sdk-dvpn OpenSpec change: a reusable, pure-Rust building
block for a paid 1-/2-hop WireGuard dVPN tunnel that pushes ordinary tokio
traffic (TCP/UDP, tonic/hyper) without root or an OS tun device.
New crates:
- common/smol-core: transport-agnostic userspace smoltcp stack (IP-packet
Vec<u8> stream -> TcpStream/UdpSocket + tunnel DNS resolver). smolmix is
refactored onto it with its public API preserved.
- sdk/rust/nym-sdk-session: provisioning facade (mnemonic -> zk-nym ticketbook
issuance + persistent storage, gateway selection by identity/country/random,
single- and two-hop LP registration), cancellation-aware.
- sdk/rust/smol-dvpn (nym-smol-dvpn): boringtun userspace datapath on smol-core;
WgPacketTransport seam (Direct UDP + inline QUIC bridge, ALPN hq-29 +
ed25519-SPKI pinning + 2-byte framing); single-hop and nested two-hop
encapsulation; timer pump; lifecycle + CancellationToken; DNS-in-tunnel;
tonic/hyper connectors; background bandwidth top-up; smol-dvpn-config /
smol-dvpn-topup example CLIs.
boringtun/quinn/quinn-proto are declared crate-local (not the workspace table).
Verified: conformance spikes (two-hop nesting, QUIC framing/pinning) pass;
smol-core TCP/UDP/DNS + smolmix API-lock + bridge conformance + session unit
tests pass; and the full stack is proven end-to-end against live sandbox
gateways (single-hop and two-hop both resolve through the tunnel) via
sdk/rust/smol-dvpn/tests/live_bringup.rs.
40/42 tasks complete. Remaining: runtime MTU resize-while-up (tokio-smoltcp 0.5
cannot resize a live interface) and a full tonic gRPC round-trip test (needs a
proto/generated client + live service).
jstuczyn
force-pushed
the
feature/nym-sdk-dvpn
branch
from
July 25, 2026 15:06
11cddec to
22aa2d5
Compare
jstuczyn
approved these changes
Jul 25, 2026
jstuczyn
pushed a commit
that referenced
this pull request
Jul 25, 2026
remove garbage data
post rebasing fixes
openspec: archive dvpn-registration-reuse after mainnet validation
Mainnet validation passed (2026-07-24): consecutive runs against the same
gateways reused the cached registrations with zero ticket spend, post-rename.
Deltas merged into live specs: dvpn-session gains 'Gateway registration
persistence and reuse', dvpn-tunnel gains 'Awaitable per-hop session
establishment'. The same run completes smoldvpn-rename task 4.1 (live run
under the new name), leaving that change fully implemented and ready to
archive whenever convenient.
smoldvpn: move to repo root and rename from nym-smol-dvpn
Implements openspec change smoldvpn-rename: the dVPN datapath crate is now
consistent with its sibling smolmix — sdk/rust/smol-dvpn -> smoldvpn/ (git
mv, history follows), package + lib renamed nym-smol-dvpn/nym_smol_dvpn ->
smoldvpn. Never published to crates.io, so the rename precedes first
publication (no legacy name to manage).
Rename mapping for archaeology:
nym-smol-dvpn -> smoldvpn (package, -p flags, prose)
nym_smol_dvpn -> smoldvpn (use paths, RUST_LOG targets)
smol-dvpn-* -> smoldvpn-* (example binaries + their data/ dirs)
- Examples smol-dvpn-{config,grpc,topup} renamed smoldvpn-*; the shared
examples (zcash-sync, two-hop-*, quic-probe) and their data dirs are
name-stable. README gains a migration note (RUST_LOG target change;
rename local data/smol-dvpn-* dirs to keep credentials).
- Workspace member updated (sorted next to smolmix); Cargo.lock follows.
- Prose updated: crate README, common/smol-core README, nym-sdk-session
README + doc comments; docs/design/sdk/smol-dvpn -> docs/design/smoldvpn.
- Live specs dvpn-tunnel/dvpn-tools/dvpn-quic-bridge updated in place; the
change's MODIFIED deltas carry identical text so archiving is idempotent.
Historical openspec change artifacts deliberately untouched.
Verified: straggler grep clean outside historical artifacts; full test
suites green for smoldvpn + nym-sdk-session; clippy clean incl. examples;
`cargo run -p smoldvpn --example zcash-sync -- --help` works; examples'
default log filter now spells smoldvpn=info,boringtun=info.
openspec: archive dvpn-signer-failure-tests + dvpn-signer-fault-http-harness
Both changes fully implemented and validated (the signer-tolerance change
including its live mainnet validation). Deltas merged into live specs:
dvpn-session gains the 'Tolerance to unresponsive ecash signers'
requirement; the new dvpn-signer-fault-harness capability spec is created
(harness, multi-API characterization, threshold probe).
dvpn-registration-reuse stays active pending its manual mainnet validation
(task 5.3). Also records the smoldvpn-rename D6 resolution: nym-smol-dvpn
was never published to crates.io, so the rename precedes first publication.
docs: ignore docs/scratch (working notes stay uncommitted)
nym-sdk-session + smol-dvpn: address code-quality review findings
Thermo-nuclear review of the branch's session work; behavior-preserving
restructurings, all suites green (47 tests) and clippy clean:
- session.rs had crossed the 1k-line boundary (747 -> 1145): split
SessionConfig + RestockPolicy into config.rs and the unit tests into
session_tests.rs (#[path] module) — session.rs is back to 792. In
passing this fixed a latent export bug: RestockPolicy was reachable only
as an unnameable public field type; it is now re-exported.
- The persist-then-assemble HopConfig tail was copy-pasted across all three
registration sites: extracted as Session::finalize_hop (testable
identity+GatewayInfo signature, same as cached_hop).
- The reg-cache lock/poison-recovery incantation was triplicated:
collapsed into Session::with_cache.
- datapath() had grown to 8 args behind a clippy::too_many_arguments allow
while two of them already formed the named StackChannels type: pass the
named pair, allow removed.
- clone_cfg's serde round-trip now documents WHY deriving Clone upstream is
wrong (PresharedKey is ZeroizeOnDrop secret material that deliberately
omits Clone), not just that the derive is missing.
Reviewed and left as-is (justified): TimeoutFetcher decorator shape,
opt-out-as-Option cache design, the examples' per-file self-containment
duplication, engine progress flags.
nym-sdk-session: HTTP-level signer-fault harness (tests only)
Implements openspec change dvpn-signer-fault-http-harness: drive the REAL
NyxdGlobalDataFetcher — real DKG discovery, real HTTP client, real
try-each-API loop — against local mock nym-apis whose ecash routes are
independently scriptable as healthy / slow / erroring / hanging (accept the
connection, never respond: the exact client-visible behavior observed
against mainnet). Dev-only; no common/ production code touched, no new
mock-server dependency (the Hang mode is trivial on a raw TcpListener).
- tests/support/http_harness.rs: MockNymApi with per-route FaultMode and
runtime re-scripting; teardown releases parked Hang connections.
- tests/support/fake_dkg.rs: single-method DkgQueryClient double answering
vk-share + threshold queries from fabricated ContractVKShares announcing
the mock URLs (owners must be real-format bech32 — discovery parses them
into cosmrs AccountIds). Fails loud on unexpected queries.
- tests/support: TestEcash gains vk-share bs58 accessors and annotated
expiration-date-signature fixtures (parseable by the real client).
- tests/signer_fault_http.rs (6 tests, ~2.6s): fixture sanity through real
discovery; characterize_* reproduction of the mainnet hang on the raw
stack; TimeoutFetcher guard measured end-to-end; multi-API bounded
retries; threshold probe both sides (K >= threshold: partials retrievable
during a full aggregated outage — the client-side aggregation go-signal;
K < threshold: insufficiency reported).
Findings recorded in the change's design.md, notably: the SDK's per-fetch
bound wraps the fetcher's whole try-each-API loop, so per-API fallback needs
a per-request bound inside query_random_apis_until_success
(common/bandwidth-fetcher) — the concrete item to hand the ecash owners.
cargo test -p nym-sdk-session: 40 tests green; clippy --tests clean.
nym-sdk-session + smol-dvpn: reuse gateway registrations across runs
Every registration used to generate fresh WireGuard keys and spend one
zk-nym ticket per hop, even against a gateway registered with moments
earlier — abandoning the previous peer's remaining ~477 MiB allowance on
the gateway (observed on mainnet: two identical runs = 4 tickets for
~500 MB used). Implements openspec change dvpn-registration-reuse; no
common/ production code touched.
- registration_cache (new): registrations.json in the session data dir —
client WG key + gateway-returned WireguardConfiguration keyed by
(network, gateway, role); atomic writes, 0600 on unix, 30-day hygiene
pruning (validity is proven by use, never assumed). Contains private
keys/PSKs: same sensitivity class as the adjacent creds.db.
- Session: register_single_hop/register_two_hop(_quic) consult the cache
after gateway selection — full hit returns with NO gateway exchange and
NO ticket spent ('reusing cached registration … — no ticket spent' at
info); partial two-hop hits register (and provision ticketbooks for)
only the missing hop. Fresh registrations persist immediately after
register_dvpn. New: invalidate_registration(gateway, role) for the
fallback path; SessionConfig::reuse_registrations (default on; opt-out
documents the peer-linkability trade-off and neither reads nor writes
the cache).
- smol-dvpn: Tunnel::await_established(timeout) — per-hop establishment
published from the engine's handshake tracking via a watch channel;
NotEstablished reports entry/exit status so callers invalidate exactly
the dead hop.
- Examples: common::connect() implements register → build →
await_established(15s) → invalidate-failed-hops → re-register-once;
adopted by zcash-sync/two-hop-ip/two-hop-quic (and smol-dvpn-grpc
inline), replacing the blind 10×3s ipinfo warmup loops with an
establishment gate + short display probe.
- Tests: 8 cache unit tests; 4 offline session-level reuse tests
(counting provider proves zero-spend cache hits, opt-out, invalidation,
restart survival); 4 loopback await_established tests with a real
boringtun responder (incl. exit-only failure attribution). The
await_established tests run on a manual runtime with shutdown_timeout —
the smol-core stack parks blocking-pool workers that outlive the
tunnel, which would hang #[tokio::test] forever in Runtime::drop (the
examples sidestep the same via std::process::exit(0)).
cargo test: 28 (session lib) + 7 (signer_failure) + 4 (await_established)
+ crate suites all green; clippy clean on lib/tests/examples for both.
Remaining: manual mainnet validation (task 5.3).
smol-dvpn: make WireGuard handshake progress observable at info level
Debugging a mainnet two-hop tunnel that registered fine but passed no data
(DNS-in-tunnel timeouts) showed the datapath is a black box between
'datapath starting' and the first application traffic: handshake success is
never logged at any level, and decapsulate errors (e.g. a rejected handshake
response) were silently swallowed.
- decap(): log TunnResult::Err at warn instead of dropping it silently.
- WgEngine: one-shot info markers — 'first datagram received from the entry
transport', 'entry-hop wireguard session established', 'exit-hop wireguard
session established' (single-hop: 'wireguard session established'), driven
off boringtun's stats().0 flipping to Some. A single default-log run now
localizes a dead tunnel: no inbound datagrams at all vs. entry handshake
never completing vs. exit handshake never completing vs. established but
no upstream data.
- 'dVPN tunnel datapath starting' now logs the entry/exit WG endpoints, so
the exact UDP destinations being dialed appear in the log.
- Fix the pre-existing clippy while-let warning in the decap drain loop.
smol-dvpn: close the session credential store cleanly in the examples
The example CLIs never called Session::shutdown(): error paths dropped the
session without awaiting the bandwidth controller's cleanup (surfacing
'SqlitePoolGuard dropped without explicit close()' for creds.db), and the
happy paths called std::process::exit(0), which skips destructors entirely —
leaving the sqlite WAL uncheckpointed (the multi-hundred-KB creds.db-wal
observed after mainnet runs).
Wrap each session-using flow in an async block and always await
session.shutdown() afterwards — success or failure — before printing the
final verdict / exiting. The shutdown awaits the controller task, which
closes the fetcher's pending-request store and the credential store
(WAL checkpoint); stored tickets are retained. Applies to all six
session-building examples: two-hop-ip, two-hop-quic, zcash-sync,
smol-dvpn-config, smol-dvpn-grpc, smol-dvpn-topup.
openspec: record successful mainnet validation of signer-failure tolerance
zcash-sync against mainnet (2026-07-23): provisioning completed in ~1m47s
instead of hanging forever; both SignerTimeout warnings fired at the 15s
bound; both wireguard ticketbooks (entry + exit, 50 tickets each) persisted
to data/zcash-sync/mainnet/creds.db. Ticks the last open task (5.4).
nym-sdk-session: tolerate unresponsive ecash signers (no hang, no lost funds)
Mainnet's aggregated-expiration-date-signatures endpoint accepts connections
and never responds (offline distributed signers — a permanent operating
condition). With no request timeout anywhere in the fetch stack, the
bandwidth controller's store_ticketbook wedged mid-persist on its run loop:
provisioning hung forever, the freshly issued (paid-for) ticketbook was never
stored, and every retry re-deposited NYM. SDK-confined fix per openspec
change dvpn-signer-failure-tests; no common/ production code touched.
- TimeoutFetcher decorator (new fetcher.rs): bounds the three read-only
global-signing-data fetches at 15s per call, surfacing a SignerTimeout
FetcherError naming unresponsive signers. fetch_ticketbooks (the deposit)
is deliberately NOT timed. Wired around NyxdCredentialFetcher in
Session::spawn_controller. A bounded failure is recoverable: the
controller's best-effort store path persists the ticketbook anyway and
readiness resolves from stored stock; missing signatures back-fill later.
- ensure_ticketbooks gains an overall 5-minute budget mapping to a new
SessionError::ProvisioningTimeout (defense in depth; cancellation race and
funds-safety unchanged). Note: SessionError is not #[non_exhaustive], so
exhaustive matchers gain one arm.
- Tier 1 tests (virtual clock, in fetcher.rs): hang→bounded error for all
three fetches, under/over-threshold boundary, error passthrough, and
issuance provably unbounded by the decorator.
- Tier 2 tests (tests/signer_failure.rs + tests/support): the real
BandwidthController over ephemeral storage with a FlakyFetcher reproducing
the mainnet outage, and a locally fabricated 2-of-3 threshold-signed
ticketbook (no chain, no network, no funds). Proves: the pre-fix wedge and
empty store (the exact observed creds.db state), post-fix persistence +
resolved readiness, zero re-purchase on retry, and fail-fast spend during
the outage.
- openspec: dvpn-signer-failure-tests artifacts (tasks ticked; only optional
manual mainnet validation remains) and the follow-up
dvpn-signer-fault-http-harness proposal (HTTP-level fault harness, may not
be implemented).
cargo test -p nym-sdk-session: 23 passed. clippy --tests: clean.
resolve .gitignore merge conflict
Keep both sides of the conflict: tmp/ + the operator-tools nodes.csv
(HEAD) and *storybook.log (NYM-1199).
smol-dvpn: log example output via tracing, restructure example data dirs
Wire a tracing subscriber into every example so the crate's logs (and the
examples' own narration) are visible out of the box, and convert the examples
from println! to tracing macros.
- Add tracing-subscriber (env-filter) dev-dep; each example installs a
stderr fmt subscriber honouring RUST_LOG. Default filter (RUST_LOG unset)
is the running example plus nym_smol_dvpn and boringtun at info; the
example crate root is derived from module_path! so its own info! shows.
- Convert progress/result output to info!, retries/probe failures to warn!,
and fatal main errors to error!. Logs go to stderr; stdout stays reserved
for genuine deliverables (the smol-dvpn-config WireGuard config and --help).
- Log the per-example credential store / data directory at startup.
- Restructure example data output to data/<example>/<network> (e.g.
data/zcash-sync/sandbox) instead of flat *-data dirs; create it if missing.
- Ignore /data/ in the crate and repo-root .gitignore.
- Update README developer notes for the logging behaviour and data layout.
use nym native types on 'PeerConfig' and remove Clone derivation
use concrete inner error for SessionError::Api
fix deps of nym-registration-client
ibid for 'nym-authenticator-client'
mark 'nym-registration-client' as publishable
clippy
openspec: sync specs + design doc with delivered dVPN design
Reconcile the OpenSpec deltas with what actually shipped and apply them to the live
specs (archiving fix-dvpn-review-6953 and fix-coderabbit-findings-6953):
- bandwidth-type-scoping (new capability): reflects the PR #6976 design — the
controller's `managed_ticket_types` config field is the sole scoping mechanism
(empty set = no proactive restock; post-spend restock gated on the set). The
earlier fetcher-level veto approach was dropped in review and is no longer
described.
- dvpn-session / dvpn-tunnel / smol-core-stack: add the delivered requirements
(running-controller provisioning, opt-in restock, registration-not-cancellable-
past-spend, non-empty QUIC pin, in-tunnel top-up + events, set_mtu survival,
transport error handling, IP-literal handling, IPv4 bridge preference, bounded QUIC
connect, validated/truncation-aware DNS).
Also update docs/design/sdk/smol-dvpn/design.md: replace the removed `auto_restock`
reference with the managed-ticket-types model, and clarify that a registration
ticket-spend runs to completion (a cancel can't drop it mid-spend).
nym-sdk-session: fix two issues found reviewing the CodeRabbit fixes
- dvpn.rs `into_quic`: iterate all `quic_plain` transports (find_map) instead of
only inspecting the first — a malformed earlier transport (no routable address or
no identity pin) no longer shadows a valid later one. Also store the `id_pubkey`
(and SNI host) trimmed, so a whitespace-padded directory value still base64-decodes
at connect time. Adds unit tests.
- session.rs: keep the ticket-free LP `perform_handshake` cancellable. The previous
fix made the whole registration exchange non-cancellable; only the ticket-spending
`register_dvpn`/`handshake_and_register_dvpn` calls need that, so a stalled/
black-holed entry gateway would otherwise ignore the cancel token until the OS TCP
timeout. The handshake now races the cancel token again; only the spend runs bare.
smol-dvpn: address CodeRabbit review findings (#6953)
Correctness:
- nym-sdk-session: registration is no longer cancellable once a WireGuard ticket is
spent. Gateway selection (topology fetch) stays cancellable via
fetch_topology_cancellable, but register_single_hop/register_two_hop/
register_two_hop_quic run the LP registration exchange to completion without racing
the caller's cancel token — closing the lost-ticket window if a cancel fired after
the gateway processed the spend.
- smol-core dns: reject a truncated (TC-bit) response with a new DnsTruncated error
instead of returning the partial answers it carries (RFC 1035).
- smol-dvpn bridge: bound conn.open_bi() by the same CONNECT_TIMEOUT + cancellation
as the QUIC handshake, so a stalled bridge can't hang connect().
Robustness / cleanup:
- smol-core: remove the dead StackConfig::mtu field + with_mtu (Stack::new never read
it; the caller-supplied device is the MTU source of truth).
- nym-sdk-session: reject a blank id_pubkey so an unpinned bridge is never advertised
as QUIC-capable.
- live_bringup test: drop std::process::exit(0) (which could kill sibling tests);
the tokio::time::timeout around shutdown() already bounds teardown.
Docs:
- smol-dvpn README: reword the kill-switch/no-leaks line to a scoped guarantee (only
sockets opened through the tunnel are protected; ordinary sockets and host DNS
bypass it).
- design doc: clarify the bridge SNI must be an accepted cert alt-name (the "decoy"
is how ordinary it looks to DPI, not a free-form hostname), and document the
top-up headroom (top-up fires at a threshold above zero so the in-tunnel metadata
request still fits).
Adds a smol-core test asserting truncated responses are rejected.
smol-dvpn: address PR #6953 review feedback
Datapath / correctness:
- topup: dial the gateway metadata endpoint THROUGH the tunnel (hyper/1 over
TunnelConnector) instead of the host network, closing the IP-leak/broken-topup
blocker.
- tunnel: fix the set_mtu stack-swap race (biased select! with the swap branch
first, break on swap-channel closure) and serialise concurrent set_mtu behind a
mutex; classify transport recv errors (fatal bridge close exits; transient UDP
backs off) to end the busy-loop.
- connectors/smol-core: connect IP-literal hosts directly instead of routing them
through DNS (strip IPv6 brackets); short-circuit in resolve/tcp_connect_host.
- bridge: prefer IPv4 candidate addresses for v4-only clients.
smol-core DNS: random per-query id validated against the response id AND source,
recv loop until match/timeout, distinct DnsServerFailure for SERVFAIL/REFUSED,
and skip AAAA on a v4-only stack (without discarding resolved A records).
nym-sdk-session: run the bandwidth controller (single writer) and spend via its
request sender; scope acquisition to WireGuard ticket types; opt-in chain restock
(with_automatic_topups), gateway-side top-up on by default via a shipped
ProviderCredentialSource; external-provider escape hatch; hashed+zeroized
client id (no mnemonic clone); single topology fetch for two-hop; drop the mixnet
entry/exit role restriction for dVPN selection; split Nyxd/Api errors.
Performance: engine owns one reusable encrypt/decrypt scratch buffer and the
Direct receiver reuses its recv buffer (no per-packet 64 KiB alloc+zero).
Also: BandwidthEvent stream (tunnel.bandwidth_events()), typed WG keys in
PeerConfig, redacted Debug, target-locked mobile MTU, #[must_use] builders, and
doc corrections (design §10 top-up model; nym-bridges now on crates.io).
update nym-bridges dep to use version published to crates.io
permit usage of AGPL-3.0 dependencies
relax doc comment compile-time check restrictions
update the sample to no_run
fix mnemonic parsing
ignore tests when env is not set
chore(smol-dvpn): remove conformance spikes, normalize license headers
- delete the throwaway conformance spikes (tools/internal/dvpn-spikes) and
drop the now-dead path pointer from the archived OpenSpec design doc
- replace the copyright header in the three new crates (smol-core,
nym-sdk-session, nym-smol-dvpn) with the 2026 notice + Apache-2.0 SPDX
identifier, in both source files and Cargo.toml headers
- add a License section to the smol-dvpn README (Apache-2.0)
docs(smol-dvpn): move design docs, archive OpenSpec change, sync specs
Move sdk/rust/docs/nym-sdk-dvpn -> docs/design/sdk/smol-dvpn and rework the
docs now that the implementation is done:
- drop the exploration framing (open-questions.md, "no implementation yet",
spike A/B, D-numbered decision jargon) from the design docs, crate READMEs,
and source doc-comments
- reconcile the QUIC bridge design with reality: it now depends on the
git-pinned nym-bridges client rather than a byte-match reimpl
- add a builder-focused "What can I use nym-smol-dvpn for?" section to the
crate README, with a pointer to nym-credential-proxy for gifting zk-nyms
- archive the completed nym-sdk-dvpn OpenSpec change and sync its five delta
specs into openspec/specs/ (dvpn-tunnel, dvpn-quic-bridge, dvpn-tools,
dvpn-session, smol-core-stack)
- link the synced capability specs from the READMEs
chore(nym-sdk-session): migrate off deprecated NymApiClient
`nym_validator_client::client::NymApiClient` is deprecated in favour of
`nym_http_api_client::Client`. Use the client directly and reach the paged
`get_all_described_nodes_v2` via the `NymApiClientExt` trait, matching the
pattern already used in `nym-sdk`. Behaviour is preserved: the session's only
API call bypassed the wrapper's bincode flag, so `Client::new(url, timeout)` is
an exact substitute. Adds `nym-http-api-client` as a direct dependency.
refactor(smol-dvpn): use the nym-bridges QUIC client instead of a byte-match reimpl
The inline QUIC bridge client duplicated the reference cert-pinning verifier,
ALPN and dialing logic and had to be kept byte-for-byte in sync with the server
by hand. Delegate the connection to the canonical `nym_bridges` crate
(`transport::quic::{ClientOptions, transport_conn}`) so the client can never
drift from the bridge:
- Drop the local `IdentityBasedVerifier`, cert SPKI extraction and quinn client
config (~150 LOC); keep only the datapath framing (one `open_bi()` stream,
2-byte big-endian length prefix per WireGuard packet) plus the cancellation +
connect-timeout wrapper.
- `probe` and `connect` keep their signatures. `BridgeParams` now carries the
directory's base64 `id_pubkey` string and passes it straight to
`nym_bridges::ClientOptions`, dropping the `[u8; 32]` field and the
hand-rolled base64 codec that only existed to feed the old local verifier.
- This picks up the reference client's IPv6-first dial and dual-stack `[::]:0`
bind, replacing the previous IPv4-only path.
`nym-bridges` is not published to crates.io, so it is pulled as a pinned git
dependency; `cargo publish` of this crate is therefore blocked until it is
released. Drop the now-unused `ed25519-dalek` and `x509-parser` deps; add `bs58`
and `base64` as dev-deps for the conformance test, which now exercises framing +
ed25519 pinning against a real `nym_bridges` QUIC server.
chore(smol-dvpn): lower zcash-sync default blocks to 10_000
Faster default test run; still overridable with --blocks <N>. Docs + OpenSpec
updated to match.
perf(smol-core): widen TCP window + uncap device burst (10x tunnel throughput)
Bulk transfers through the tunnel (e.g. zcash-sync) were ~10x slower than direct
because of two self-inflicted caps in the smoltcp stack:
1. tokio-smoltcp's default per-socket TCP buffer is 8 KiB, and the rx buffer IS
the TCP receive window, so throughput was capped at 8 KiB / RTT — crippling on
a higher-RTT two-hop path. StackConfig now defaults to a 512 KiB window
(DEFAULT_TCP_BUFFER, tunable via with_tcp_buffer) applied through
NetConfig.buffer_size; smoltcp window scaling handles >64 KiB.
2. ChannelDevice set max_burst_size = Some(1), making smoltcp process one packet
per poll and serializing the datapath. Left unbounded (None).
Measured (release, zcash-sync --blocks 1000, sandbox two-hop):
before: tunnel 32 blocks/s, 87x slower than direct (debug baseline)
after: tunnel 3937 blocks/s, 1.61x slower than direct
smol-core unit tests still pass.
feat(nym-sdk-session): two-hop selection never reuses the entry gateway as exit
gateway::select gains an `exclude` parameter; register_two_hop(_quic) passes the
chosen entry's identity when selecting the exit, so random/country selection can
never resolve both hops to the same gateway. An exit spec that can only match the
entry gateway now fails with the new SessionError::SameGatewaySelected rather than
building a degenerate one-gateway "two-hop" tunnel.
Unit test added (excluded_identity_is_rejected); dvpn-session spec + session
README + tasks updated.
feat(smol-dvpn): configurable zcash-sync block count; document release builds
- zcash-sync: add --blocks <N> (shared example CLI) to choose how many compact
blocks to sync; default raised 1000 -> 100_000. Verified live that the count
flows into the GetBlockRange query (250 -> 250 blocks synced direct).
- Document building the examples with --release across the READMEs and example
usage docs: boringtun's userspace crypto is much slower in debug, which
dominates the through-tunnel timing (acute at the new 100k default).
- Sync OpenSpec (tasks 7.5/7.6, dvpn-tools spec) with the configurable count.
docs(nym-sdk-dvpn): sync READMEs + OpenSpec with QUIC/metadata/examples
READMEs:
- nym-sdk-session: document SessionConfig.dvpn_directory_url, the dVPN directory
(gateway monikers + QUIC entry selection), register_two_hop_quic / QuicBridge /
NoQuicGateway, and the GatewayInfo metadata now on HopConfig.
- smol-dvpn: note the MTU is runtime-adjustable via Tunnel::set_mtu().
OpenSpec (validates --strict):
- proposal: list the full example set (grpc, two-hop-ip, two-hop-quic, zcash-sync
+ shared CLI) and the directory/QUIC session capability.
- dvpn-session spec: add requirements for optional dVPN directory metadata and
QUIC-bridge entry gateway selection.
- dvpn-tools spec: add requirements for the gRPC, IP-relocation, Zcash-sync
examples and the shared selection CLI.
- tasks: record 3.9/3.10 (directory + QUIC selection) and 7.3–7.6 (new examples
+ CLI), all implemented.
docs(smol-dvpn): configurable example CLI + detailed README
- Add a shared example CLI (examples/common): --one-hop/--two-hop,
--entry/--exit/--gateway <SPEC> (random | ISO country | base58 identity),
and --quic, with validation (QUIC is two-hop entry only) and --help. The
two-hop-ip, two-hop-quic, and zcash-sync examples now honour it via a common
register()/build_tunnel() dispatch (single- vs two-hop, Direct vs QUIC entry).
- Expand the README Examples section: per-example table, full option/SPEC
reference, invocation recipes, and a mermaid sequence diagram of the zcash-sync
flow (ticketbooks -> two-hop registration -> gRPC compact-block sync -> teardown).
- Add a Developers section documenting how to point the examples at sandbox
(source envs/sandbox.env + funded MNEMONIC; DVPN_DIRECTORY_URL / RUST_LOG).
Verified: cargo clippy --examples clean; --help/validation paths; and a live
default two-hop-ip run still relocates the public IP to the exit gateway.
feat(sdk): dVPN example programs + QUIC-bridge gateway selection
nym-sdk-session:
- Add a lightweight dVPN gateway-directory client (dvpn.rs). QUIC bridge params
live in the VPN/dvpn directory, not the nym-api described-nodes, so the session
fetches the directory (best-effort) to enrich gateway metadata and enable QUIC
entry selection. SessionConfig gains `dvpn_directory_url: Option<String>`.
- GatewayInfo gains `name` (directory moniker) and already carried node_id/country/ip;
HopConfig gains `bridge: Option<QuicBridge>` (set only for a QUIC entry hop).
- New `QuicBridge` (addresses/SNI/base64 id_pubkey) and
`register_two_hop_quic(entry, exit)`, which requires a QUIC-capable entry
(honoring the GatewaySpec) and fails with `NoQuicGateway` otherwise.
register_single_hop / register_two_hop are unchanged (bridge = None).
smol-dvpn examples:
- two-hop-ip: prove a two-hop tunnel relocates your public IP (ipinfo.io direct
vs. through the tunnel); `--quic` requires a QUIC entry.
- two-hop-quic: two-hop tunnel whose entry leg is carried over a QUIC bridge.
- zcash-sync: time syncing the last 1000 Zcash compact blocks from a public
lightwalletd (gRPC-over-TLS) directly vs. through the tunnel; `--quic` supported.
- Shared examples/common/ (session/tunnel setup, hop->PeerConfig, gateway
printing, HTTPS-over-any-stream ipinfo fetch, a TLS-wrapping tower connector
for gRPC-over-TLS through the tunnel, QuicBridge->BridgeParams mapping).
- Thread the new SessionConfig field through the existing examples + live test.
Verified against sandbox: two-hop-quic selects the designated QUIC entry gateway
from the directory, populates its moniker, extracts bridge params and dials the
correct IPv4 endpoint; the sandbox QUIC bridge endpoint is currently unreachable
(ICMP-alive host, QUIC/UDP:4443 times out) so the live handshake is infra-blocked,
but the full selection/param/datapath path is exercised. cargo build/clippy clean
across both crates (examples + tests).
feat(smol-dvpn): runtime-adjustable MTU via WG-preserving stack rebuild (task 4.7)
Add Tunnel::set_mtu(): rebuilds the smol-core interface at the new MTU while
keeping the WireGuard engine/session alive (no re-handshake), by swapping the
datapath's stack-side channels through a control channel. The stack is held
behind a swappable handle (Arc<RwLock<Arc<Stack>>>) so the tokio socket
surfaces and the tower connector keep working across the swap.
tokio-smoltcp 0.5 fixes the interface MTU at construction (BufferDevice caches
capabilities), so this rebuilds the interface rather than a fully seamless
in-place resize; open sockets at the instant of change are reset while the
tunnel stays connected.
Verified live against sandbox: resolve -> set_mtu(MOBILE) -> resolve still
flows through the same WireGuard session (live_bringup.rs).
Completes 42/42 tasks for nym-sdk-dvpn.
feat(smol-dvpn): tonic gRPC example through the tunnel (task 5.4)
Add examples/smol-dvpn-grpc.rs: brings up a single-hop tunnel, builds a tonic
Channel over tunnel.connector() (so gRPC/HTTP2 flows inside the tunnel), and
issues a gRPC Health.Check using tonic-health's ready-made client (no custom
proto/build.rs). Compiles; run against a live gRPC health service like the
other live examples.
Also annotate task 4.7 as partial: configurable MtuConfig + reference
DESKTOP/MOBILE defaults are implemented, but MTU resize-while-up is blocked by
tokio-smoltcp 0.5 (BufferDevice caches the interface MTU at construction).
41/42 tasks; 4.7 partial pending an upstream tokio-smoltcp capability.
feat(sdk): pure-Rust userspace dVPN datapath (smol-core, nym-sdk-session, smol-dvpn)
Implements the nym-sdk-dvpn OpenSpec change: a reusable, pure-Rust building
block for a paid 1-/2-hop WireGuard dVPN tunnel that pushes ordinary tokio
traffic (TCP/UDP, tonic/hyper) without root or an OS tun device.
New crates:
- common/smol-core: transport-agnostic userspace smoltcp stack (IP-packet
Vec<u8> stream -> TcpStream/UdpSocket + tunnel DNS resolver). smolmix is
refactored onto it with its public API preserved.
- sdk/rust/nym-sdk-session: provisioning facade (mnemonic -> zk-nym ticketbook
issuance + persistent storage, gateway selection by identity/country/random,
single- and two-hop LP registration), cancellation-aware.
- sdk/rust/smol-dvpn (nym-smol-dvpn): boringtun userspace datapath on smol-core;
WgPacketTransport seam (Direct UDP + inline QUIC bridge, ALPN hq-29 +
ed25519-SPKI pinning + 2-byte framing); single-hop and nested two-hop
encapsulation; timer pump; lifecycle + CancellationToken; DNS-in-tunnel;
tonic/hyper connectors; background bandwidth top-up; smol-dvpn-config /
smol-dvpn-topup example CLIs.
boringtun/quinn/quinn-proto are declared crate-local (not the workspace table).
Verified: conformance spikes (two-hop nesting, QUIC framing/pinning) pass;
smol-core TCP/UDP/DNS + smolmix API-lock + bridge conformance + session unit
tests pass; and the full stack is proven end-to-end against live sandbox
gateways (single-hop and two-hop both resolve through the tunnel) via
sdk/rust/smol-dvpn/tests/live_bringup.rs.
40/42 tasks complete. Remaining: runtime MTU resize-while-up (tokio-smoltcp 0.5
cannot resize a live interface) and a full tonic gRPC round-trip test (needs a
proto/generated client + live service).
jstuczyn
force-pushed
the
feature/nym-sdk-dvpn
branch
from
July 25, 2026 18:05
22aa2d5 to
b159f5d
Compare
remove garbage data
post rebasing fixes
openspec: archive dvpn-registration-reuse after mainnet validation
Mainnet validation passed (2026-07-24): consecutive runs against the same
gateways reused the cached registrations with zero ticket spend, post-rename.
Deltas merged into live specs: dvpn-session gains 'Gateway registration
persistence and reuse', dvpn-tunnel gains 'Awaitable per-hop session
establishment'. The same run completes smoldvpn-rename task 4.1 (live run
under the new name), leaving that change fully implemented and ready to
archive whenever convenient.
smoldvpn: move to repo root and rename from nym-smol-dvpn
Implements openspec change smoldvpn-rename: the dVPN datapath crate is now
consistent with its sibling smolmix — sdk/rust/smol-dvpn -> smoldvpn/ (git
mv, history follows), package + lib renamed nym-smol-dvpn/nym_smol_dvpn ->
smoldvpn. Never published to crates.io, so the rename precedes first
publication (no legacy name to manage).
Rename mapping for archaeology:
nym-smol-dvpn -> smoldvpn (package, -p flags, prose)
nym_smol_dvpn -> smoldvpn (use paths, RUST_LOG targets)
smol-dvpn-* -> smoldvpn-* (example binaries + their data/ dirs)
- Examples smol-dvpn-{config,grpc,topup} renamed smoldvpn-*; the shared
examples (zcash-sync, two-hop-*, quic-probe) and their data dirs are
name-stable. README gains a migration note (RUST_LOG target change;
rename local data/smol-dvpn-* dirs to keep credentials).
- Workspace member updated (sorted next to smolmix); Cargo.lock follows.
- Prose updated: crate README, common/smol-core README, nym-sdk-session
README + doc comments; docs/design/sdk/smol-dvpn -> docs/design/smoldvpn.
- Live specs dvpn-tunnel/dvpn-tools/dvpn-quic-bridge updated in place; the
change's MODIFIED deltas carry identical text so archiving is idempotent.
Historical openspec change artifacts deliberately untouched.
Verified: straggler grep clean outside historical artifacts; full test
suites green for smoldvpn + nym-sdk-session; clippy clean incl. examples;
`cargo run -p smoldvpn --example zcash-sync -- --help` works; examples'
default log filter now spells smoldvpn=info,boringtun=info.
openspec: archive dvpn-signer-failure-tests + dvpn-signer-fault-http-harness
Both changes fully implemented and validated (the signer-tolerance change
including its live mainnet validation). Deltas merged into live specs:
dvpn-session gains the 'Tolerance to unresponsive ecash signers'
requirement; the new dvpn-signer-fault-harness capability spec is created
(harness, multi-API characterization, threshold probe).
dvpn-registration-reuse stays active pending its manual mainnet validation
(task 5.3). Also records the smoldvpn-rename D6 resolution: nym-smol-dvpn
was never published to crates.io, so the rename precedes first publication.
docs: ignore docs/scratch (working notes stay uncommitted)
nym-sdk-session + smol-dvpn: address code-quality review findings
Thermo-nuclear review of the branch's session work; behavior-preserving
restructurings, all suites green (47 tests) and clippy clean:
- session.rs had crossed the 1k-line boundary (747 -> 1145): split
SessionConfig + RestockPolicy into config.rs and the unit tests into
session_tests.rs (#[path] module) — session.rs is back to 792. In
passing this fixed a latent export bug: RestockPolicy was reachable only
as an unnameable public field type; it is now re-exported.
- The persist-then-assemble HopConfig tail was copy-pasted across all three
registration sites: extracted as Session::finalize_hop (testable
identity+GatewayInfo signature, same as cached_hop).
- The reg-cache lock/poison-recovery incantation was triplicated:
collapsed into Session::with_cache.
- datapath() had grown to 8 args behind a clippy::too_many_arguments allow
while two of them already formed the named StackChannels type: pass the
named pair, allow removed.
- clone_cfg's serde round-trip now documents WHY deriving Clone upstream is
wrong (PresharedKey is ZeroizeOnDrop secret material that deliberately
omits Clone), not just that the derive is missing.
Reviewed and left as-is (justified): TimeoutFetcher decorator shape,
opt-out-as-Option cache design, the examples' per-file self-containment
duplication, engine progress flags.
nym-sdk-session: HTTP-level signer-fault harness (tests only)
Implements openspec change dvpn-signer-fault-http-harness: drive the REAL
NyxdGlobalDataFetcher — real DKG discovery, real HTTP client, real
try-each-API loop — against local mock nym-apis whose ecash routes are
independently scriptable as healthy / slow / erroring / hanging (accept the
connection, never respond: the exact client-visible behavior observed
against mainnet). Dev-only; no common/ production code touched, no new
mock-server dependency (the Hang mode is trivial on a raw TcpListener).
- tests/support/http_harness.rs: MockNymApi with per-route FaultMode and
runtime re-scripting; teardown releases parked Hang connections.
- tests/support/fake_dkg.rs: single-method DkgQueryClient double answering
vk-share + threshold queries from fabricated ContractVKShares announcing
the mock URLs (owners must be real-format bech32 — discovery parses them
into cosmrs AccountIds). Fails loud on unexpected queries.
- tests/support: TestEcash gains vk-share bs58 accessors and annotated
expiration-date-signature fixtures (parseable by the real client).
- tests/signer_fault_http.rs (6 tests, ~2.6s): fixture sanity through real
discovery; characterize_* reproduction of the mainnet hang on the raw
stack; TimeoutFetcher guard measured end-to-end; multi-API bounded
retries; threshold probe both sides (K >= threshold: partials retrievable
during a full aggregated outage — the client-side aggregation go-signal;
K < threshold: insufficiency reported).
Findings recorded in the change's design.md, notably: the SDK's per-fetch
bound wraps the fetcher's whole try-each-API loop, so per-API fallback needs
a per-request bound inside query_random_apis_until_success
(common/bandwidth-fetcher) — the concrete item to hand the ecash owners.
cargo test -p nym-sdk-session: 40 tests green; clippy --tests clean.
nym-sdk-session + smol-dvpn: reuse gateway registrations across runs
Every registration used to generate fresh WireGuard keys and spend one
zk-nym ticket per hop, even against a gateway registered with moments
earlier — abandoning the previous peer's remaining ~477 MiB allowance on
the gateway (observed on mainnet: two identical runs = 4 tickets for
~500 MB used). Implements openspec change dvpn-registration-reuse; no
common/ production code touched.
- registration_cache (new): registrations.json in the session data dir —
client WG key + gateway-returned WireguardConfiguration keyed by
(network, gateway, role); atomic writes, 0600 on unix, 30-day hygiene
pruning (validity is proven by use, never assumed). Contains private
keys/PSKs: same sensitivity class as the adjacent creds.db.
- Session: register_single_hop/register_two_hop(_quic) consult the cache
after gateway selection — full hit returns with NO gateway exchange and
NO ticket spent ('reusing cached registration … — no ticket spent' at
info); partial two-hop hits register (and provision ticketbooks for)
only the missing hop. Fresh registrations persist immediately after
register_dvpn. New: invalidate_registration(gateway, role) for the
fallback path; SessionConfig::reuse_registrations (default on; opt-out
documents the peer-linkability trade-off and neither reads nor writes
the cache).
- smol-dvpn: Tunnel::await_established(timeout) — per-hop establishment
published from the engine's handshake tracking via a watch channel;
NotEstablished reports entry/exit status so callers invalidate exactly
the dead hop.
- Examples: common::connect() implements register → build →
await_established(15s) → invalidate-failed-hops → re-register-once;
adopted by zcash-sync/two-hop-ip/two-hop-quic (and smol-dvpn-grpc
inline), replacing the blind 10×3s ipinfo warmup loops with an
establishment gate + short display probe.
- Tests: 8 cache unit tests; 4 offline session-level reuse tests
(counting provider proves zero-spend cache hits, opt-out, invalidation,
restart survival); 4 loopback await_established tests with a real
boringtun responder (incl. exit-only failure attribution). The
await_established tests run on a manual runtime with shutdown_timeout —
the smol-core stack parks blocking-pool workers that outlive the
tunnel, which would hang #[tokio::test] forever in Runtime::drop (the
examples sidestep the same via std::process::exit(0)).
cargo test: 28 (session lib) + 7 (signer_failure) + 4 (await_established)
+ crate suites all green; clippy clean on lib/tests/examples for both.
Remaining: manual mainnet validation (task 5.3).
smol-dvpn: make WireGuard handshake progress observable at info level
Debugging a mainnet two-hop tunnel that registered fine but passed no data
(DNS-in-tunnel timeouts) showed the datapath is a black box between
'datapath starting' and the first application traffic: handshake success is
never logged at any level, and decapsulate errors (e.g. a rejected handshake
response) were silently swallowed.
- decap(): log TunnResult::Err at warn instead of dropping it silently.
- WgEngine: one-shot info markers — 'first datagram received from the entry
transport', 'entry-hop wireguard session established', 'exit-hop wireguard
session established' (single-hop: 'wireguard session established'), driven
off boringtun's stats().0 flipping to Some. A single default-log run now
localizes a dead tunnel: no inbound datagrams at all vs. entry handshake
never completing vs. exit handshake never completing vs. established but
no upstream data.
- 'dVPN tunnel datapath starting' now logs the entry/exit WG endpoints, so
the exact UDP destinations being dialed appear in the log.
- Fix the pre-existing clippy while-let warning in the decap drain loop.
smol-dvpn: close the session credential store cleanly in the examples
The example CLIs never called Session::shutdown(): error paths dropped the
session without awaiting the bandwidth controller's cleanup (surfacing
'SqlitePoolGuard dropped without explicit close()' for creds.db), and the
happy paths called std::process::exit(0), which skips destructors entirely —
leaving the sqlite WAL uncheckpointed (the multi-hundred-KB creds.db-wal
observed after mainnet runs).
Wrap each session-using flow in an async block and always await
session.shutdown() afterwards — success or failure — before printing the
final verdict / exiting. The shutdown awaits the controller task, which
closes the fetcher's pending-request store and the credential store
(WAL checkpoint); stored tickets are retained. Applies to all six
session-building examples: two-hop-ip, two-hop-quic, zcash-sync,
smol-dvpn-config, smol-dvpn-grpc, smol-dvpn-topup.
openspec: record successful mainnet validation of signer-failure tolerance
zcash-sync against mainnet (2026-07-23): provisioning completed in ~1m47s
instead of hanging forever; both SignerTimeout warnings fired at the 15s
bound; both wireguard ticketbooks (entry + exit, 50 tickets each) persisted
to data/zcash-sync/mainnet/creds.db. Ticks the last open task (5.4).
nym-sdk-session: tolerate unresponsive ecash signers (no hang, no lost funds)
Mainnet's aggregated-expiration-date-signatures endpoint accepts connections
and never responds (offline distributed signers — a permanent operating
condition). With no request timeout anywhere in the fetch stack, the
bandwidth controller's store_ticketbook wedged mid-persist on its run loop:
provisioning hung forever, the freshly issued (paid-for) ticketbook was never
stored, and every retry re-deposited NYM. SDK-confined fix per openspec
change dvpn-signer-failure-tests; no common/ production code touched.
- TimeoutFetcher decorator (new fetcher.rs): bounds the three read-only
global-signing-data fetches at 15s per call, surfacing a SignerTimeout
FetcherError naming unresponsive signers. fetch_ticketbooks (the deposit)
is deliberately NOT timed. Wired around NyxdCredentialFetcher in
Session::spawn_controller. A bounded failure is recoverable: the
controller's best-effort store path persists the ticketbook anyway and
readiness resolves from stored stock; missing signatures back-fill later.
- ensure_ticketbooks gains an overall 5-minute budget mapping to a new
SessionError::ProvisioningTimeout (defense in depth; cancellation race and
funds-safety unchanged). Note: SessionError is not #[non_exhaustive], so
exhaustive matchers gain one arm.
- Tier 1 tests (virtual clock, in fetcher.rs): hang→bounded error for all
three fetches, under/over-threshold boundary, error passthrough, and
issuance provably unbounded by the decorator.
- Tier 2 tests (tests/signer_failure.rs + tests/support): the real
BandwidthController over ephemeral storage with a FlakyFetcher reproducing
the mainnet outage, and a locally fabricated 2-of-3 threshold-signed
ticketbook (no chain, no network, no funds). Proves: the pre-fix wedge and
empty store (the exact observed creds.db state), post-fix persistence +
resolved readiness, zero re-purchase on retry, and fail-fast spend during
the outage.
- openspec: dvpn-signer-failure-tests artifacts (tasks ticked; only optional
manual mainnet validation remains) and the follow-up
dvpn-signer-fault-http-harness proposal (HTTP-level fault harness, may not
be implemented).
cargo test -p nym-sdk-session: 23 passed. clippy --tests: clean.
resolve .gitignore merge conflict
Keep both sides of the conflict: tmp/ + the operator-tools nodes.csv
(HEAD) and *storybook.log (NYM-1199).
smol-dvpn: log example output via tracing, restructure example data dirs
Wire a tracing subscriber into every example so the crate's logs (and the
examples' own narration) are visible out of the box, and convert the examples
from println! to tracing macros.
- Add tracing-subscriber (env-filter) dev-dep; each example installs a
stderr fmt subscriber honouring RUST_LOG. Default filter (RUST_LOG unset)
is the running example plus nym_smol_dvpn and boringtun at info; the
example crate root is derived from module_path! so its own info! shows.
- Convert progress/result output to info!, retries/probe failures to warn!,
and fatal main errors to error!. Logs go to stderr; stdout stays reserved
for genuine deliverables (the smol-dvpn-config WireGuard config and --help).
- Log the per-example credential store / data directory at startup.
- Restructure example data output to data/<example>/<network> (e.g.
data/zcash-sync/sandbox) instead of flat *-data dirs; create it if missing.
- Ignore /data/ in the crate and repo-root .gitignore.
- Update README developer notes for the logging behaviour and data layout.
use nym native types on 'PeerConfig' and remove Clone derivation
use concrete inner error for SessionError::Api
fix deps of nym-registration-client
ibid for 'nym-authenticator-client'
mark 'nym-registration-client' as publishable
clippy
openspec: sync specs + design doc with delivered dVPN design
Reconcile the OpenSpec deltas with what actually shipped and apply them to the live
specs (archiving fix-dvpn-review-6953 and fix-coderabbit-findings-6953):
- bandwidth-type-scoping (new capability): reflects the PR #6976 design — the
controller's `managed_ticket_types` config field is the sole scoping mechanism
(empty set = no proactive restock; post-spend restock gated on the set). The
earlier fetcher-level veto approach was dropped in review and is no longer
described.
- dvpn-session / dvpn-tunnel / smol-core-stack: add the delivered requirements
(running-controller provisioning, opt-in restock, registration-not-cancellable-
past-spend, non-empty QUIC pin, in-tunnel top-up + events, set_mtu survival,
transport error handling, IP-literal handling, IPv4 bridge preference, bounded QUIC
connect, validated/truncation-aware DNS).
Also update docs/design/sdk/smol-dvpn/design.md: replace the removed `auto_restock`
reference with the managed-ticket-types model, and clarify that a registration
ticket-spend runs to completion (a cancel can't drop it mid-spend).
nym-sdk-session: fix two issues found reviewing the CodeRabbit fixes
- dvpn.rs `into_quic`: iterate all `quic_plain` transports (find_map) instead of
only inspecting the first — a malformed earlier transport (no routable address or
no identity pin) no longer shadows a valid later one. Also store the `id_pubkey`
(and SNI host) trimmed, so a whitespace-padded directory value still base64-decodes
at connect time. Adds unit tests.
- session.rs: keep the ticket-free LP `perform_handshake` cancellable. The previous
fix made the whole registration exchange non-cancellable; only the ticket-spending
`register_dvpn`/`handshake_and_register_dvpn` calls need that, so a stalled/
black-holed entry gateway would otherwise ignore the cancel token until the OS TCP
timeout. The handshake now races the cancel token again; only the spend runs bare.
smol-dvpn: address CodeRabbit review findings (#6953)
Correctness:
- nym-sdk-session: registration is no longer cancellable once a WireGuard ticket is
spent. Gateway selection (topology fetch) stays cancellable via
fetch_topology_cancellable, but register_single_hop/register_two_hop/
register_two_hop_quic run the LP registration exchange to completion without racing
the caller's cancel token — closing the lost-ticket window if a cancel fired after
the gateway processed the spend.
- smol-core dns: reject a truncated (TC-bit) response with a new DnsTruncated error
instead of returning the partial answers it carries (RFC 1035).
- smol-dvpn bridge: bound conn.open_bi() by the same CONNECT_TIMEOUT + cancellation
as the QUIC handshake, so a stalled bridge can't hang connect().
Robustness / cleanup:
- smol-core: remove the dead StackConfig::mtu field + with_mtu (Stack::new never read
it; the caller-supplied device is the MTU source of truth).
- nym-sdk-session: reject a blank id_pubkey so an unpinned bridge is never advertised
as QUIC-capable.
- live_bringup test: drop std::process::exit(0) (which could kill sibling tests);
the tokio::time::timeout around shutdown() already bounds teardown.
Docs:
- smol-dvpn README: reword the kill-switch/no-leaks line to a scoped guarantee (only
sockets opened through the tunnel are protected; ordinary sockets and host DNS
bypass it).
- design doc: clarify the bridge SNI must be an accepted cert alt-name (the "decoy"
is how ordinary it looks to DPI, not a free-form hostname), and document the
top-up headroom (top-up fires at a threshold above zero so the in-tunnel metadata
request still fits).
Adds a smol-core test asserting truncated responses are rejected.
smol-dvpn: address PR #6953 review feedback
Datapath / correctness:
- topup: dial the gateway metadata endpoint THROUGH the tunnel (hyper/1 over
TunnelConnector) instead of the host network, closing the IP-leak/broken-topup
blocker.
- tunnel: fix the set_mtu stack-swap race (biased select! with the swap branch
first, break on swap-channel closure) and serialise concurrent set_mtu behind a
mutex; classify transport recv errors (fatal bridge close exits; transient UDP
backs off) to end the busy-loop.
- connectors/smol-core: connect IP-literal hosts directly instead of routing them
through DNS (strip IPv6 brackets); short-circuit in resolve/tcp_connect_host.
- bridge: prefer IPv4 candidate addresses for v4-only clients.
smol-core DNS: random per-query id validated against the response id AND source,
recv loop until match/timeout, distinct DnsServerFailure for SERVFAIL/REFUSED,
and skip AAAA on a v4-only stack (without discarding resolved A records).
nym-sdk-session: run the bandwidth controller (single writer) and spend via its
request sender; scope acquisition to WireGuard ticket types; opt-in chain restock
(with_automatic_topups), gateway-side top-up on by default via a shipped
ProviderCredentialSource; external-provider escape hatch; hashed+zeroized
client id (no mnemonic clone); single topology fetch for two-hop; drop the mixnet
entry/exit role restriction for dVPN selection; split Nyxd/Api errors.
Performance: engine owns one reusable encrypt/decrypt scratch buffer and the
Direct receiver reuses its recv buffer (no per-packet 64 KiB alloc+zero).
Also: BandwidthEvent stream (tunnel.bandwidth_events()), typed WG keys in
PeerConfig, redacted Debug, target-locked mobile MTU, #[must_use] builders, and
doc corrections (design §10 top-up model; nym-bridges now on crates.io).
update nym-bridges dep to use version published to crates.io
permit usage of AGPL-3.0 dependencies
relax doc comment compile-time check restrictions
update the sample to no_run
fix mnemonic parsing
ignore tests when env is not set
chore(smol-dvpn): remove conformance spikes, normalize license headers
- delete the throwaway conformance spikes (tools/internal/dvpn-spikes) and
drop the now-dead path pointer from the archived OpenSpec design doc
- replace the copyright header in the three new crates (smol-core,
nym-sdk-session, nym-smol-dvpn) with the 2026 notice + Apache-2.0 SPDX
identifier, in both source files and Cargo.toml headers
- add a License section to the smol-dvpn README (Apache-2.0)
docs(smol-dvpn): move design docs, archive OpenSpec change, sync specs
Move sdk/rust/docs/nym-sdk-dvpn -> docs/design/sdk/smol-dvpn and rework the
docs now that the implementation is done:
- drop the exploration framing (open-questions.md, "no implementation yet",
spike A/B, D-numbered decision jargon) from the design docs, crate READMEs,
and source doc-comments
- reconcile the QUIC bridge design with reality: it now depends on the
git-pinned nym-bridges client rather than a byte-match reimpl
- add a builder-focused "What can I use nym-smol-dvpn for?" section to the
crate README, with a pointer to nym-credential-proxy for gifting zk-nyms
- archive the completed nym-sdk-dvpn OpenSpec change and sync its five delta
specs into openspec/specs/ (dvpn-tunnel, dvpn-quic-bridge, dvpn-tools,
dvpn-session, smol-core-stack)
- link the synced capability specs from the READMEs
chore(nym-sdk-session): migrate off deprecated NymApiClient
`nym_validator_client::client::NymApiClient` is deprecated in favour of
`nym_http_api_client::Client`. Use the client directly and reach the paged
`get_all_described_nodes_v2` via the `NymApiClientExt` trait, matching the
pattern already used in `nym-sdk`. Behaviour is preserved: the session's only
API call bypassed the wrapper's bincode flag, so `Client::new(url, timeout)` is
an exact substitute. Adds `nym-http-api-client` as a direct dependency.
refactor(smol-dvpn): use the nym-bridges QUIC client instead of a byte-match reimpl
The inline QUIC bridge client duplicated the reference cert-pinning verifier,
ALPN and dialing logic and had to be kept byte-for-byte in sync with the server
by hand. Delegate the connection to the canonical `nym_bridges` crate
(`transport::quic::{ClientOptions, transport_conn}`) so the client can never
drift from the bridge:
- Drop the local `IdentityBasedVerifier`, cert SPKI extraction and quinn client
config (~150 LOC); keep only the datapath framing (one `open_bi()` stream,
2-byte big-endian length prefix per WireGuard packet) plus the cancellation +
connect-timeout wrapper.
- `probe` and `connect` keep their signatures. `BridgeParams` now carries the
directory's base64 `id_pubkey` string and passes it straight to
`nym_bridges::ClientOptions`, dropping the `[u8; 32]` field and the
hand-rolled base64 codec that only existed to feed the old local verifier.
- This picks up the reference client's IPv6-first dial and dual-stack `[::]:0`
bind, replacing the previous IPv4-only path.
`nym-bridges` is not published to crates.io, so it is pulled as a pinned git
dependency; `cargo publish` of this crate is therefore blocked until it is
released. Drop the now-unused `ed25519-dalek` and `x509-parser` deps; add `bs58`
and `base64` as dev-deps for the conformance test, which now exercises framing +
ed25519 pinning against a real `nym_bridges` QUIC server.
chore(smol-dvpn): lower zcash-sync default blocks to 10_000
Faster default test run; still overridable with --blocks <N>. Docs + OpenSpec
updated to match.
perf(smol-core): widen TCP window + uncap device burst (10x tunnel throughput)
Bulk transfers through the tunnel (e.g. zcash-sync) were ~10x slower than direct
because of two self-inflicted caps in the smoltcp stack:
1. tokio-smoltcp's default per-socket TCP buffer is 8 KiB, and the rx buffer IS
the TCP receive window, so throughput was capped at 8 KiB / RTT — crippling on
a higher-RTT two-hop path. StackConfig now defaults to a 512 KiB window
(DEFAULT_TCP_BUFFER, tunable via with_tcp_buffer) applied through
NetConfig.buffer_size; smoltcp window scaling handles >64 KiB.
2. ChannelDevice set max_burst_size = Some(1), making smoltcp process one packet
per poll and serializing the datapath. Left unbounded (None).
Measured (release, zcash-sync --blocks 1000, sandbox two-hop):
before: tunnel 32 blocks/s, 87x slower than direct (debug baseline)
after: tunnel 3937 blocks/s, 1.61x slower than direct
smol-core unit tests still pass.
feat(nym-sdk-session): two-hop selection never reuses the entry gateway as exit
gateway::select gains an `exclude` parameter; register_two_hop(_quic) passes the
chosen entry's identity when selecting the exit, so random/country selection can
never resolve both hops to the same gateway. An exit spec that can only match the
entry gateway now fails with the new SessionError::SameGatewaySelected rather than
building a degenerate one-gateway "two-hop" tunnel.
Unit test added (excluded_identity_is_rejected); dvpn-session spec + session
README + tasks updated.
feat(smol-dvpn): configurable zcash-sync block count; document release builds
- zcash-sync: add --blocks <N> (shared example CLI) to choose how many compact
blocks to sync; default raised 1000 -> 100_000. Verified live that the count
flows into the GetBlockRange query (250 -> 250 blocks synced direct).
- Document building the examples with --release across the READMEs and example
usage docs: boringtun's userspace crypto is much slower in debug, which
dominates the through-tunnel timing (acute at the new 100k default).
- Sync OpenSpec (tasks 7.5/7.6, dvpn-tools spec) with the configurable count.
docs(nym-sdk-dvpn): sync READMEs + OpenSpec with QUIC/metadata/examples
READMEs:
- nym-sdk-session: document SessionConfig.dvpn_directory_url, the dVPN directory
(gateway monikers + QUIC entry selection), register_two_hop_quic / QuicBridge /
NoQuicGateway, and the GatewayInfo metadata now on HopConfig.
- smol-dvpn: note the MTU is runtime-adjustable via Tunnel::set_mtu().
OpenSpec (validates --strict):
- proposal: list the full example set (grpc, two-hop-ip, two-hop-quic, zcash-sync
+ shared CLI) and the directory/QUIC session capability.
- dvpn-session spec: add requirements for optional dVPN directory metadata and
QUIC-bridge entry gateway selection.
- dvpn-tools spec: add requirements for the gRPC, IP-relocation, Zcash-sync
examples and the shared selection CLI.
- tasks: record 3.9/3.10 (directory + QUIC selection) and 7.3–7.6 (new examples
+ CLI), all implemented.
docs(smol-dvpn): configurable example CLI + detailed README
- Add a shared example CLI (examples/common): --one-hop/--two-hop,
--entry/--exit/--gateway <SPEC> (random | ISO country | base58 identity),
and --quic, with validation (QUIC is two-hop entry only) and --help. The
two-hop-ip, two-hop-quic, and zcash-sync examples now honour it via a common
register()/build_tunnel() dispatch (single- vs two-hop, Direct vs QUIC entry).
- Expand the README Examples section: per-example table, full option/SPEC
reference, invocation recipes, and a mermaid sequence diagram of the zcash-sync
flow (ticketbooks -> two-hop registration -> gRPC compact-block sync -> teardown).
- Add a Developers section documenting how to point the examples at sandbox
(source envs/sandbox.env + funded MNEMONIC; DVPN_DIRECTORY_URL / RUST_LOG).
Verified: cargo clippy --examples clean; --help/validation paths; and a live
default two-hop-ip run still relocates the public IP to the exit gateway.
feat(sdk): dVPN example programs + QUIC-bridge gateway selection
nym-sdk-session:
- Add a lightweight dVPN gateway-directory client (dvpn.rs). QUIC bridge params
live in the VPN/dvpn directory, not the nym-api described-nodes, so the session
fetches the directory (best-effort) to enrich gateway metadata and enable QUIC
entry selection. SessionConfig gains `dvpn_directory_url: Option<String>`.
- GatewayInfo gains `name` (directory moniker) and already carried node_id/country/ip;
HopConfig gains `bridge: Option<QuicBridge>` (set only for a QUIC entry hop).
- New `QuicBridge` (addresses/SNI/base64 id_pubkey) and
`register_two_hop_quic(entry, exit)`, which requires a QUIC-capable entry
(honoring the GatewaySpec) and fails with `NoQuicGateway` otherwise.
register_single_hop / register_two_hop are unchanged (bridge = None).
smol-dvpn examples:
- two-hop-ip: prove a two-hop tunnel relocates your public IP (ipinfo.io direct
vs. through the tunnel); `--quic` requires a QUIC entry.
- two-hop-quic: two-hop tunnel whose entry leg is carried over a QUIC bridge.
- zcash-sync: time syncing the last 1000 Zcash compact blocks from a public
lightwalletd (gRPC-over-TLS) directly vs. through the tunnel; `--quic` supported.
- Shared examples/common/ (session/tunnel setup, hop->PeerConfig, gateway
printing, HTTPS-over-any-stream ipinfo fetch, a TLS-wrapping tower connector
for gRPC-over-TLS through the tunnel, QuicBridge->BridgeParams mapping).
- Thread the new SessionConfig field through the existing examples + live test.
Verified against sandbox: two-hop-quic selects the designated QUIC entry gateway
from the directory, populates its moniker, extracts bridge params and dials the
correct IPv4 endpoint; the sandbox QUIC bridge endpoint is currently unreachable
(ICMP-alive host, QUIC/UDP:4443 times out) so the live handshake is infra-blocked,
but the full selection/param/datapath path is exercised. cargo build/clippy clean
across both crates (examples + tests).
feat(smol-dvpn): runtime-adjustable MTU via WG-preserving stack rebuild (task 4.7)
Add Tunnel::set_mtu(): rebuilds the smol-core interface at the new MTU while
keeping the WireGuard engine/session alive (no re-handshake), by swapping the
datapath's stack-side channels through a control channel. The stack is held
behind a swappable handle (Arc<RwLock<Arc<Stack>>>) so the tokio socket
surfaces and the tower connector keep working across the swap.
tokio-smoltcp 0.5 fixes the interface MTU at construction (BufferDevice caches
capabilities), so this rebuilds the interface rather than a fully seamless
in-place resize; open sockets at the instant of change are reset while the
tunnel stays connected.
Verified live against sandbox: resolve -> set_mtu(MOBILE) -> resolve still
flows through the same WireGuard session (live_bringup.rs).
Completes 42/42 tasks for nym-sdk-dvpn.
feat(smol-dvpn): tonic gRPC example through the tunnel (task 5.4)
Add examples/smol-dvpn-grpc.rs: brings up a single-hop tunnel, builds a tonic
Channel over tunnel.connector() (so gRPC/HTTP2 flows inside the tunnel), and
issues a gRPC Health.Check using tonic-health's ready-made client (no custom
proto/build.rs). Compiles; run against a live gRPC health service like the
other live examples.
Also annotate task 4.7 as partial: configurable MtuConfig + reference
DESKTOP/MOBILE defaults are implemented, but MTU resize-while-up is blocked by
tokio-smoltcp 0.5 (BufferDevice caches the interface MTU at construction).
41/42 tasks; 4.7 partial pending an upstream tokio-smoltcp capability.
feat(sdk): pure-Rust userspace dVPN datapath (smol-core, nym-sdk-session, smol-dvpn)
Implements the nym-sdk-dvpn OpenSpec change: a reusable, pure-Rust building
block for a paid 1-/2-hop WireGuard dVPN tunnel that pushes ordinary tokio
traffic (TCP/UDP, tonic/hyper) without root or an OS tun device.
New crates:
- common/smol-core: transport-agnostic userspace smoltcp stack (IP-packet
Vec<u8> stream -> TcpStream/UdpSocket + tunnel DNS resolver). smolmix is
refactored onto it with its public API preserved.
- sdk/rust/nym-sdk-session: provisioning facade (mnemonic -> zk-nym ticketbook
issuance + persistent storage, gateway selection by identity/country/random,
single- and two-hop LP registration), cancellation-aware.
- sdk/rust/smol-dvpn (nym-smol-dvpn): boringtun userspace datapath on smol-core;
WgPacketTransport seam (Direct UDP + inline QUIC bridge, ALPN hq-29 +
ed25519-SPKI pinning + 2-byte framing); single-hop and nested two-hop
encapsulation; timer pump; lifecycle + CancellationToken; DNS-in-tunnel;
tonic/hyper connectors; background bandwidth top-up; smol-dvpn-config /
smol-dvpn-topup example CLIs.
boringtun/quinn/quinn-proto are declared crate-local (not the workspace table).
Verified: conformance spikes (two-hop nesting, QUIC framing/pinning) pass;
smol-core TCP/UDP/DNS + smolmix API-lock + bridge conformance + session unit
tests pass; and the full stack is proven end-to-end against live sandbox
gateways (single-hop and two-hop both resolve through the tunnel) via
sdk/rust/smol-dvpn/tests/live_bringup.rs.
40/42 tasks complete. Remaining: runtime MTU resize-while-up (tokio-smoltcp 0.5
cannot resize a live interface) and a full tonic gRPC round-trip test (needs a
proto/generated client + live service).
remove stuff
mmsinclair
force-pushed
the
feature/nym-sdk-dvpn
branch
from
July 25, 2026 18:44
b159f5d to
5888ebf
Compare
This was referenced Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
smol-dvpn: a pure-Rust, userspace WireGuard dVPN datapath that lets a Rustapp tunnel its own traffic over the Nym network — 1-hop or 2-hop — with no
OS
tundevice and no root. Traffic goes in through ordinary tokio surfaces(
TcpStream,UdpSocket,AsyncRead + AsyncWrite), sotonic/hyper/reqwestwork inside the tunnel unchanged. Because the tunnel is scoped to the app rather
than the whole OS, closing it cuts the app off — a natural per-app kill-switch.
What's added
Three new crates:
common/smol-core— transport-agnostic userspace TCP/IP stack (smoltcp):turns a stream of IP packets into
TcpStream/UdpSocket/ DNS. Extracted soboth
smolmix(5-hop mixnet) andsmol-dvpn(dVPN) share it.sdk/rust/nym-sdk-session— shared provisioning facade: mnemonic-fundedzk-nym ticketbook issuance, persistent credential storage, and gateway
selection/registration. Used by both mixnet and dVPN modes.
sdk/rust/smol-dvpn— the WireGuard datapath (boringtun) and transportstrategy.
Capabilities
Tunns), andQUIC-tunnelling two-hop — the entry leg rides a QUIC bridge (via the
nym-bridgesclient) to evade DPI / UDP blocking.and issues its own WireGuard identity — no centralised shared public key.
network. Builders can gift credentials to end-users via
nym-credential-proxyso users never hold NYM.
in-tunnel by default; configurable, runtime-adjustable per-hop MTU;
CancellationTokenlifecycle.Examples (
sdk/rust/smol-dvpn/examples/)smol-dvpn-config(export a plain WG config),smol-dvpn-topup,smol-dvpn-grpc(gRPC through the tunnel),
two-hop-ip(prove IP relocation),two-hop-quic(QUIC-bridged), and
zcash-sync(throughput benchmark vs. direct).Docs & specs
docs/design/sdk/smol-dvpn/.openspec/specs/(dvpn-tunnel,dvpn-quic-bridge,dvpn-tools,dvpn-session,smol-core-stack); thenym-sdk-dvpnchange is archived.Testing
cargo test -p smol-core— stack integration tests (TCP/UDP/DNS).cargo test -p nym-smol-dvpn— QUIC bridge conformance (framing + ed25519-SPKIpinning). End-to-end tunnel bring-up is validated against a live Nym gateway via
the examples (needs a funded mnemonic + network).
This change is
Summary by CodeRabbit
New Features
Documentation
Compatibility