Surface Transfer Failures

[FrankRay78]

Closes #206.

Why

Certain upload endpoints (e.g. the Deutsche Telekom servers in #206) reported 0 bps while working on speedtest.net. The cause: the parallel test loop caught and discarded every per-request exception, counting failed requests as 0 bytes transferred. When every request failed, 0 bytes ÷ elapsed = 0 bps was reported as if it were a valid measurement — with no log, no message, and no non-zero exit code. This makes that failure visible.

Implements the approved specification in #206 (comment). Governing principle: Core reports the truth via request counts; the CLI applies policy. The exit code reflects only whether NetPace itself functioned — network conditions are data, not errors.

What changes

NetPace.Core (public API — MINOR bump)

• SpeedTestResult gains RequestsAttempted / RequestsSucceeded / RequestsFailed. Per-request failures (transport errors, timeouts, non-success HTTP status) are aggregated into these counts instead of being swallowed; the method no longer throws for network outcomes (only caller cancellation and operational faults propagate).
• Upload now validates the HTTP status — a rejected upload is a failed request, not throughput (mirrors the download path).
• SpeedTestProgress gains FailedRequestReason to stream each failure live; nothing is retained on the result.
• ISpeedTestService doc reversed to describe the counts-not-exceptions contract.

NetPace.Console

• Exit code reflects only NetPace's health: total outage, 100% request failure, no servers found, and cancellation all exit 0; only operational failures (e.g. can't write --file) exit non-zero.
• New --fail-on <total|partial|none> (default none) to opt in to a failure exit code; fail-fast and uniform across single / --count / --loop.
• Counts appear in every output format: token annotation (Upload: 0 bps (32 of 32 requests failed)), CSV columns (DownloadSucceeded/DownloadFailed/…), and JSON integer fields. Human notices and operational errors now go to stderr; at --verbosity Debug each failure reason is streamed live.

Docs: README --help snapshot and a new USER_GUIDE "Detecting failed measurements" section (counts, exit-code model, --fail-on).

Non-obvious things a reviewer should know

• This deliberately re-introduces a "return 0, don't throw" shape — the very shape that caused the bug — but only because two guarantees make a 0 bps un-ignorable: it always co-travels with the counts, and the CLI is obligated to surface them. Known accepted trade-off: netpace && next proceeds on a total outage by default; use --fail-on or inspect the counts to detect it.
• JSON diverges slightly from the spec wording. The spec says an all-failed dimension's speed is "emitted as null"; I omit the speed field instead (System.Text.Json WhenWritingNull, consistent with how --no-download already drops fields), letting "UploadSucceeded": 0 carry validity. Per-instance explicit null isn't achievable without also emitting null for skipped dimensions. Happy to switch to explicit null if preferred.
• Version: the MINOR bump is tag-driven in CI, so Directory.Build.props is intentionally untouched — tag v1.1.0 at release.
• An earlier "continue multiple tests on exception" test was repurposed: a throwing service is now operational (exit 1), whereas an all-failed measurement is data (exit 0, loop continues).

How to verify

• dotnet build src and dotnet test src are clean (633 tests, 0 warnings).
• netpace --json against a server whose uploads all fail → JSON has "UploadSucceeded":0/"UploadFailed":N, no UploadSpeed, nothing on stderr, exit 0.
• Same with --csv → the data row shows …,0,N,…; with default output → Upload: 0 bps (N of N requests failed) plus a stderr notice.
• --fail-on total on an all-failed dimension → exit 1; default → exit 0.
• --server <unreachable> (latency on) → exit 0 with the no-servers notice (regression: previously exit 1 only in this path).

[FrankRay78]

@claude Review this pull request. Analyse the code changes and provide feedback covering:

• Bugs or correctness issues
• Security concerns (including any flagged by static analysis)
• Adherence to the project conventions in CLAUDE.md
• Test coverage — do the tests adequately cover the new behaviour?
• Any spec/test-plan mismatches

[claude]

Claude finished @FrankRay78's task in 8m 17s —— View job

---

Review: Surface Transfer Failures (#222)

Todo list

• Gather context (diff, changed files, CLAUDE.md conventions)
• Review NetPace.Core changes (OoklaSpeedtest, SpeedTestResult, SpeedTestProgress, ISpeedTestService)
• Review NetPace.Console changes (SpeedTestCommand, writers, ErrorConsole, FailOn, exit-code logic)
• Review test coverage (Core.Tests, Console.Tests, snapshot expectations)
• Check docs (README, USER_GUIDE) for consistency with behaviour
• Check for spec/test-plan mismatches vs issue some servers show an upload speed of 0 bps, unlike the speetest.net website #206
• Post final review

Overall this is a well-scoped, thoroughly-tested implementation of the approved spec, and CI (Build and Test + CodeQL) is green. The core design — counts-not-exceptions in NetPace.Core, policy-in-the-CLI via --fail-on — is sound and consistently applied. Below are the issues I found, most-significant first.

---

1. --verbosity Debug live failure-reason streaming silently doesn't work with --csv / --json / --json-pretty

IConsoleWriter.PerformSpeedTestAsync now takes an errorConsole parameter for exactly this purpose, but only DefaultConsoleWriter (src/NetPace.Console/ConsoleWriters/DefaultConsoleWriter.cs:88-104) actually uses it. CSVConsoleWriter, JsonConsoleWriter, and MinimalConsoleWriter accept errorConsole but never reference it, and — more fundamentally — they all call the 2-arg GetDownloadSpeedAsync/GetUploadSpeedAsync overloads (no IProgress<SpeedTestProgress>), so FailedRequestReason progress updates are never even produced on those paths.

USER_GUIDE.md states unconditionally: "At --verbosity Debug, the raw reason for each failed request is streamed live to standard error." In practice netpace --csv --verbosity Debug (or --json/--json-pretty --verbosity Debug) prints nothing extra to stderr beyond the (also-suppressed, since ShouldEmitFailureNotice excludes CSV/JSON) failure notice — a real doc/behaviour mismatch.

This is also untested: the only debug-streaming test (Debug_Streams_Each_Failure_Reason_To_Stderr in NetPaceConsoleTests.Failures.cs) runs with no --csv/--json flag, so it only ever exercises DefaultConsoleWriter.

Suggest either scoping the USER_GUIDE line to interactive/Normal output, or wiring progress + errorConsole into the other three writers for consistency.

Fix this →

---

2. CSV header can be permanently skipped if the first --loop/--count iteration finds no server (pre-existing, but now much easier to hit)

SpeedTestCommand.ExecuteAsync drives the CSV header via initialSpeedTest: firstLoop (loop) / initialSpeedTest: (i == 0) (count) — both are keyed purely on iteration index, not on whether that first iteration actually produced a data row. Since finally { firstLoop = false; } runs even when PerformSpeedTestAsync returns SpeedTestOutcome.NoServers (no header/row written), a transient discovery failure on iteration 1 followed by success on iteration 2+ means no CSV header is ever printed, even though data rows are.

This flag/index pattern predates this PR, so it isn't a regression this PR introduces — but the PR's entire premise is that "no servers found" is now a routine, expected, non-throwing outcome in long-running --loop/--count sessions (see Should_Continue_Multiple_Speed_Tests_When_A_Measurement_All_Fails), which makes this latent edge case meaningfully more likely to actually occur in the field than it was when discovery failures usually threw. Worth a follow-up issue/test (--csv --loop where the first probe fails, later ones succeed) even if out of scope for this branch.

---

3. Minor / non-blocking observations

• OoklaSpeedtest.cs:341-354 — the rewritten cancellation-vs-failure branch (e is OperationCanceledException && cancellationToken.IsCancellationRequested replacing the old !wasCancelledLocally check) is a genuine correctness fix beyond the stated scope: the old condition would propagate any OperationCanceledException not caused by the local byte-cap cancel — including, e.g., an HttpClient-internal per-request timeout (TaskCanceledException) — as if the user had cancelled, aborting the whole test. Good catch, worth calling out since it's not mentioned in the PR description.
• GenericTestSpeedAsync's catch (Exception e) technically diverges from docs/conventions/csharp-style.md's "catch specific exception types" rule, but it's clearly the deliberate, well-commented mechanism for turning arbitrary per-request failures (transport, TLS, timeout, bad status) into counted data — the stated goal of the whole PR. Not a real concern, just flagging that a future style-linter pass shouldn't flag it blind.
• JsonResult.cs — Latency/DownloadSpeed/UploadSpeed are required string (non-nullable) but assigned via latencyFormatted!/downloadFormatted!/uploadFormatted!, relying on DefaultIgnoreCondition = WhenWritingNull to drop them. This pattern predates this PR (already used for Latency) and is explicitly flagged by the author in "Non-obvious things," so no action needed — just noting it now covers two more fields.
• No test exercises --quiet combined with an all-failed measurement to confirm the stderr notice still fires (existing quiet+file tests only cover the "no servers" case). Low priority, easy to add given ScriptedSpeedTester already supports it.

---

Adherence to CLAUDE.md / constitution

• ✅ No new NetPace.Core dependencies (no .csproj changes).
• ✅ XML docs present on all new/changed public NetPace.Core API surface (SpeedTestResult, SpeedTestProgress.FailedRequestReason, ISpeedTestService remarks).
• ✅ README --help snapshot and USER_GUIDE both updated (per the "CLI feature issues must scope docs from the start" rule).
• ✅ MINOR version bump correctly left tag-driven, Directory.Build.props untouched, as called out by the author.
• ✅ Program.cs's top-level catch now writes usage/operational errors to errorConsole (stderr) instead of stdout — a nice incidental clig.dev-alignment fix.
• No skipped tests, no build-warning risk introduced (CI confirms).

Test coverage

Coverage of the new behaviour is genuinely thorough: all-failed / partial-failure aggregation, status-code-as-failure for uploads, cancellation-still-propagates, and live-reason-streaming are all covered at the NetPace.Core level (OoklaSpeedtestTests.Failures.cs); exit-code policy, --fail-on (Total/Partial/None, fail-fast under --count), per-format notice suppression, and quiet/file interactions are covered at the NetPace.Console level via the new ScriptedSpeedTester (NetPaceConsoleTests.Failures.cs). The gaps I found are items #1 and #3's last bullet above.

Spec/test-plan mismatches

I couldn't fetch the linked issue #206 spec comment from this sandboxed environment (no network access) to do a line-by-line AC check, so I relied on the PR description + code/tests. The two divergences the author already flagged (JSON null-vs-omit, and the deliberate "return 0, don't throw" re-introduction) both read as reasonable, well-justified trade-offs. Test names reference AC5, AC8a/b, AC9–AC12 but not AC1–AC4/AC6/AC7 — likely because those map to the untagged NetPace.Core-level tests, but worth a quick self-check against the original comment before merge.
· Branch: feature/206-surface-transfer-failures

[FrankRay78]

@claude Please re-review, identify only showstoppers or important issues (P1/P2).

[claude]

Claude finished @FrankRay78's task in 1m 29s —— View job

---

Re-review: Surface Transfer Failures (#222)

Todo list

• Gather context (diff vs origin/main, prior review findings, latest commit)
• Check whether prior findings (Add testing project for speed test console #1 debug-streaming gap, Help includes application description at root. #2 CSV header, CodeQL shadowing/Dispose/Where) were addressed
• Re-scan NetPace.Core changes for correctness
• Re-scan NetPace.Console changes for correctness
• Post final review

No P1/P2 (showstopper/important) issues found.

Commit 3d8eff6 addressed both substantive items from the prior review:

• CSV header drop — SpeedTestCommand.cs now keys the header on firstWrite (set false only once outcome.ServersFound), not iteration index, so a leading no-server iteration in --loop/--count no longer permanently suppresses the header. Verified in SpeedTestCommand.cs:51-92.
• --verbosity Debug streaming doc mismatch — USER_GUIDE now scopes the live-streaming claim to normal/interactive output, matching the actual behaviour (CSV/JSON/Minimal writers never wire errorConsole/progress for this). Regression test added confirming JSON+Debug writes nothing extra to stderr.
• CodeQL shadowing in ScriptedSpeedTester.cs (server → defaultServer) is fixed.

The two remaining CodeQL notes (CommandLineTestHost.cs:38 missing-Dispose, SpeedTestCommand.cs:194 foreach-vs-Where) are both non-issues on inspection: testErrorConsole is already declared with using var, and the foreach/continue over a 2-element array is a style nit, not a correctness or security concern — neither rises to P1/P2.

Spot-re-checked the core failure-aggregation path (OoklaSpeedtest.cs GenericTestSpeedAsync), the CLI exit-code/--fail-on logic (SpeedTestCommand.cs), and the CSV/JSON writers for the IsAllFailed()/HasFailures() contract (SpeedTestResultReporting.cs:15-21, correctly guarding RequestsAttempted > 0 so a skipped dimension like --no-download isn't misreported as all-failed) — all consistent with the approved spec and no new correctness issues. CI (Build and Test) is green; CodeQL was still running at the time of this review.

Nothing blocking merge from this pass.
· Branch: feature/206-surface-transfer-failures