Skip to content

fix(shell): decode Windows output reliably - #29

Merged
h3c-hexin merged 5 commits into
pinvou3-cleanfrom
fix/windows-shell-output-encoding
Aug 26, 2026
Merged

fix(shell): decode Windows output reliably#29
h3c-hexin merged 5 commits into
pinvou3-cleanfrom
fix/windows-shell-output-encoding

Conversation

@h3c-hexin

Copy link
Copy Markdown

Background

This supersedes #28 after GitHub refused to reopen it because its head was rebased while temporarily closed. It continues #24 from a Pinvou-owned branch because the original head repository is no longer available for follow-up commits or workflow dispatch.

The first topic commit preserves @zhuowp as author and retains the original DCO trailer. Follow-up commits resolve the Clippy finding, make the test fixture deterministic, and add executable Windows coverage. The branch is rebased onto pinvou3-clean after the dependency advisory fix in #27 merged.

Windows-native tools may emit stdout or stderr using the active ANSI code page even when the surrounding shell is configured for UTF-8. The previous lossy, per-poll UTF-8 decoding could therefore produce mojibake and could corrupt a multibyte character split across background polls.

The reusable fix was first contributed upstream in Hmbown/CodeWhale #5602.

Changes

  • preserve strict UTF-8 decoder state across shell reads
  • fall back to the current Windows ACP only after genuine invalid UTF-8
  • share decoding semantics across synchronous output, snapshots, raw deltas, tails, and detached readers
  • seal detached streams without discarding bytes captured before the cutoff
  • derive completion events and evidence from one output cutoff
  • add decoder, shell-lifecycle, and forkguard_* regression coverage
  • remove a behavior-neutral loop whose branches all returned, resolving clippy::never_loop
  • make the incomplete UTF-8 unit test independent of the host ACP
  • run focused shell regressions on windows-latest in Pinvou fork CI
  • rerun the required contribution gate after PR head updates

No-Issue: replaces #28 and continues #24 from a maintainable repository-owned branch while preserving contributor credit.

Testing

  • cargo fmt --all -- --check
  • git diff --check
  • cargo test -p codewhale-tui --lib tools::shell --locked locally (110 passed)
  • cargo test -p codewhale-tui --lib tools::shell::output::tests --locked locally after the Windows test-fixture fix (9 passed)
  • cargo clippy -p codewhale-tui --lib --locked --no-deps -- -A warnings -D clippy::never_loop
  • Native windows-latest shell regressions before the dependency-baseline rebase (103 passed)
  • Required checks and Windows regressions on the rebased combined tree (running on this PR)
  • Full workspace Clippy (the public r10 baseline has unrelated Rust 1.98 warnings under -D warnings)

Risks

ACP detection is deliberately a fallback after invalid UTF-8 and cannot infer OEM or arbitrary child-selected encodings. The change is confined to the shell byte-decoding lifecycle and does not modify sandbox, PowerShell, Python, or command-generation behavior.

The contribution gate previously ran only when a PR was opened or reopened, so strict branch protection lost the required context after every update. This branch adds the missing synchronize event.

Checklist

  • Updated docs or comments as needed
  • Added or updated tests where relevant
  • Verified TUI behavior manually if UI changes (not a UI change)
  • Harvested/co-authored credit uses a GitHub numeric noreply address (not applicable; the original signed commit is preserved)

zhuowp and others added 5 commits August 26, 2026 13:36
Keep UTF-8 and legacy multibyte decoder state across background polls, use the Windows ANSI code page only after invalid UTF-8, and derive completion evidence from one output cutoff.

Keep the Windows-only ACP mapping cross-platform tested without triggering dead_code in non-Windows library builds. This incorporates the behavior from upstream follow-up 0a85b13 without its policy-incompatible automated co-author trailer.

Signed-off-by: zhuowp <zwp-1989@qq.com>
Return directly from the decoder match so Clippy no longer reports never_loop. This is behavior-neutral and preserves the original PR author commit unchanged.

Signed-off-by: hexin <he.xin@h3c.com>
Run the focused shell test module on windows-latest so native output decoding changes receive executable Windows coverage without being blocked by unrelated r10 warnings in the upstream full-suite workflow.

Signed-off-by: hexin <he.xin@h3c.com>
Exercise incomplete UTF-8 finalization without the host ACP so the assertion has identical semantics on Windows and Unix. Legacy fallback remains covered by its injected-encoding tests. Preserve the Windows build cache after failures to speed follow-up diagnostics.

Signed-off-by: hexin <he.xin@h3c.com>
Include pull_request_target synchronize events so the required gate context is attached to the current head after contributors push follow-up commits.

Signed-off-by: hexin <he.xin@h3c.com>
@h3c-hexin
h3c-hexin merged commit 665b46c into pinvou3-clean Aug 26, 2026
14 of 15 checks passed
@zhuowp

zhuowp commented Aug 26, 2026

Copy link
Copy Markdown

@h3c-hexin Pinvou/pinvou-agent#348 has now been realigned to the final merged maintenance head 665b46cd9e67326459223aa662931bd36d726004. The immutable pinvou-v0.9.5-r11 tag is still absent, and zhuowp has read-only access to Pinvou/CodeWhale, so I could not publish it. Please create the annotated immutable tag at that exact SHA; the parent verifier will remain fail-closed until the tag is public.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants