Skip to content

v0.0.3: Fix Buffer.toString to match Node.js UTF-8 decoding behavior

Choose a tag to compare

View all tags
@iwater iwater released this 17 Dec 18:42
· 8 commits to main since this release
v0.0.3
6e4ad59

Fix Buffer.toString to match Node.js UTF-8 decoding behavior

Summary

Updated Buffer.toString to handle invalid UTF-8 sequences correctly by replacing malformed bytes with U+FFFD (�) replacement characters, matching Node.js behavior per WHATWG standards.

Changes

  1. C++ Implementation (HybridNitroBuffer.cpp):

    • Fixed UTF-8 decoding to replace invalid sequences with U+FFFD
    • Added support for latin1/binary encodings (0x00-0xFF mapping)
    • Added ASCII encoding with U+FFFD replacement for non-ASCII bytes

  2. Documentation (README.md, README_zh.md):

    • Added compatibility notes explaining ASCII encoding differences
    • Clarified that Nitro follows Node.js standard (U+FFFD replacement) vs craftzdog (byte discarding)

Key Behavior

✅ UTF-8 decoding now matches Node.js WHATWG standards
✅ Invalid UTF-8 sequences replaced with U+FFFD (�) instead of malformed output
✅ ASCII encoding preserves length with replacement characters
✅ Latin1/Binary encodings for raw byte data handling

The fix ensures compatibility with binary protocols like Microsoft TTS WebSocket that mix text headers with audio data streams.

Assets 2
Loading