Skip to content

v1.0.0-alpha.2

Pre-release
Pre-release

Choose a tag to compare

@bjohansebas bjohansebas released this 03 Jul 19:20
Immutable release. Only release title and notes can be modified.
6980d9e

⚠️ Breaking changes

  • Remove support for Node <22 - by @bjohansebas in #396

    Node.js versions prior to 22 are no longer supported. This allows us to migrate to ESM and rely on require(esm), which does not work correctly on earlier versions.

  • Make the UTF-16 decoder WHATWG-conformant by @bjohansebas in #402

    UTF-16LE/BE (and the ucs2/ucs-2 aliases) now decode through the standard TextDecoder, matching the Encoding Standard's shared UTF-16 decoder. As a result, unpaired/invalid surrogates and a trailing odd byte are replaced with U+FFFD instead of being passed through, and the Node and Web backends now behave identically. The decoder also no longer depends on the backend.

    The utf-16 label still auto-detects endianness (UTF-16LE vs UTF-16BE) from the BOM and a space-based heuristic, defaulting to UTF-16LE. This is kept as an iconv-lite extension even though it is not spec-compliant — the Encoding Standard maps the utf-16 label directly to UTF-16LE.

  • Reject encoding labels with forbidden characters, per WHATWG - by @bjohansebas in #403

    Following the Encoding Standard's label-trimming rules, only ASCII whitespace (tab, LF, FF, CR, space) may surround an encoding label. Labels wrapped in other control or separator characters — NUL, vertical tab, NBSP, or the line/paragraph separators — are now rejected instead of being silently stripped and accepted. Punctuation such as dashes and underscores within an otherwise-valid label is still normalized.

  • Make the UTF-7 and UTF-7-IMAP codecs RFC 2152 conformant - by @bjohansebas in #406

    When decoding, ill-formed input — an incomplete code unit, non-zero Base64 padding bits, a shift-in ("+"/"&") not followed by Base64 or "-", or a non-ASCII byte outside a shifted run — is now replaced with U+FFFD instead of being passed through (lone surrogates still pass through as raw 16-bit code units). When encoding UTF-7, the optional "Set O" punctuation is now left as direct ASCII, so the output differs byte-for-byte for those characters, though it still decodes to the same text.

  • Make the UTF-32 codecs strict and browser-native - by @bjohansebas in #407

    UTF-32LE/BE and the auto-detecting utf-32 codec now decode strictly per the Unicode Standard: a code unit that is a surrogate code point (U+D800–U+DFFF), is above U+10FFFF, or is a truncated trailing code unit is replaced with U+FFFD instead of being passed through. Encoding likewise replaces a lone (unpaired) surrogate with U+FFFD. The opt-in { fatal: true } decoding option makes ill-formed input throw instead. The codecs no longer use the Node Buffer internally, so they also work on the Web backend (browsers), like UTF-16. The internal _utf32 codec name (a private, undocumented implementation detail of the old codec-options indirection) was removed.

🚀 Improvements

  • Speed up the UTF-32 codecs - by @bjohansebas in #407

    UTF-32 decoding and encoding are noticeably faster, and encoding no longer allocates a Node Buffer.

  • Recognize more WHATWG encoding labels - by @bjohansebas in #403

    Added the WHATWG label aliases for encodings iconv-lite already implements: unicode/csunicode/iso-10646-ucs-2/unicodefeff (UTF-16LE) and unicodefffe (UTF-16BE); x-cp1250x-cp1258 (windows-1250–1258); dos-874; koi/koi8 (KOI8-R); x-mac-cyrillic/x-mac-ukrainian/x-mac-roman; x-euc-jp/cseucpkdfmtjapanese (EUC-JP); the iso-8859-6/iso-8859-8 -e/-i and visual/logical variants; csisolatin9; sun_eu_greek; unicode-2.0-utf-8; and more. (Labels whose encoding iconv-lite does not implement, such as iso-2022-jp and x-user-defined, remain unsupported.)

  • Add an opt-in fatal decoding option - by @bjohansebas in #402

    Passing { fatal: true } to iconv.decode(...) makes the TextDecoder-backed encodings (UTF-8, UTF-16LE/BE) throw on invalid input, per the WHATWG Encoding Standard, instead of replacing it with U+FFFD. The default stays lenient (replacement).

New Contributors

Full Changelog: v1.0.0-alpha.1...v1.0.0-alpha.2