feat(op_crates/web): Add all single byte encodings to TextDecoder

[AliBasicCoder]

i added this ecodings

    ibm866
    iso-8859-2
    iso-8859-3
    iso-8859-4
    iso-8859-5
    iso-8859-6
    iso-8859-7
    iso-8859-8
    iso-8859-10
    iso-8859-13
    iso-8859-14
    iso-8859-15
    iso-8859-16
    koi8-r
    koi8-u
    macintosh
    windows-874
    windows-1250
    windows-1251
    windows-1253
    windows-1254
    windows-1255
    windows-1256
    windows-1257
    windows-1258
    x-mac-cyrillic

NOTES:

• i needed to make code points start from 0 not 128
• i made code points 8 items per line

[CLAassistant]

All committers have signed the CLA.

[ry]

@AliBasicCoder Thanks for the patch. I haven't looked at this in detail yet - but this needs tests before it can be landed.

[kitsonk]

Just to point out, browsers only support encoding UTF-8. https://encoding.spec.whatwg.org/#interface-textencoder so this would be a significant variation from the spec.

[lucacasonato]

Just to point out, browsers only support encoding UTF-8. https://encoding.spec.whatwg.org/#interface-textencoder so this would be a significant variation from the spec.

@kitsonk This is about decoding though. Encoding should stay utf-8 only to be spec compliant, but decoder can have many encodings (Chrome has many encodings for decoding, but only utf-8 for encoding).

[AliBasicCoder]

Just to point out, browsers only support encoding UTF-8. https://encoding.spec.whatwg.org/#interface-textencoder so this would be a significant variation from the spec.

@kitsonk i just added for decoding and didn't touch TextEncoder at all

[kitsonk]

🤦 I can't read. Sorry folks.

[AliBasicCoder]

i have just added all the labels for the encodings

[ry]

@AliBasicCoder I'm all for this but we need tests...

Maybe you can take some tests from here: https://github.com/web-platform-tests/wpt/blob/master/encoding/single-byte-decoder.html

[AliBasicCoder]

so, i found some issuses and fixed theme

and tested it with the following

then made this script

import { TextDecoder } from "./cli/js/web/text_encoding.ts";

// @ts-ignore
window.TextDecoder = TextDecoder;

then bundled it via deno bundle

then converted it to the browser via babel

then downloaded https://github.com/web-platform-tests/wpt

and commneted the XMLHttpRequest test

then included the bundle via a script tag

and this is the result:

[ry]

@AliBasicCoder We need those tests to run continuously so they don't get broken in the future.

[AliBasicCoder]

@AliBasicCoder We need those tests to run continuously so they don't get broken in the future.

so, where to add theme

[ry]

@AliBasicCoder How about in cli/tests/unit/text_encoding_test.ts? Or maybe create a new test module neighboring that one. (If you create a new one, be sure to add it to cli/tests/unit/unit_tests.ts)

Tests can be run with cargo test unit

[AliBasicCoder]

@AliBasicCoder How about in cli/tests/unit/text_encoding_test.ts? Or maybe create a new test module neighboring that one. (If you create a new one, be sure to add it to cli/tests/unit/unit_tests.ts)

Tests can be run with cargo test unit

tests are added now

[ry]

@AliBasicCoder We are missing a CLA from you. It's preventing this PR from going green. I believe you've signed it before - it's the same terms - but we updated it changing my personal name to an entity Deno Land Inc.

[AliBasicCoder]

@AliBasicCoder We are missing a CLA from you. It's preventing this PR from going green. I believe you've signed it before - it's the same terms - but we updated it changing my personal name to an entity Deno Land Inc.

i signed the CLA

[bartlomieju]

@AliBasicCoder I'm interested in landing this PR, could you rebase it and move the logic to op_crates/web/08_text_encoding.js?

[AliBasicCoder]

@AliBasicCoder I'm interested in landing this PR, could you rebase it and move the logic to op_crates/web/08_text_encoding.js?

done

[AliBasicCoder]

the problem is
git status -uno --porcelain --ignore-submodules
prints

 M cli/tests/unit/single_byte_encoding_test.ts
 M op_crates/web/08_text_encoding.js
 M op_crates/web/text_encoding_test.js

[bartlomieju]

LGTM. Huge thanks to @AliBasicCoder and sorry it took so long to land

[linux-china]

@AliBasicCoder any plan to add utf-16le? Now emscripten's js code use utf-16le for decoding. I want to run wasm compiled by emscripten, and utf-16le is a problem. if you can implement it, and it will be very greatful to run emscripten wasm on Deno. thanks.

// Given a pointer 'ptr' to a null-terminated UTF16LE-encoded string in the emscripten HEAP, returns
// a copy of that string as a Javascript String object.

var UTF16Decoder = typeof TextDecoder !== 'undefined' ? new TextDecoder('utf-16le') : undefined;

[AliBasicCoder]

@AliBasicCoder any plan to add utf-16le? Now emscripten's js code use utf-16le for decoding. I want to run wasm compiled by emscripten, and utf-16le is a problem. if you can implement it, and it will be very greatful to run emscripten wasm on Deno. thanks.

// Given a pointer 'ptr' to a null-terminated UTF16LE-encoded string in the emscripten HEAP, returns
// a copy of that string as a Javascript String object.

var UTF16Decoder = typeof TextDecoder !== 'undefined' ? new TextDecoder('utf-16le') : undefined;

see #8108

[xaliphostes]

@AliBasicCoder any plan to add utf-16le? Now emscripten's js code use utf-16le for decoding. I want to run wasm compiled by emscripten, and utf-16le is a problem. if you can implement it, and it will be very greatful to run emscripten wasm on Deno. thanks.

// Given a pointer 'ptr' to a null-terminated UTF16LE-encoded string in the emscripten HEAP, returns
// a copy of that string as a Javascript String object.

var UTF16Decoder = typeof TextDecoder !== 'undefined' ? new TextDecoder('utf-16le') : undefined;

Agreed
Same problem while using wasm with deno.