Implement TextEncoder (fixes #4768).

[yodalee]

This PR need modification, but I think some review is fine.
I run the wpt test, there are some issues:

• The textencoder constructor test will fail, because it return IndexError instead of RangeError in spec.
• Other fail I cannot understand why....

[highfive]

Warning

• These commits modify unsafe code. Please review it carefully!

[hoppipolla-critic-bot]

Critic review: https://critic.hoppipolla.co.uk/r/4064

This is an external review system which you may optionally use for the code review of your pull request.

In order to help critic track your changes, please do not make in-place history rewrites (e.g. via git rebase -i or git commit --amend) when updating this pull request.

[jdm]

What are the other failures you see?

[yodalee]

hmm, there are so many, most of them are decoder fail. Here is the link to log summary.
https://dl.dropboxusercontent.com/u/3192346/log.txt

• There is other error like USVString handling, since USVString is not supported now.
• Another error is Non-UTF encodings supported only for decode, not encode, which... is weird. I don't thine I support Non-UTF encoding.
  btw, can I run wpt-test encoding section only, run all wpt-test really takes many time.

[jdm]

Yes, ./mach test tests/wpt/web-platform-tests/path/to/dir --processes=N should work.

[jdm]

Ah. My recommendation with the failures is to make a PR that enables the encoding directory and following the steps at http://mxr.mozilla.org/servo/source/tests/wpt/README.md#71 to save the default failures that occur without any other changes. Then you can apply your other changes on top of that and see which failures go away.

[yodalee]

I see, let me try this and give you a clear log.

[jdm]

I've left comments on Critic :)

[yodalee]

I rebase the branch to newest. Also, I found that only 4 tests being passed after the textencoder added T_T

[jdm]

Yeah, I suspect the tests are relying on things like document.inputEncoding and TextDecoder, unfortunately.

[yodalee]

rebase to newest master and solve datatype issue. Though I think it still cast usize to u32 and causing some loss.

[jdm]

I believe there's just one nit left here, based on the Critic review - there should be a space before the { on struct TextEncoder{.

[yodalee]

I didn't notice the new issue, should be fixed now.

[yodalee]

Done. I squash it into three commits: constructor, implement method, and enable testings

[jdm]

r+

[jdm]

@bors: retry

[jdm]

Thanks!

[bors-servo]

saw approval from jdm
at yodalee@314bf79

[bors-servo]

merging yodalee/servo/issue4768-textencoder = 314bf79 into auto

[bors-servo]

yodalee/servo/issue4768-textencoder = 314bf79 merged ok, testing candidate = db21010

[bors-servo]

some tests failed:
failure: http://build.servo.org/builders/mac2/builds/1227

[bors-servo]

saw approval from jdm
at yodalee@314bf79

[bors-servo]

merging yodalee/servo/issue4768-textencoder = 314bf79 into auto

[bors-servo]

yodalee/servo/issue4768-textencoder = 314bf79 merged ok, testing candidate = 0b2d983

[bors-servo]

some tests failed:
failure: http://build.servo.org/builders/mac2/builds/1228

[yodalee]

I cannot reproduce the crash result of /encoding/single-byte-decoder.html, it end normally at my local test OAO

[jdm]

Yeah, that looks kind of like resource exhaustion on the builder machine. It's weird that it's happening for this test in particular :/

[yodalee]

@jdm
on my local computer, I change the async_test into test in single-byte-decoder.html. Passed subtest increase to 334 from 158. Maybe we can try this on remote test?

[Ms2ger]

You mean you made the test not test anything.

[yodalee]

oops, I think I misunderstand the meaning of async_test and test.

[jdm]

The good news is that I can reproduce the failure locally:

godot2:servo jdm$ ./mach test tests/wpt/web-platform-tests/encoding/single-byte-decoder.html
 0:03.60 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) Full command: /Users/jdm/src/servo/components/servo/target/servo --cpu --hard-fail -z http://localhost:8000/encoding/single-byte-decoder.html
(pid:72816) "thread 'PaintTask' panicked at 'failed to spawn native thread: 35', /Users/larsberg/rust/src/libstd/sys/unix/thread.rs:229"
 0:03.60 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread 'LayoutTask' panicked at 'failed to spawn native thread: 35', /Users/larsberg/rust/src/libstd/sys/unix/thread.rs:229"
 0:03.60 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread 'PaintTask' panicked at 'failed to spawn native thread: 35', /Users/larsberg/rust/src/libstd/sys/unix/thread.rs:229"
 0:03.60 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread 'LayoutTask' panicked at 'failed to spawn native thread: 35', /Users/larsberg/rust/src/libstd/sys/unix/thread.rs:229"
 0:03.60 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread 'Constellation' panicked at 'failed to spawn native thread: 35', /Users/larsberg/rust/src/libstd/sys/unix/thread.rs:229"
 0:03.60 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread 'PaintTask' panicked at 'failed to spawn native thread: 35', /Users/larsberg/rust/src/libstd/sys/unix/thread.rs:229"
 0:03.60 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread 'ScriptTask' panicked at 'failed to spawn native thread: 35', /Users/larsberg/rust/src/libstd/sys/unix/thread.rs:229"
 0:03.61 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread 'LayoutWorker worker 1/6' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', /Users/larsberg/rust/src/libcore/result.rs:743"
 0:03.61 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread 'LayoutWorker worker 2/6' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', /Users/larsberg/rust/src/libcore/result.rs:743"
 0:03.61 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread 'LayoutWorker worker 3/6' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', /Users/larsberg/rust/src/libcore/result.rs:743"
 0:03.61 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', /Users/larsberg/rust/src/libcore/result.rs:743"
 0:03.61 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', /Users/larsberg/rust/src/libcore/result.rs:743"
 0:03.61 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', /Users/larsberg/rust/src/libcore/result.rs:743"
 0:03.61 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', /Users/larsberg/rust/src/libcore/result.rs:743"
 0:03.61 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', /Users/larsberg/rust/src/libcore/result.rs:743"
 0:03.61 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "thread '<unnamed>thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', /Users/larsberg/rust/src/libcore/result.rs:743"
 0:03.61 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', /Users/larsberg/rust/src/libcore/result.rs:743"
 0:03.61 PROCESS_OUTPUT: Thread-TestrunnerManager-2 (pid:72816) "fatal runtime error: Could not unwind stack, error = 5"

[jdm]

Wow, that test actually does spawn an incredible number of tasks for some reason.

[jdm]

So it's the async_test that creates a lot of iframes that is doing us in, specifically. For the purposes of this PR, I suggest just annotating that a crash is expected from this test, and we can file another bug for improving our thread creation eagerness and making this test not crash.

[metajack]

@mbrubeck Assigned to you to figure out next steps.

[mbrubeck]

I moved these patches to a new PR #5469 and added the suggested test change, so we can hopefully land them. Thank you, @yodalee!