Fixed too sure .unwrap in tokenizer.rs

[agbaroni]

I propose the change because I had this problem when I did cargo test:

[....]$ RUST_BACKTRACE=1 cargo test
     Running target/debug/html5ever-ebbb862bf0664388

running 20 tests
....................
test result: ok. 20 passed; 0 failed; 0 ignored; 0 measured

     Running target/debug/serializer-bd6e4b6794d0e313

running 40 tests
........................................
test result: ok. 40 passed; 0 failed; 0 ignored; 0 measured

     Running target/debug/tokenizer-9fee19ffe67616b5
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { repr: Os { code: 2, message: "No such file or directory" } }', ../src/libcore/result.rs:788
stack backtrace:
   1:        0x10f9e54bb - std::sys::backtrace::tracing::imp::write::h46e546df6e4e4fe6
   2:        0x10f9e775a - std::panicking::default_hook::_$u7b$$u7b$closure$u7d$$u7d$::h077deeda8b799591
   3:        0x10f9e738a - std::panicking::default_hook::heb8b6fd640571a4f
   4:        0x10f9dc178 - std::panicking::rust_panic_with_hook::hd7b83626099d3416
   5:        0x10f9e7d36 - std::panicking::begin_panic::h941ea76fc945d925
   6:        0x10f9dcdb8 - std::panicking::begin_panic_fmt::h30280d4dd3f149f5
   7:        0x10f9e798f - rust_begin_unwind
   8:        0x10fa0d570 - core::panicking::panic_fmt::h2d3cc8234dde51b4
   9:        0x10f903469 - core::result::unwrap_failed::h77e8f7274d900d66
  10:        0x10f902d9a - _<core..result..Result<T, E>>::unwrap::h832a73c540c5d590
  11:        0x10f902782 - tokenizer::foreach_html5lib_test::foreach_html5lib_test::h2ce9421430543c73
  12:        0x10f901f8d - tokenizer::tests::h3cbd3447622b6aa9
  13:        0x10f905c2f - tokenizer::main::h4149a347b29284d8
  14:        0x10f9e6f4d - std::panicking::try::call::hca715a47aa047c49
  15:        0x10f9e7e0b - __rust_try
  16:        0x10f9e7da5 - __rust_maybe_catch_panic
  17:        0x10f9e6d71 - std::rt::lang_start::h162055cb2e4b9fe7
  18:        0x10f907399 - main
error: test failed

---

This change is

[nox]

This makes the test pass instead of crash if read_dir returns an error, no?

[agbaroni]

Yes ..... I think that if someone download this project alone and he/she wants run the tests it's better giving OK if he/she cannot list html5lib-tests (I have an empty folder indeed).

[nox]

If you don't have html5lib-tests, you ran almost no test.

Your problem is fixed by doing the things written in the README.

[agbaroni]

Ok .... sorry, I thought that it could be useful :-(

[SimonSapin]

We could have the test harness run the appropriate git submodule command when it starts.

[nox]

That would mean fetching things when running cargo test though, that sounds like a weird thing to do.

[SimonSapin]

I don’t think it’s that weird, and I think it’s better than having someone seeing failing tests and not knowing why. (We can’t assume everyone reads the README.)

But yeah, I agree that tests should fail if the test data is not available.

[nox]

I don’t think it’s that weird, and I think it’s better than having someone seeing failing tests and not knowing why. (We can’t assume everyone reads the README.)

We can make it print the git submodule command if it sees that the directory is empty.

[agbaroni]

@SimonSapin I made an error skipping that part of README.md ..... maybe some message it would be useful as @nox said

[SimonSapin]

Yes, a better message would be useful. But I think the test harness running the appropriate command without asking you to do it would be even more helpful.

[agbaroni]

And if there was an update snapshot of test harness directly in html5ever? There are licensing problems?

[SimonSapin]

I don’t understand this question but by test harness I mean Rust code in this repository, not anything in the html5lib-tests repository. Git submodules record a specific commit hash that doesn’t change automatically. And finally, html5lib-tests is distributed under an MIT license so I don’t expect any licensing problem.

[jdm]

@bors-servo: r+
Thanks for making that change!

[bors-servo]

📌 Commit ee636e1 has been approved by jdm

[bors-servo]

⚡ Test exempted - status