thread 'main' panicked at 'byte index 1 is not a char boundary

[flodolo]

I'm running mdBook 0.3 on macOS, and for some reasons it fails to build one of the file containing the ë character. The thing I don't understand is that I have other files with that character, and they build just fine.

Minimal file content in README.md

# Test

spatiëring.

Error

2019-07-04 11:52:26 [INFO] (mdbook::book): Book building has started
2019-07-04 11:52:26 [INFO] (mdbook::book): Running the html backend
thread 'main' panicked at 'byte index 1 is not a char boundary; it is inside 'ë' (bytes 0..2) of `ër`', src/libcore/str/mod.rs:2027:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

[flodolo]

Additional note: it doesn't seem to happen if the content of the file is just

spatiëring.

[zbraniecki]

Any chance you can run it with RUST_BACKTRACE=1 to get the trace?

[flodolo]

thread 'main' panicked at 'byte index 1 is not a char boundary; it is inside 'ë' (bytes 0..2) of `ër`', src/libcore/str/mod.rs:2027:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
   1: std::sys_common::backtrace::_print
   2: std::panicking::default_hook::{{closure}}
   3: std::panicking::default_hook
   4: std::panicking::rust_panic_with_hook
   5: std::panicking::continue_panic_fmt
   6: rust_begin_unwind
   7: core::panicking::panic_fmt
   8: core::str::slice_error_fail
   9: core::str::traits::<impl core::slice::SliceIndex<str> for core::ops::range::Range<usize>>::index::{{closure}}
  10: elasticlunr::lang::en::stemmer
  11: elasticlunr::pipeline::Pipeline::run
  12: elasticlunr::Index::add_doc
  13: mdbook::renderer::html_handlebars::search::add_doc
  14: mdbook::renderer::html_handlebars::search::create_files
  15: <mdbook::renderer::html_handlebars::hbs_renderer::HtmlHandlebars as mdbook::renderer::Renderer>::render
  16: mdbook::book::MDBook::build
  17: mdbook::cmd::build::execute
  18: mdbook::main
  19: std::rt::lang_start::{{closure}}
  20: std::panicking::try::do_call
  21: __rust_maybe_catch_panic
  22: std::rt::lang_start_internal
  23: main

[zbraniecki]

Ok, seems to be a bug in https://github.com/mattico/elasticlunr-rs

Basically, it doesn't expect english to use non-latin characters.

The kind of code that triggers the crash is https://github.com/mattico/elasticlunr-rs/blob/master/src/lang/en.rs#L213

It should instead use a safe slicing, and likely a different regexp.

[flodolo]

Thanks for digging. I assume the best next step is to report the issue in the other repository?

[bwinton]

I've made a pull request to elasticlunr-rs that looks like it'll fix this issue.

[Edited to add: Oh no, there are more… I'll have to update my PR.]
[Further edit: Wait, no, those are all indexing into a Captures struct. I think my PR might be fine…]

[zbraniecki]

Thanks Blake! That looks like a good fix.

[bwinton]

And the PR has landed, so hopefully this will be fixed when we pull in the next version of that library! 😃

[EUSKR]

I am facing the same issue now with the latest version of mdBook, running on macOS, text is bilingual and includes ё characters too:

❯ mdbook serve
[INFO] (mdbook::book): Book building has started
[INFO] (mdbook::book): Running the html backend

thread 'main' panicked at /Users/xxxxx/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/elasticlunr-rs-3.0.3/src/lang/en.rs:518:47:
index out of bounds: the len is 3 but the index is 18446744073709551615
stack backtrace:
   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::panic_bounds_check
   3: <elasticlunr::lang::en::Stemmer as elasticlunr::pipeline::PipelineFn>::filter
   4: elasticlunr::pipeline::Pipeline::run
   5: elasticlunr::Index::add_doc
   6: mdbook::renderer::html_handlebars::search::add_doc
   7: mdbook::renderer::html_handlebars::search::create_files
   8: <mdbook::renderer::html_handlebars::hbs_renderer::HtmlHandlebars as mdbook::renderer::Renderer>::render
   9: mdbook::book::MDBook::execute_build_process
  10: mdbook::book::MDBook::build
  11: mdbook::cmd::serve::execute
  12: mdbook::main

[szabgab]

@EUSKR , thanks for the report. As I could not reproduce it locally, could you share a minimal example of the files that trigger this problem?