Sometimes crashes with UTF8 error

[kimtore]

Some prompts fail with:

thread 'main' panicked at 'called Result::unwrap() on an Err value: Utf8Error { valid_up_to: 0, error_len: None }', /home/kimt/.cargo/registry/src/index.crates.io-6f17d22bba15001f/llama_cpp_rs-0.3.0/src/lib.rs:528:46

I'm using this model: https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GGUF

Anything I can do to try to fix this problem?

[markcda]

Yes, see this commit in fork.

[kimtore]

@markcda thanks for the workaround. Are some words now going to be dropped from the model's outputif they contain illegal UTF-8 characters?

[markcda]

Are some words now going to be dropped from the model's outputif they contain illegal UTF-8 characters?

The above commit only made specific deployment of the bug. Roughly speaking, the response will be reset if it contains at least one incorrect character. But the application will not crash. To solve your specific problem, I advise you to read the following post.