Bug: llama-perplexity error using multiple-choice binary data

[fedric95]

What happened?

"The multiple choice evaluation has been broken in llama.cpp via commit 6ff1398.

The multiple choice evaluation uses binary data stored in params.prompt. Commit 6ff1398 adds prompt escape character processing, which modifies the binary data and renders it unusable. To preserve whatever utility 6ff1398 might have added, we add a flag indicating if the data stored in params.prompt is binary and, if so, avoid the escape processing." @ikawrakow

@ikawrakow solved the problem in his llama.cpp fork in the following PR: ikawrakow/ik_llama.cpp#33

Name and Version

I tested the issue with the docker release of llama.cpp:

ghcr.io/ggerganov/llama.cpp:full-cuda--b1-98a532d

What operating system are you seeing the problem on?

Linux

Relevant log output

No response

[kallewoof]

@fedric95 Can you double check whether this is an issue still? I initially thought I ran into this but after fixing my launch arguments I am able to use llama-perplexity as is.