Bug: No docs explain the value for cache-type-k/v

[phazei]

What happened?

-ctk,  --cache-type-k TYPE              KV cache data type for K (default: f16)
                                        (env: LLAMA_ARG_CACHE_TYPE_K)
-ctv,  --cache-type-v TYPE              KV cache data type for V (default: f16)
                                        (env: LLAMA_ARG_CACHE_TYPE_V)

It gives the default, but what are the other choices? I've been googling and can't find it anywhere.

Name and Version

docker run --rm --runtime nvidia --gpus all ghcr.io/ggerganov/llama.cpp:server-cuda --help

What operating system are you seeing the problem on?

Windows

Relevant log output

-ctk,  --cache-type-k TYPE              KV cache data type for K (default: f16)
                                        (env: LLAMA_ARG_CACHE_TYPE_K)
-ctv,  --cache-type-v TYPE              KV cache data type for V (default: f16)
                                        (env: LLAMA_ARG_CACHE_TYPE_V)

[mirkonasato]

Looking at the source, the types accepted by the kv_cache_type_from_str function are:
"f32" "f16" "bf16" "q8_0" "q4_0" "q4_1" "iq4_nl" "q5_0" "q5_1"

[MichelleTanPY]

Hi, @mirkonasato / @slaren , I'm new to this project and saw that this is labeled with 'good first issue'. I hope it's ok to pick this up as a starting point and have created this PR: #10802 to update the README on this. Can someone review and let me know I had made the right changes? Thanks.

[MichelleTanPY]

Hi @slaren/ @ngxson , is there a way to close this bug? The PR #10802 is closed and superseded by #10806 which got merged. I've noticed a few 'good first issues' tags were already address - which can be confusing for newcomers who wanted to contribute. Is there anyway I can help out? Thanks.

[ngxson]

I didn't know we had an issue for that. For future contributors, please make sure you link the issue to the PR, so github can close it once PR is merged.

[ngxson]

Also, some "good first issue" are pretty much outdated (i.e. we no longer need it after some refactoring). It's best to leave a comment before you gonna take the task.

[MichelleTanPY]

@ngxson , thanks for explaining. Will definitely link a PR for next time. I'll keep an eye on something which I can try to do my first contribution to in this project. 😊
One last question, is this something as simple as updating the has_new_line & model attributes (I think you made the original changes)? It had the same 'good first issue' tag as well: #10728