Failure to load Kimi-K2-Instruct-0905-GGUF

[ShadowSocks12]

Greetings,

I downloaded the Kimi-K2-Instruct-0905-GGUF Q6_K from here https://huggingface.co/bartowski/moonshotai_Kimi-K2-Instruct-0905-GGUF
and I tried to load the model using Koboldcpp v 1.101.1 but I get error

I tried CUDA, CLBlast also GPU layers = 28 , FlashAttention Enabled , Context size = 14336

and I get this:

llama_model_load_from_file_impl: failed to load model
Traceback (most recent call last):
File "koboldcpp.py", line 7989, in
main(launch_args=parser. parse_args() , default_args=parser. parse_args([]))
File "koboldcpp.py", line 6989, in main
kcpp_main_process(args,global_memory, using_gui_launcher)
File "koboldcpp.py", line 7443, in kcpp_main_process
loadok = load_model (modelname)
File "koboldcpp.py", line 1467, in load_model
ret = handle. load_model(inputs)
OSError: exception: access violation reading 0x0000000000000004
[18416] Failed to execute script 'koboldcpp' due to unhandled exception!

[LostRuins]

What are your hardware specs? That model is huge, i doubt you have the resources to run it

[ShadowSocks12]

AMD Ryzen AI 9 HX 370 w/ Radeon 890M
NVIDIA GeForce RTX 5070 Ti Laptop GPU
Ram = 32 GB

[LostRuins]

Yeah that will not fit. Fyi this model needs hundreds of GB of memory

[ShadowSocks12]

Thank You