Cookbook Model List

[pewdiepie-archdaemon]

Cookbook is a curated list of models.

If you have a request for models missing, please leave a comment below:

• Hf Link
• What hardware you ran on (optional)
• What engine (llama.cpp, vllm, etc)
• Reason to add

Thank you

[DeanLemans]

i think one of the fine tuned 'uncencored" ai should be added:
one of the dphn models would be great: https://huggingface.co/dphn
i personally use:

• https://huggingface.co/dphn/dolphin-2.9-llama3-8b
• https://huggingface.co/dphn/Dolphin-X1-Trinity-Nano

[CommanderTurtle]

I have just updated a solid 5090 setup for one of these, actually.

Would love to see further integration or (maybe if you know) LiteLLM filtering for said models. Anything honestly at this point to fix-up and filter runners that contain "unhinged" outputs. I was using DavidAUs' for a time, until eventually switching over to AEON's nvfp4 instructs for very big context that also worked with toolcalls (although, they were not as interesting for creative writing, like DavidAUs' reasoning window, which was indeed quite good)

If you have a 5090, 4090, or 5080, blackwell, a solid choice for these types of models is my repo where I seem to have gotten modelopt turbo running well on consumer hardware. That is, unquanted "but nvfp4" vision-capable Google Gemma4 with both chat templates preserved from David's "instruct" (i.e. toolcalling) and "non instruct" which just has an insanely detailed reasoning block of the model re-reasoning with itself. Around 170tok/s on Gemma A4B MoE model at four parallel 65k context chats capable with this kind of setup.

https://huggingface.co/sHEL1562/gemma-4-A4B-it-MoE-HERETIC-nvfp4-blackwell

[nutter77-fossmc]

I think lfm2.5 models should be added

its better then most qwen models around its weights

also running on m1 air base model 8gb ram 7gpu 8cpu 256 gb the very budget friendly MacBook Air m1 sold from Walmart

[pewdiepie-archdaemon]

Can you link so I know which one you mean, updated the original post

[nutter77-fossmc]

Yeah sure

I’ve got a few more I can link to they are people / orgs /authors tho
https://huggingface.co/Crownelius
https://huggingface.co/DavidAU
https://huggingface.co/Goekdeniz-Guelmez
https://huggingface.co/LiquidAI

Bit to sort through

also if it helps I think mobile apps might be cool for this

[RenanOSX]

• https://huggingface.co/HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive (Q4_K_M)
• RTX 4060 8Gb + 32 DDR4 RAM (It bleeds a lot but I can run with a stable 22 tokens/s at 8k context window) [Fit: TOO TIGHT]
• llama.cpp
• Qwen performance but allows users to study cybersecurity and RE without refusals from default models (Good for coding and dissecting samples)

[Whitefirelion]

Would also be great if we got the uncensored qwen3.5-9b model, works great on my older harder
https://huggingface.co/HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive (Q4_K_M)

[SPOOKEXE]

Qwopus models are really nice, qwen trained on opus traces;
https://huggingface.co/Jackrong/Qwopus3.6-27B-v2-MTP-GGUF
https://huggingface.co/Jackrong/Qwopus3.5-9B-Coder
https://huggingface.co/Jackrong/collections

[sacb0y]

Good uncensored Gemma 4 model that won't stop working cause it encounters spicy words >_>
https://huggingface.co/zerofata/G4-MeroMero-26B-A4B-gguf

[josephspurrier]

Gemma 4 12B

• HF Link: https://huggingface.co/google/gemma-4-12b
• Hardware: Runs on consumer laptops with 16GB VRAM or unified memory
• Engine: Ollama, LM Studio, vLLM
• Reason to add: Google's first mid-sized model with native audio inputs alongside text and vision. Encoder-free architecture (no separate vision/audio encoders) reduces latency. Performance nears their larger 27B MoE model at less than half the memory footprint. Includes Multi-Token Prediction drafters for faster inference. Apache 2.0 license.

Blog post: https://blog.google/innovation-and-ai/technology/developers-tools/introducing-gemma-4-12B/

[AbishekKoushik]

Gemma 4 12B

1. HF LINK: google/gemma-4-12B-it-qat-q4_0-gguf
2. Hardware: Runs on comsumer gpu with 12GB VRAM

[DericHuynh]

Qwen3.5 Mixture of Quants + Multi-Token-Prediction

1. HF LINK: w-ahmad/Qwen3.5-9B-GGUF-MoQ-MTP
2. Hardware: Runs on consumer GPU with 8GB VRAM

Specifically moq-3.6, runs on 8GB Vram, fast generation, modern model with quantization that is 97% accuracy against baseline. A generally good model for constrained hardware.

[T-Bozz28]

HI,
Anyone have an idea why i can't see models?
"Couldn't scan device" error

[deucebucket]

https://huggingface.co/deucebucket/Qwen3.6-35B-A3B-Cerebellum-GGUF
Cerebellum v3 Qwen3.6-35B-A3B-Cerebellum-v3.gguf, 11 GB

https://huggingface.co/deucebucket/Gemma-4-26B-A4B-it-Heretic-Cerebellum-GGUF
gemma-4-26B-A4B-it-heretic-cerebellum-v1.1-templatefix.gguf, 11.7 G

https://huggingface.co/deucebucket/Qwen3.6-27B-Cerebellum-GGUF
Ablation-informed mixed-precision quantization of, 12 GB file size, 7.034 perplexity, 181 per-tensor quant overrides.

[rendika695]

put uncensored to model filter