abliterated/community models work?

[carcifosupr3mo]

Hi! I have a quick question about llmfit's model recommendations.

Does llmfit recommend only official/base/instruct models, or does its model catalog also include community fine-tunes such as abliterated models?

For example, if I provide my hardware specifications (24 GB VRAM, 64 GB RAM), can llmfit rank compatible abliterated/community models alongside the official ones, or do I need to add those models manually?

I'm mainly interested in understanding how broad the model catalog is and whether community variants are automatically discovered and evaluated.

Thanks!

[AlexsJones]

We just need to add them in, but yes as long as we have the parameters there is no problem

[AlexsJones]

Jumping in alongside Alex's answer with one practical note: community fine-tunes do get ranked once they are in the catalog. The catalog mixes curated entries with models discovered from Hugging Face trending via llmfit update, so an abliterated variant with a known parameter count will show up next to the official ones for your 24 GB VRAM and 64 GB RAM setup. If a specific variant is missing, running llmfit update is the first thing to try.

[carcifosupr3mo]

Thank you very much <3

[behrnt-slatgng]

Disclosure: I build Grunz, a hosted chat + coding agent that runs abliterated models, so I've spent a while on the difference between an abliterated variant and its base. One practical note to add to the answers above.

Memory-wise an abliterated variant fits exactly like its base, since the parameter count and architecture are the same, so llmfit's ranking is right on that axis. Quality at a given quant level is where they differ. Abliteration tends to hurt instruction following and output format (stop sequences, clean JSON, tool-call syntax) before it hurts knowledge, and quantization seems to stack on top of that. So a Q4 of an abliterated model can be noticeably worse at structured output than the same Q4 of the base, even though chat feels fine.

With 24 GB of VRAM, if you plan to use one for chat, the same pick as the base is fine. If you plan to use one behind a coding agent or anything that parses output, go one quant level higher than you would for the base (Q5_K_M or Q6_K instead of Q4_K_M), or pick a smaller model at higher precision. Checking it with a few strict-JSON prompts before trusting it with tool calls is also worth doing.

@AlexsJones one idea, if it's useful: since community variants now show up in the catalog, tagging a model as abliterated/uncensored and raising its recommended quant floor for agent use cases would fit nicely with #914.

[AlexsJones]

Thanks for the disclosure and the practical note. The point that abliteration hits stop sequences and tool-call syntax before it hits knowledge is exactly what matters when someone picks a model for an agent.

The tagging idea fits best on #914, which is about recommendations for tool-using coding agents. If you add it there with a couple of models where you've seen the structured-output drop, it'll sit next to the other agent-use requirements when that work gets scoped.

[behrnt-slatgng]

cool, dropped it on #914. kept it general and didn't name specific models, it really depends on who made the variant and how hard they went with the edit