Abliterated models lose format compliance before they lose knowledge, and perplexity will not warn you

[behrnt-slatgng]

Disclosure, since it is the reason I have the data: I run abliterated open weights in production behind a hosted chat and coding agent (Grunz). That is not what GPT4All is for and I am not suggesting it should be — this is a pattern about the weights, which applies identically whether you run them on your own machine or not, and this community runs more of these models than almost anywhere else.

Abliteration costs obedience before it costs knowledge

If you download an uncensored or abliterated variant — huihui, Heretic, the various -abliterated uploads — the degradation order is consistent and not the one most people expect:

1. instruction-following and output-format adherence degrade first
2. knowledge and reasoning degrade later

The model still knows the material. What it stops doing reliably is respecting the instruct template, honouring stop sequences, and holding a system-prompt constraint across a long conversation.

The reason this is worth writing down is that the normal way people evaluate these variants will not detect it. You download it, chat with it, confirm it does not refuse, confirm the prose is good — and it passes, often looking better than the base model. Then you point it at anything that parses its output (a LocalDocs workflow, a template, a tool integration, a script consuming the API server) and the failure rate is materially higher than the base model at the same quant.

A useful test, if you are comparing two variants: send N requests with an explicit output contract — "reply with only a JSON object containing these keys", or a required stop token — and score the fraction that come back well-formed, independently of whether the answer is correct. Two models that feel identical in a chat window will often separate immediately.

Practical quant guidance

Anecdotally, ablation damage and quantization damage seem to compound rather than sit beside each other: an ablated model degrades faster down a quant ladder than its base does, measured on format compliance rather than perplexity.

So:

• Stay at Q6_K or above on smaller ablated models if you can afford the RAM. A Q8_0 of a 4B is only around 4.3GB, so there is not much to save by going lower.
• If you do go below Q6 on an ablated model, an imatrix quant is worth more than it is on undisturbed weights — the importance matrix matters more when the weights have already been perturbed.
• Do not use perplexity to decide. On a generic corpus it barely moves while compliance falls off a cliff, so it will tell you everything is fine right up until nothing parses.

I have not run this controlled — it is high volume rather than clean — so I would genuinely like to be corrected by anyone who has measured it properly. If someone here has compliance-rate numbers across a quant ladder for an ablated model and its base, that would be more useful than anything I can offer.