[Project idea] Chimera: A dynamic LoRA inference server that can do amazing things combined with Heretic

[p-e-w]

The Chimera Inference Server: A companion for Heretic

This idea has been ripening in my mind for several weeks, and I mentioned it informally on Discord a few times. It's probably a good time now to write it down before I forget the details.

Ingredients

• The base model (no processing)
• The Heretic abliteration as a LoRA (Heretic has had the ability to export adapters since 1.4)
• A Jev-type decision model capable of noul (boolean confidence) and score decisions
• An inference engine that can load and dynamically enable/disable LoRAs (which I believe is supported by vLLM, among others)

Setup

Chimera offers a standard OpenAI-compatible inference API. The magic is in its configuration. Here are two examples:

Only suppress refusals if the model is likely to refuse

model = "/models/Qwen3.8-27B"

[[adapter]]
path = "/models/loras/Qwen3.8-27B-heretic-lora"
question = "Is this instruction potentially harmful?"
type = "noul"
threshold = 0.3

When a prompt arrives over the API, the question is sent to the decision model along with the prompt as a "noul" decision, and if the resulting noul value from the decision model response exceeds threshold, the adapter is applied.

In other words, the Heretic LoRA is only activated when necessary. This keeps the full intelligence of the model intact for harmless prompts, without the user ever having to think about the difference.

Modulate the level of humor in response to the situation

Heretic can be used to suppress humor in a model. A hypothetical Chimera configuration leveraging a "nohumor" LoRA might look like this:

model = "/models/gemma-4-26B-A4B"

[[adapter]]
path = "/models/loras/gemma-4-26B-A4B-heretic-lora-nohumor"
question = "Rate how appropriate it is to respond with humor to this."
type = "score"
legend = [
  "The more humor the better",
  "Some humor can be used",
  "A neutral response is best",
  "Humor is completely inappropriate",
]
score_to_weight_transform = [0.666, -1.0]

This is a much more powerful setup: When a prompt arrives over the API, the question and legend are sent to the decision model along with the prompt as a "score" decision. The resulting score value from the decision model response is put through the linear function defined by score_to_weight_transform (in this case, 0.666 * score - 1.0). The resulting value is used as the weight of the LoRA.

In other words, the degree of humor suppression (or amplification) can be dynamically controlled by a decision model. This enables all kinds of new steering mechanisms.

Of course, it would be possible to configure any number of adapters this way. Note that decision models can return responses in milliseconds, and toggling a loaded LoRA is effectively instantaneous, so all of this carries essentially no performance overhead.

See https://github.com/heterodoxin/AbliteratedOutTheBox for an existing inference engine that determines per-prompt whether to abliterate. Note that while that project is innovative, I don't consider it the right architecture as IMO, abliteration is a task that you want to do as well as possible once, and then use the result many times. With the Chimera system, you can use Heretic to do a full optimization run to obtain a high-quality adapter, and then just toggle it on and off as needed.