Skip to content

Proposal: first-class user-defined MoA preset profiles with live switching #57018

Description

@AbertKiwinski

Summary

I would like to propose first-class support for user-defined MoA preset profiles in Hermes.

This is not meant as a bug report. It is a feature proposal based on a working local prototype I have been using: named MoA presets such as full, workhorse, and ultra, where each preset defines its own reference models, aggregator model, temperatures, and switching behavior.

The goal is to make MoA practical as a normal user-facing routing mode, not only as a hardcoded or one-shot advanced tool.

Related prior work/proposal: #38952. This issue is a more concrete follow-up focused on named presets, live switching, and CLI UX based on an implemented local workflow.

Motivation

Different tasks need different MoA ensembles:

  • daily work should use a cheaper/faster profile,
  • code review may need stronger reasoning but not the most expensive setup,
  • very hard architecture/research/debugging tasks may deserve a high-quality ensemble,
  • users should be able to tune MoA without editing source files.

In my local setup this has been useful because I can switch the active MoA profile depending on the task instead of treating MoA as a single hardcoded mode.

Example: local ultra preset

One of my local presets is intended for the most demanding tasks:

moa:
  default_preset: ultra
  active_preset: ultra
  presets:
    ultra:
      enabled: true
      reference_models:
        - provider: deepseek
          model: deepseek-v4-pro
        - provider: zai
          model: glm-5.2
      aggregator:
        provider: kimi-coding
        model: kimi-k2.7
      reference_temperature: 0.55
      aggregator_temperature: 0.35

The idea is simple:

  • DeepSeek Pro and GLM 5.2 provide independent reference reasoning,
  • Kimi K2.7 acts as the aggregator/synthesizer,
  • the preset has a stable name (ultra) that can be selected from the CLI or used as the active MoA model.

Other users could define their own presets, for example budget, review, research, math, coding, etc.

Proposed config shape

A first-class config section could look like this:

moa:
  default_preset: workhorse
  active_preset: workhorse
  presets:
    workhorse:
      enabled: true
      reference_models:
        - provider: deepseek
          model: deepseek-v4-pro
        - provider: openai-codex
          model: gpt-5.5
      aggregator:
        provider: kimi-coding
        model: kimi-k2.7
      reference_temperature: 0.5
      aggregator_temperature: 0.3

    ultra:
      enabled: true
      reference_models:
        - provider: deepseek
          model: deepseek-v4-pro
        - provider: zai
          model: glm-5.2
      aggregator:
        provider: kimi-coding
        model: kimi-k2.7
      reference_temperature: 0.55
      aggregator_temperature: 0.35

Proposed CLI UX

The local workflow I have been using exposes commands like:

/moa-preset list
/moa-preset check
/moa-preset set ultra

Optional friendly aliases are also useful:

/moa preset list
/moa preset set ultra

Expected behavior:

  1. /moa-preset list shows all configured MoA presets, their reference models, aggregator, and which preset is active/default.
  2. /moa-preset check shows the effective currently selected MoA configuration.
  3. /moa-preset set <name> should switch the current session to that MoA preset and optionally persist it as the default.
  4. /moa <prompt> can remain useful as a one-shot MoA invocation, but it should be distinct from persistent preset switching.

Important behavior: live switching

The most important part of the prototype is that changing a preset should not only edit config. It should update the active session.

For example, if the current session is on a normal model and the user runs:

/moa-preset set ultra

Then the session should effectively become:

model:
  provider: moa
  default: ultra
  base_url: moa://local
  api_key: moa-virtual-provider

and the current in-memory agent/client should be rebuilt or switched so the next turn actually uses moa/ultra.

Similarly, if the session is already using another MoA preset, for example workhorse, then:

/moa-preset set ultra

should switch from workhorse to ultra without requiring a full restart.

Why this is useful

This makes MoA more practical for real use:

  • users can maintain multiple quality/cost tiers,
  • difficult tasks can be escalated to a stronger ensemble,
  • normal tasks can stay on a cheaper preset,
  • the setup is transparent and reproducible in config.yaml,
  • no source-code edits are needed,
  • CLI, dashboard, desktop, gateway, and future automation can all use the same named presets.

Possible commands / behavior summary

/moa-preset list
  Show configured presets and active/default markers.

/moa-preset check
  Show effective config for the active preset.

/moa-preset set <name>
  Set active/default preset and live-switch the current session to provider=moa, model=<name>.

/model <name> --provider moa
  Should also resolve named MoA presets.

hermes --provider moa --model <name>
  Should start directly with the named MoA preset.

Suggested validation

Hermes could validate:

  • preset name exists,
  • at least one reference model is configured,
  • aggregator is configured,
  • providers are known or have credentials,
  • temperatures and token limits are sane,
  • disabled presets do not appear as selectable active models.

A diagnostic command could also help:

hermes moa diagnose ultra

or:

/moa-preset diagnose ultra

This could test whether all reference models and the aggregator are reachable before the user starts a long task.

Implementation notes from local prototype

The local prototype required four pieces:

  1. A config-backed moa.presets section.
  2. A slash command registry entry for /moa-preset.
  3. CLI handling for list, check, and set.
  4. Live session switching so set <preset> updates the active model/provider, not only config.yaml.

The important distinction is:

  • /moa <prompt> = one-shot MoA route for a single prompt,
  • /moa-preset set <name> = persistent/live switch to a named MoA preset.

Suggested labels

type/feature, comp/cli, comp/agent, area/config

Closing note

This proposal is based on a real local workflow that made MoA much more usable for day-to-day tasks. I think first-class named presets would fit Hermes well because Hermes already emphasizes configurable models, profiles, toolsets, and user-specific workflows.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Low — cosmetic, nice to havecomp/agentCore agent runtime: loop, agent_init, prompt builder, context-compression, responses endpointcomp/cliCLI entry point, hermes_cli/, setup wizardsweeper:implemented-on-mainSweeper: behavior already present on current maintype/featureNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions