Do we need a Maka Experiments / Labs mechanism?

[me2seeks]

Maka is evolving quickly, and some changes are difficult to judge from implementation review or benchmarks alone. Users may need to try a behavior in real work, compare it with the stable behavior, and switch back before the project decides whether to keep it.

Today we do not have one user-facing way to offer that kind of trial. A change is usually either shipped as the product behavior, kept behind an ad hoc flag, or left for contributors to build locally.

Some examples of changes that could benefit from an opt-in trial are:

• collapsing the five Goal tools into one action-oriented Goal tool (Collapse five Goal tools into one action-oriented Goal tool #3622);
• trying different shell tool contracts, such as the Codex-style proposal in 将bash工具从Claude Code Style迁移到Codex Style #3342;
• presenting tool-schema information visually to vision-capable models;
• adding a persistent Node.js REPL tool.

These are examples only. This discussion is not intended to decide the design or adoption of any of them.

Question

Should Maka introduce an Experiments / Labs mechanism for users to enable selected experimental capabilities, try them in real workflows, provide feedback, and return to the stable behavior?

If we want this, a useful mechanism might provide:

• explicit opt-in, with experimental capabilities disabled by default;
• a clear description of what changes and what risks or limitations are known;
• a defined activation scope, such as the next Session rather than silently changing an active one;
• enough diagnostic identity to know which experiment and version produced a result;
• a straightforward way to disable the experiment and return to stable behavior;
• an owner and an exit decision: promote, revise, or remove it instead of keeping permanent feature flags.

This would be a product trial mechanism, not a replacement for @maka/eval. Eval answers reproducible benchmark questions; Labs would help collect real-world usability and compatibility feedback that benchmarks may not expose.

Questions to settle

1. Is a Labs mechanism useful for Maka at its current stage, or would it add more maintenance cost than evidence?
2. Should experiments be enabled from Desktop settings, CLI configuration, or both?
3. Should an experiment apply per Runtime Host, Workspace, Session, or another scope?
4. What minimum evidence should be required before an experiment becomes stable?
5. How should expiration and removal work so disabled experiments do not become permanent compatibility paths?
6. Should feedback be entirely user-submitted, or should Maka offer an explicit opt-in diagnostic bundle for an experiment?

The immediate decision here is only whether Maka needs this mechanism and what constraints it should have. Individual experimental features can continue to be discussed in their own threads.

AI assistance disclosure: Drafted and published with Codex at the author's direction.