A Formalization Tool for Future Harness Self-evolution

[Byte-Naut]

The Cordis paper invokes the landscape that harness utilizes its dynamic plugin-based system thoroughly to achieve active-self-evolution during runtime. So I developed this tool for anyone who is enchanted by harness evolution research. The tool's function is simple: you install the plugin and skill (or preset), it provides your agent timely clear runtime overviews, and checks a pre-modification's influence fast and formalized. It compresses your agent's labour to look at all interfaces related in one tool call.
In a four-stage test scenario running on Gemini 3.8 Flash, it saves 70%+ tokens in Cordis dialogue & fixes tasks with same success rates (It means, all tasks are totally passed). And is observed to reduce 20 tool call operations in a test from official minimal preset for just a 9B local LLM.

Those days, the same model has distinctly different performance on different harnesses turning up even in most-advanced models OpenAI's GPT-6 Astra on ARC-AGI-3. It's an interesting and charming phenomenon emphasizing the irreplaceable role of elaborate harness design, and self-constructed dynamic mechanism to take advantage of the models' various fitness in varied scenarios. I would appreciate any discussions around the profound topic, thanks for the great work from dsh development team and all plugin developers! It's you that bring us to the horizon of next-generation agent harness!

https://github.com/Byte-Naut/dsh_semantic_tool

[goatliamia]

Really interesting direction. Our work is somewhat complementary.

We’ve been exploring runtime-first agent design: instead of putting more and more behavior into prompts or Skills, we try to move deterministic responsibilities into the runtime, so the model handles uncertainty while the runtime owns real state, lifecycle, and reactions to actual execution results.

In DSH, we’ve been experimenting mainly with post-execution Runtime React so far: observe what really happened, update runtime state, and react when needed, without making the model reconstruct everything from the conversation.

Your semantic interface looks like a natural counterpart on the observation side: Runtime → Agent: “this is what the system actually looks like now.” Ours is more about Runtime → Runtime/Agent: “this is what just happened, and here is what the system should do next.”

I think there could be an interesting intersection between the two.
https://github.com/goatliamia/dsh-runtime-react

[Byte-Naut]

Interesting work. I haven’t tested it yet. Do you think the harness should move session-event-based decisions out of the main LLM and handle them through rule-based conditional logic? Or could a smaller model that continuously interacts with the current runtime/session state play a smarter role here?

[goatliamia]

I think my point is a little different from that.

I’m not really thinking about adding another model to make these decisions. My thinking is more that, as harnesses start moving into a more engineering-oriented stage, we’ll probably care more about things like resilience, stability, and recovery.

And there’s already a lot of useful information in the runtime/session state. The system may already know that something failed, that a state transition didn’t happen, or that some condition changed, but that information isn’t always available to the LLM when it would actually be useful.

In some cases, I think not giving the model that information can actually make the situation worse, because it keeps reasoning from a state that is no longer true.

So I’m more interested in how we can make better use of information the harness already has, rather than moving the decision-making to rules or introducing another model.

The LLM would still make the actual judgment; I’m mostly thinking about what information the harness should reliably expose to it.