Repository Level Harness Coverage

[Ian Johnson (tacoda)]

Hello! I have been playing around a lot with harnesses recently. I actually just wrote a book about what I've learned in the past six months: https://leanpub.com/harness-engineering

There's a lot of confusion around the word harness right now. I was focusing on the harness as the "repo-centric environment for the agent". Of course LangChain refers to their product as a harness. And others see Claude Code as a harness. This generates a lot of confusion. Here's the model I have in my book:

I was referring to the middle areas as the harness - the stuff the developer can change. I don't really care what we call it so long as we disambiguate it. I shipped a tool not long ago called Keystone. It is a harness at the project level. At first I thought LangChain would mostly be used for orchestrators - but this project proves it can also be what I refer to as a 'coding agent'. It makes me wonder what else can be covered. I'm assuming we can cover the layers in between with Python code - or with some way to load rules, etc. This is already done by the coding agent, I'm sure, but they are primitives. The idea of this project was to provide a way to quickly bootstrap and maintain a repo-centered harness long-term.

Is this something that can be covered? Would it be making this project do too much? Could it be done with a toml file instead of a markdown file? Would that be easier or harder for users to consume?

[Mason Daugherty (mdrxy)]

Thanks for writing this up — I agree the term “harness” is overloaded. I tend to think of Deep Agents as a general harness for building agents, where a coding agent is one possible application built on top of it, rather than the harness being something wrapped around a coding agent.

I think the repo-centered framing may be more specific to coding agents than to Deep Agents itself. Deep Agents is meant to support many kinds of long-running/task-oriented agents, not only agents operating inside a repository. For now, we generally prefer to expose composable primitives — tools, instructions, middleware, memory, subagents, persistence, HITL, etc. — and adopt existing conventions where they make sense, rather than introduce a new repo-level configuration standard.

So I don’t think this is an area we’re actively trying to define as a first-class abstraction right now. If there are specific missing primitives that would make Keystone-style or repo-centered workflows easier to build on Deep Agents, those would be useful to hear.

[Ian Johnson (tacoda)]

Deep Agents as a general harness for building agents, where a coding agent is one possible application built on top of it

Great info - thank you.

Agreed on it not being applicable due to the misunderstanding - this is exactly the kind of semantic diffusion I'd like to address. I will see if I can get this discussion going in a different appropriate thread, since it doesn't apply to this repo.

The primitives in Keystone are abstractions over many standard things - rules, hooks, etc. It allows for extension, but it's really just a naming difference and I don't think that's necessary. I may try to build something with Deep Agents as an experiment. I already have an experiment going in Python.

The more valuable piece, IMO, is the structure (in terms of flywheels and feedback loops). If a coding agent could 'bootstrap' itself with a structured set of these primitives, then that could be an easy way to implement what I've been calling a harness. Perhaps these could just be 'libraries' or 'starter packs'.

For example, if I could just import the definition of ports and adapters, then that could constrain my code by adding rules, hooks, etc. that instruct and verify it was followed.

[Marcus Chang (hardness1020)]

The ambiguity gets smaller if you split it by what owns the loop. In Claude Code's design the harness is the runtime that owns the agent loop, tool dispatch, permissions, and context management, while the repo-level setup (CLAUDE.md, skills, hooks) is configuration the harness loads. Calling both "harness" breeds the confusion, since one is executable machinery and the other is declarative environment.

I traced Claude Code's version of this layer by layer and rebuilt it as runnable Python examples:
https://github.com/hardness1020/awesome-agent-architecture/tree/main/sections/00-harness-thesis

[Ian Johnson (tacoda)]

Agreed. I have recently updated this but didn't add it here because the discussion was closed.
You are correct that this repo-level setup is the configuration the harness loads.
I have recently been calling this notion the charter - it's the agreement that you have with the harness.
What it's called doesn't matter so much as it isn't conflated. I've updated the project to be a 'charter framework'. I've also recently found other alternative tools recently and still digging into this topic in depth. Thanks so much for the link - I'll definitely check it out.