docs: README, CONTRIBUTING and CLAUDE.md reflect the harness x model benchmark - #53
Merged
Merged
Conversation
- README: Harnesses section (axis explanation, table of felix/baseline/ opencode/grok-build/template with install lines, matrix examples, how coding-agent harnesses attach over MCP); results table labelled as the pre-1.2 felix-only spread; plan_coherence caveat and quarantine/metadata notes under Scoring; Development shows both CI configurations; Related Repos lists the harness repos - CONTRIBUTING: smoke test means mock LLM too (old 'real LLM smoke' recipe removed), no-felix test recipe, live coding-agent example, 'Testing a harness plugin' section - CLAUDE.md: Harnesses table; Felix-specific sections (agents, CentralPost, conflict resolution, helix) labelled as such; SSE flow via harness.step; provider/model semantics per harness; conventions and CI job list updated Co-authored-by: Jason <jkbennitt@users.noreply.github.com>
Co-authored-by: Jason <jkbennitt@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #52 (which landed before these docs edits were pushed). No code changes.
README
--harness×--model), a table offelix/baseline/opencode/grok-build/templatewith where each lives and how to install it, matrix examples, and how coding-agent harnesses attach over the RLE MCP server.plan_coherenceis measured post-resolution and is a floor, not a harness ranker; notes theharness_failedquarantine and per-run harness metadata.felixextra), boundary check, contract job.CONTRIBUTING
--smoke-testnow means mock LLM as well as mock game; the old "smoke test with real LLM" recipe (no longer true) is removed.rle.testing.run_harness_smoke,ScriptedMcpHarness).CLAUDE.md
harness.step(...); "all 10 metrics" fixed.test-no-felix,external-plugin-contract, boundary check).