Skip to content

AgentJury v0.4.3, Public Alpha

Choose a tag to compare

View all tags
@madad-rashid madad-rashid released this 03 Sep 19:47
· 2 commits to main since this release
v0.4.3
cbb2970
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

AgentJury is an open-source peer-review layer for AI agents.

When an agent says a task is finished, AgentJury sends the work to independent, blind AI reviewers and combines their opinions using deterministic aggregation rules. No final LLM gets a deciding vote.

What's in v0.4.3

  • Public-alpha README and quick start
  • PyPI installation
  • OpenAI and Anthropic judge adapters
  • Blind, independent reviewer panels
  • Approve, revise, and abstain votes
  • Strict-majority quorum
  • Multi-provider verification
  • No unilateral blocking
  • Deterministic aggregation
  • Prompt-injection defenses
  • Retry and JSON repair handling
  • Stable request, run, review, configuration, and finding IDs
  • Custom reviewer roles
  • Human finding-level adjudication
  • Append-only adjudication history
  • Hermes integration as the first live framework adapter
  • Architecture and contributor documentation
  • Trusted Publishing through GitHub Actions and PyPI

Install

Install AgentJury from PyPI:

pip install "agentjury[all]"

Then set your OpenAI and Anthropic API keys and run:

agentjury review task.md output.md

Developers who want the latest unreleased code can install directly from GitHub:

pip install "agentjury[all] @ git+https://github.com/madad-rashid/AgentJury.git"

How it works

Agent completes task
        ↓
   ReviewRequest
        ↓
┌─────────────────────┐
│ Independent judges  │
│ OpenAI              │
│ Anthropic           │
│ Custom / local      │
└─────────────────────┘
        ↓
Deterministic aggregation
        ↓
▲ Approve   ▼ Revise   – Abstain
        ↓
Verified / Needs revision / Blocked

Reviewers do not see one another's votes before submitting their own assessment.

Public alpha

The main goal of this release is real-world testing.

I'm looking for developers willing to run AgentJury on real agent workflows and report:

  • false positives
  • missed errors
  • reviewer disagreements
  • model instability
  • latency
  • review cost
  • installation friction
  • integration issues

Human adjudication from these tests will later support reviewer reputation and jury diversity weighting.

Hermes integration

AgentJury already includes a Hermes integration that reviews substantial agent responses in the background, stores the verdict, annotates generated notes, and feeds important findings back into the agent workflow.

See integrations/hermes/ for setup instructions.

Feedback

Found a problem, disagreement, or strange jury result?

Open an issue:

https://github.com/madad-rashid/AgentJury/issues

Please do not include proprietary company data, credentials, or API keys in public issues.

License

MIT

Assets 2
Loading