Hybrid architecture patterns: from code review to contract review — cross-domain lessons

[GlobalAIMedia]

The hybrid architecture — deterministic pipeline + LLM agent — caught my attention because we use the exact same pattern in a completely different domain: contract risk scanning.

We run 224 AI agents across 18 departments as part of an AI workforce platform. One of our internal skills is a contract risk scanner that works almost identically to open-code-review:

1. Deterministic pass: regex scans for known risky clauses ("unlimited liability", "perpetual confidentiality", "foreign court jurisdiction"). Each match gets a risk score from a hardcoded lookup table.

2. AI pass: the LLM reviews the full contract context, identifies risks the regex missed (vague definitions, missing acceptance criteria, unusual indemnification structures), and suggests remediation language.

3. Structured output: a risk report with per-clause ratings (High/Medium/Low), overall risk score, and actionable recommendations.

The hybrid approach is the right call. Pure deterministic misses context-dependent risks. Pure LLM hallucinates risks that aren't there. But the combination — deterministic as the reliable floor, LLM as the context-aware ceiling — has been remarkably effective.

A few questions from our experience that might be useful to the open-code-review community:

1. Rule maintenance at scale.
We started with ~30 risk patterns. Now we have 50+ across 5 categories (Liability, IP, Confidentiality, Payment, Dispute Resolution). Adding a new rule is easy; knowing which rules are still relevant as contract types evolve is hard. We're exploring automated rule efficacy tracking — measuring which rules actually trigger on real contracts and deprecating low-signal ones. Does open-code-review have a mechanism for evaluating rule effectiveness over time?

2. The "confidence gap" between deterministic and LLM findings.
Deterministic findings have 100% precision (if the keyword is there, it's there). LLM findings have variable confidence. We've experimented with having a separate "reviewer agent" that independently evaluates LLM findings before they reach the final report. The cost is an extra API call; the benefit is fewer false positives that erode user trust. Is this pattern useful for code review, or is the false-positive tolerance different in that domain?

3. Cross-agent collaboration for complex reviews.
For high-stakes contracts, we don't rely on a single agent. A Legal agent does the first pass, a Trade Compliance agent verifies regulatory aspects, and a Finance agent checks payment terms. Each produces structured findings, and a final "synthesizer" agent de-duplicates and prioritizes. This adds latency but catches things single-agent review misses. Could this multi-agent review pattern apply to large-scale code reviews — e.g., one agent for security, one for performance, one for style?

If anyone is curious about the contract-review version of this architecture, our risk scanner is part of trade-doc-generator at github.com/GlobalAI-Media/trade-doc-generator. The full 224-agent workforce is at github.com/GlobalAI-Media/224-ai-employees.

Curious to hear how the open-code-review community handles rule curation and multi-dimensional review workflows.