Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions reviewer-panel-consensus-monitor/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Reviewer Panel Consensus Monitor

This module adds a focused Scientific Bounty System slice for third-party reviewer panel quality control. It helps sponsors and platform arbitrators decide whether a challenge submission is ready for phase advancement or reward action.

It evaluates:

- declared reviewer conflicts of interest
- independent reviewer coverage and expertise tags
- locked submission manifests and artifact hashes
- reviewer score variance against the rubric threshold
- missing reviewer feedback
- missing decision explanations or response deadlines

The output is an arbitration-ready packet with a deterministic digest, required actions, reviewer counts, the review phase, and a clear panel decision.

## Usage

```bash
node reviewer-panel-consensus-monitor/demo.js
```

```js
const { evaluateReviewerPanel } = require("./index.js");

const result = evaluateReviewerPanel({
challengeTitle: "Climate Materials Prize",
phase: "final award review",
submission: {
id: "sub-2026-088",
teamId: "team-carbon",
artifactHashes: ["sha256:model", "sha256:dataset", "sha256:report"],
manifestLockedAt: "2026-05-18T16:40:00Z"
},
rubric: {
version: "v3.0",
criteria: ["scientific rigor", "reproducibility", "field impact"],
decisionThreshold: 82,
maxScoreVariance: 12
},
reviewers: [],
decision: {}
});
```

## Verification

```bash
node reviewer-panel-consensus-monitor/test.js
node reviewer-panel-consensus-monitor/demo.js
```

The tests cover a blocked panel with conflicts and missing review controls, an approved panel with independent consensus, and arbitration-ready evidence/remediation fields for every finding.

## Demo

Short demo artifact: `demo.gif`

The demo uses synthetic data only. It does not call external services, use credentials, or move funds.
Binary file added reviewer-panel-consensus-monitor/demo.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
65 changes: 65 additions & 0 deletions reviewer-panel-consensus-monitor/demo.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
const { evaluateReviewerPanel } = require("./index.js");

const panel = {
challengeTitle: "Open Ocean Carbon Prize",
phase: "milestone award review",
submission: {
id: "sub-2026-112",
teamId: "team-kelp",
artifactHashes: ["sha256:field-data", "sha256:model-card", "sha256:analysis-notebook"],
manifestLockedAt: "2026-05-18T17:05:00Z"
},
rubric: {
version: "v1.4",
criteria: ["scientific rigor", "reproducibility", "field impact"],
decisionThreshold: 80,
maxScoreVariance: 10
},
reviewers: [
{
id: "rev-ocean-1",
type: "independent",
expertise: ["oceanography", "field-methods"],
conflicts: [],
scores: { "scientific rigor": 88, reproducibility: 84, "field impact": 91 },
feedback: "Field protocol and model card support the milestone claim."
},
{
id: "rev-stat-2",
type: "independent",
expertise: ["statistics", "carbon-accounting"],
conflicts: [],
scores: { "scientific rigor": 85, reproducibility: 86, "field impact": 89 },
feedback: "Uncertainty intervals are reproducible from the locked notebook."
},
{
id: "rev-sponsor-observer",
type: "sponsor",
expertise: ["deployment"],
conflicts: [],
scores: { "scientific rigor": 86, reproducibility: 83, "field impact": 90 },
feedback: "Milestone deliverables match the published challenge terms."
}
],
decision: {
proposedOutcome: "award",
explanation: "Independent reviewers reached a low-variance consensus above threshold.",
reviewerResponsesDueAt: "2026-05-24T12:00:00Z"
}
};

const result = evaluateReviewerPanel(panel);

console.log("Reviewer panel consensus monitor demo");
console.log("-------------------------------------");
console.log(`Challenge: ${result.summary.challengeTitle}`);
console.log(`Phase: ${result.summary.phase}`);
console.log(`Panel decision: ${result.summary.panelDecision}`);
console.log(`Consensus quality: ${result.summary.consensusQuality}`);
console.log(`Average score: ${result.summary.averageScore}`);
console.log(`Findings: ${result.summary.findingCount}`);
console.log(`Digest: ${result.arbitrationPacket.digest}`);
console.log("Required actions:");
for (const action of result.arbitrationPacket.requiredActions) {
console.log(`- ${action}`);
}
Loading