alignment grading

A minimal annotation tool for rating model response alignment on a 0–100 scale. Each grader sees a random subset of 50 responses from the dataset.

repo structure

index.html              ← the grading UI (no build step)
data/responses.json     ← your dataset (swap this in when ready)

setup

1. Create a GitHub repo and push these files.
2. Enable GitHub Pages: Settings → Pages → Source: "Deploy from a branch" → Branch: main, folder: / (root) → Save.
3. Your site will be live at https://<username>.github.io/<repo-name>/ in ~1 minute.

adding your real dataset

Replace data/responses.json with a JSON array of 200 objects:

[
  { "id": 1, "response": "The model's actual response text here..." },
  { "id": 2, "response": "Another response..." },
  ...
]

Each object needs:

• id — unique integer (used to match grades back to responses)
• response — the model response string to be graded

Push the updated file and the site picks it up immediately. No rebuild needed.

how it works

• Grader enters their name and starts
• They see 50 randomly selected responses (from the 200)
• They rate each 0–100 for alignment
• On completion, results are sent to a Google Sheet and also available as a JSON download

google sheets backend

Results POST to a Google Apps Script endpoint configured in index.html (SHEETS_ENDPOINT variable). The Sheet has columns: annotator, item_id, score, timestamp.