Human Value Allocation

What did the human bring to the work?

Evidence-linked analysis of human contributions in human–AI conversations. Import a dialogue, inspect each human turn, and follow every observation back to the source text.

v0.1.0a1 · Python 3.10+ · MIT

中文说明 · Example report · Evaluation rubric · Method · Paper

Related paper: From Tasks to Turns: Measuring Human Labor Contributions in Human–Agent Joint Production studies Human-Turn-level quality and remaining computation.

Paper Figure 1. The research protocol independently continues adjacent checkpoints under matched models, budgets, and termination conditions, then compares final task quality and remaining token usage. It also supports controlled guidance comparisons and human-origin interactions.

git clone https://github.com/kling01/human-value-allocation.git
cd human-value-allocation
python -m venv .venv
source .venv/bin/activate
python -m pip install .
hva demo --out outputs/demo

Open outputs/demo/report.html in your browser. The demo runs fully offline with a synthetic conversation and authored illustrative annotations.

Configure a model through an OpenAI-compatible Chat Completions endpoint that supports JSON-object output:

export HVA_BASE_URL="https://api.openai.com/v1"
export HVA_MODEL="your-model-id"
# Set HVA_API_KEY in your environment using your own provider credential.
hva analyze examples/conversation.json --out outputs/my-first-analysis

analyze sends the supplied conversation to the provider you configure. There is no HVA-hosted upload endpoint or telemetry. Reports are local, contain source dialogue, and are only shared when you choose to share them. A new output directory is required for each run.

Outputs:

• report.html: a standalone report with source-linked observations and visible followup.
• report.json: machine-readable results, rubric fingerprint, unresolved turns, and provider usage when available.

Remote endpoints require HTTPS by default. --allow-insecure-http explicitly opts into an HTTP gateway you trust; loopback HTTP endpoints are accepted for local models.

For each turn, the report separates the human's contribution from what the assistant visibly did next. Evidence quotes are checked against the permitted source messages. An invalid quote or failed API call produces an unresolved turn, shown separately and excluded from contribution counts.

{
  "id": "my-session-full-id",
  "messages": [
    {"id": "u1", "role": "user", "content": "Please compare these two options."},
    {"id": "a1", "role": "assistant", "content": "Here is a comparison..."}
  ]
}

Each message has exactly id, role, and text content. Roles: user, assistant, system, tool. Message IDs must be unique within the session; full IDs are preserved. v0 accepts one text conversation, up to 50 human turns and 200,000 content characters. Unsupported or excessive input is rejected without silent truncation. Native tool-call trees, attachments and multimodal inputs need an adapter and are not supported by this release.

python -m pip install -e '.[dev]'
python -m pytest -q
python scripts/audit_release.py
python -m build

Tests cover source evidence, temporal boundaries, full IDs, failed evaluations, HTML escaping, shared-engine integration and request retry accounting. Integration tests exercise the analysis pipeline with a mocked model provider.

See the release validation record for isolated installation and live smoke checks.

• Human review of the rubric across several task types.
• Counterfactual replay protocols and task-specific quality measures.
• Token-equivalent contribution estimates using validated comparison measures.

If you try this on a conversation, useful feedback is which observation you disagree with and why. Share synthetic or appropriately redacted examples in Issues. Improvements to the rubric and input adapters are welcome.