Skip to content

Adapter: record CHAP human-decision events from LlamaIndex Workflows human-in-the-loop#6

Open
BrightbeamBogdanCastraveti wants to merge 2 commits into
mainfrom
adapter/llama-index
Open

Adapter: record CHAP human-decision events from LlamaIndex Workflows human-in-the-loop#6
BrightbeamBogdanCastraveti wants to merge 2 commits into
mainfrom
adapter/llama-index

Conversation

@BrightbeamBogdanCastraveti

Copy link
Copy Markdown
Collaborator

Closes #1.

Adapter mirroring chap-langgraph / chap-pydantic-ai: when a LlamaIndex Workflow step pauses on an InputRequiredEvent, the human's HumanResponseEvent is recorded as a CHAP decision.

approve   -> decide.approve
override  -> decide.override   (proposed vs returned, as an RFC 6902 diff)
reject    -> decide.reject

Design (per the plan on #1):

  • decision is explicit and authoritative — Workflows events are schemaless and carry no approve/edit/reject signal, so the adapter never infers intent. proposed/returned/rationale/tags/intent_preserved/approver default to event.get(...) with per-call overrides; arg names match the sibling bridges.
  • Recorded from the driver (where both events meet), using the two-step InputRequiredEventHumanResponseEvent loop — so step replay never double-records.
  • llama-index-workflows is optional; the bridge reads events structurally, so it and its tests run without it.
  • 0.2.6-correct by construction: approver joined, review.request addressed to that approver, so from is always a member and in the to set.

Verified against main (0.2.6): adapter pytest 12 passed; example runs on real llama-index-workflows 2.22 through approve / refining edit / substituting edit / reject; conformance harness 23/23.

…dex Workflows human-in-the-loop

Adapter mirroring chap-langgraph: when a Workflow step pauses on an
InputRequiredEvent, record the human's HumanResponseEvent as a CHAP
envelope.

  approve   -> decide.approve
  override  -> decide.override (RFC 6902 diff of proposed vs returned)
  reject    -> decide.reject

- llama-index-workflows is an optional dependency; events are read
  structurally via event.get(...), so the bridge and tests run without it.
- decision is explicit and authoritative: Workflows events carry no
  approve/edit/reject signal, so the adapter never infers intent.
- recorded from the driver using the two-step InputRequiredEvent ->
  HumanResponseEvent loop, so step replay never double-records.
- 0.2.6-correct by construction: approver joined, review.request
  addressed to that approver, so from is a member and in the to set.

Closes #1.
@BrightbeamBogdanCastraveti

Copy link
Copy Markdown
Collaborator Author

Self-review caught one edge case worth flagging: in record_decision, if decision="override" but no returned content is supplied (neither the returned= argument nor a response field on the event), _diff(proposed, None) produces a whole-artefact replace with null — i.e. it records a blank-everything override instead of refusing. Tests and the example always pass returned, so it doesn't surface there, but a malformed caller could write a misleading record.

Fix is a one-line guard — raise instead of diffing against None:

if diff is None:
    if returned is None:
        raise ValueError("an override needs `returned` content (or a `diff`)")
    diff = _diff(proposed, returned, "")

Ready locally; I'll push it once the branch-update ruleset is scoped (same blocker as #5).

Guard record_decision so decision=override without returned content (or
a diff) raises instead of recording a whole-artefact replace with null.
@brightbeamarsalan

Copy link
Copy Markdown
Collaborator

you should be able to update Bogdan, please let me know how it goes.

@BrightbeamBogdanCastraveti

Copy link
Copy Markdown
Collaborator Author

Pushed c2498ee — the branch updated cleanly now that the ruleset is scoped. That's the override guard (raise instead of a null-replace when an override has no returned content), with a test. Re-ran against main (0.2.6): pytest 14 passed, harness 23/23. Ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adapter: record CHAP human-decision events from LlamaIndex Workflows human-in-the-loop

2 participants