Context
Behavioral signals are Layer 2 of the eval engine. They are the most honest signal of response quality — a user who immediately rephrases their question tells us more than any automated scorer. The SDK must make it trivially easy for customers to report these signals.
Task
Implement the behavioral signal helper methods that customers call after a trace to report user actions.
Methods to implement
sentinel.thumbs_down(trace_id) # user clicked negative feedback
sentinel.escalate(trace_id) # user requested a human agent
sentinel.rephrased(trace_id) # user repeated or rephrased within 60s
sentinel.abandoned(trace_id) # user ended session within 30s
sentinel.acknowledged(trace_id) # user sent short ack, session ended naturally
Implementation details
- Each method sends a lightweight signal event (not a full trace) to the ingest API
- Signal payload:
{ trace_id, signal_type, timestamp, customer_id }
- Async, non-blocking — same background shipper as SDK-02
- Signal events stored in ClickHouse alongside their parent trace
Acceptance criteria
Context
Behavioral signals are Layer 2 of the eval engine. They are the most honest signal of response quality — a user who immediately rephrases their question tells us more than any automated scorer. The SDK must make it trivially easy for customers to report these signals.
Task
Implement the behavioral signal helper methods that customers call after a trace to report user actions.
Methods to implement
Implementation details
{ trace_id, signal_type, timestamp, customer_id }Acceptance criteria
thumbs_down()andrephrased()visibly