Skip to content

[SDK-03] Implement behavioral signal helper methods #3

Description

@ahmedred6

Context

Behavioral signals are Layer 2 of the eval engine. They are the most honest signal of response quality — a user who immediately rephrases their question tells us more than any automated scorer. The SDK must make it trivially easy for customers to report these signals.

Task

Implement the behavioral signal helper methods that customers call after a trace to report user actions.

Methods to implement

sentinel.thumbs_down(trace_id)        # user clicked negative feedback
sentinel.escalate(trace_id)           # user requested a human agent
sentinel.rephrased(trace_id)          # user repeated or rephrased within 60s
sentinel.abandoned(trace_id)          # user ended session within 30s
sentinel.acknowledged(trace_id)       # user sent short ack, session ended naturally

Implementation details

  • Each method sends a lightweight signal event (not a full trace) to the ingest API
  • Signal payload: { trace_id, signal_type, timestamp, customer_id }
  • Async, non-blocking — same background shipper as SDK-02
  • Signal events stored in ClickHouse alongside their parent trace

Acceptance criteria

  • All 5 signal methods implemented and tested
  • Signal payload schema agreed with ingest API engineer
  • Async dispatch confirmed (no blocking)
  • Integration test: signal correctly associated with parent trace in DB
  • Documented with usage examples in README
  • Demo script uses at least thumbs_down() and rephrased() visibly

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions