You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Record decisions in production and fit a threshold from them. Client(..., recorder=JsonlRecorder(path)) writes one row per decision with the probabilities, the reported confidence, the route and the latency; record_outcome(path, id, {...}) attaches what actually turned out to be right; rows_from joins the two into exactly what fit_temperature and fit_conformal_quantile take. decide fit <file> <question> prints the fitted values plus what each threshold would have accepted and how often it was right, which is the table that says whether a gate is safe to turn on. Recording never fails a decision: a recorder that raises is ignored.
decide fit also prints the range of confidence actually reported, since a threshold table of zeros usually means every threshold sits above what the backend reports.