Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 32 additions & 1 deletion contract/openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,13 @@ components:
$view, $partial_copy, $refine, $followup, $rephrase, $undo, $share, $flag,
$rate, $escalate, $switch_model, $retry_context, $post_accept_edit,
$blur, $return, $scroll_regression, $navigate, $interrupt,
$startup, $pageleave.
$startup, $pageleave, $sessionend.

$abandon vs $sessionend: $abandon is a user-initiated negative quality
signal (closed editor without using the output, clicked stop, rejected).
$sessionend is auto-emitted by the JS SDK when the tab closes or the
user idles out with a generation still open — a session boundary, not
a quality signal.

Event:
type: object
Expand Down Expand Up @@ -129,6 +135,31 @@ components:
user_id:
type: string
description: Identifier for the end user. Required for cross-session signals (Tier 4+).
model:
type: string
description: Model identifier (e.g. "claude-sonnet-4-20250514", "gpt-4o"). Typically set on $generation events.
provider:
type: string
description: LLM provider (e.g. "anthropic", "openai", "openrouter").
input_tokens:
type: integer
description: Number of input tokens consumed.
output_tokens:
type: integer
description: Number of output tokens produced.
total_tokens:
type: integer
description: Total tokens (input + output).
duration_ms:
type: integer
description: End-to-end generation latency in milliseconds.
ttft_ms:
type: integer
description: Time to first token in milliseconds.
cost:
type: number
format: double
Comment on lines +144 to +161
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Missing minimum: 0 constraints on numeric fields

input_tokens, output_tokens, total_tokens, duration_ms, ttft_ms, and cost are all non-negative by definition, but the schema allows any integer/number value. Adding minimum: 0 closes the door on accidental negative submissions that would corrupt reporting.

Suggested change
input_tokens:
type: integer
description: Number of input tokens consumed.
output_tokens:
type: integer
description: Number of output tokens produced.
total_tokens:
type: integer
description: Total tokens (input + output).
duration_ms:
type: integer
description: End-to-end generation latency in milliseconds.
ttft_ms:
type: integer
description: Time to first token in milliseconds.
cost:
type: number
format: double
input_tokens:
type: integer
minimum: 0
description: Number of input tokens consumed.
output_tokens:
type: integer
minimum: 0
description: Number of output tokens produced.
total_tokens:
type: integer
minimum: 0
description: Total tokens (input + output).
duration_ms:
type: integer
minimum: 0
description: End-to-end generation latency in milliseconds.
ttft_ms:
type: integer
minimum: 0
description: Time to first token in milliseconds.
cost:
type: number
format: double
minimum: 0
description: Generation cost in USD.
Prompt To Fix With AI
This is a comment left during a code review.
Path: contract/openapi.yaml
Line: 144-161

Comment:
**Missing `minimum: 0` constraints on numeric fields**

`input_tokens`, `output_tokens`, `total_tokens`, `duration_ms`, `ttft_ms`, and `cost` are all non-negative by definition, but the schema allows any integer/number value. Adding `minimum: 0` closes the door on accidental negative submissions that would corrupt reporting.

```suggestion
        input_tokens:
          type: integer
          minimum: 0
          description: Number of input tokens consumed.
        output_tokens:
          type: integer
          minimum: 0
          description: Number of output tokens produced.
        total_tokens:
          type: integer
          minimum: 0
          description: Total tokens (input + output).
        duration_ms:
          type: integer
          minimum: 0
          description: End-to-end generation latency in milliseconds.
        ttft_ms:
          type: integer
          minimum: 0
          description: Time to first token in milliseconds.
        cost:
          type: number
          format: double
          minimum: 0
          description: Generation cost in USD.
```

How can I resolve this? If you propose a fix, please make it concise.

description: Generation cost in USD.
Comment on lines +138 to +162
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 SDK sends model (and new fields) via metadata, not as top-level fields

The contract now defines model, provider, input_tokens, output_tokens, total_tokens, duration_ms, ttft_ms, and cost as top-level Event properties, but the Python SDK never serializes them at the top level. client.py's track() signature has no parameters for any of these fields, and Feature.generation() explicitly puts model into metadata — confirmed by the test at tests/test_client.py:251 which asserts gen_event["metadata"]["model"] == "gpt-4o". If the server-side is updated to read model (and the other new fields) from the event's top-level object, the SDK will silently fail to deliver this data. The SDK's track() method, Generation.event(), and the event-building block in the constructor all need to be updated to forward these fields at the wire level.

Prompt To Fix With AI
This is a comment left during a code review.
Path: contract/openapi.yaml
Line: 138-162

Comment:
**SDK sends `model` (and new fields) via `metadata`, not as top-level fields**

The contract now defines `model`, `provider`, `input_tokens`, `output_tokens`, `total_tokens`, `duration_ms`, `ttft_ms`, and `cost` as top-level `Event` properties, but the Python SDK never serializes them at the top level. `client.py`'s `track()` signature has no parameters for any of these fields, and `Feature.generation()` explicitly puts `model` into `metadata` — confirmed by the test at `tests/test_client.py:251` which asserts `gen_event["metadata"]["model"] == "gpt-4o"`. If the server-side is updated to read `model` (and the other new fields) from the event's top-level object, the SDK will silently fail to deliver this data. The SDK's `track()` method, `Generation.event()`, and the event-building block in the constructor all need to be updated to forward these fields at the wire level.

How can I resolve this? If you propose a fix, please make it concise.

metadata:
type: object
additionalProperties: true
Expand Down
Loading