In #14, we add more system-level metrics like latency distribution, response parsing, etc...
We should also compute tokens for inputs and outputs, and add aggregate and sample-level metrics for number of input and output tokens.
In documentation, we should describe how we're computing tokens for transparency, and to illustrate that we can't guarantee that we'll match token computation strategies of all providers.
In #14, we add more system-level metrics like latency distribution, response parsing, etc...
We should also compute tokens for inputs and outputs, and add aggregate and sample-level metrics for number of input and output tokens.
In documentation, we should describe how we're computing tokens for transparency, and to illustrate that we can't guarantee that we'll match token computation strategies of all providers.