fix: align _sanitize_non_numeric_tokens with _extract_output_tokens for non-whole floats (#874)

[microsasa]

Closes #874

Problem

Two parallel token-counting code paths diverged for non-integer positive float outputTokens values (e.g. 1.5):

Path	Result for outputTokens: 1.5
_extract_output_tokens (parser fast path)	None → 0 tokens in summary
AssistantMessageData.outputTokens (Pydantic model)	1 (truncated via lax coercion)

This caused a session event with outputTokens: 1.5 to show 0 tokens in summary/cost views but 1 token in the detail timeline.

Fix

Added a non-integer float guard to _sanitize_non_numeric_tokens in models.py:

• Non-whole floats (e.g. 1.5, 2.3) → 0
• Whole positive floats (e.g. 100.0) → explicitly coerced to int(100) before Pydantic
• Negative/zero floats → 0

Updated the _extract_output_tokens docstring in parser.py to reflect that both paths now fully agree without caveats.

Updated the cross-check equivalence test in test_parser.py to remove the ValidationError try/except (no longer needed since the validator now handles non-whole floats cleanly).

Tests

• test_non_whole_float_maps_to_zero — AssistantMessageData(outputTokens=1.5).outputTokens == 0
• test_non_whole_float_large_maps_to_zero — AssistantMessageData(outputTokens=2.3).outputTokens == 0
• test_whole_positive_float_coerced_to_int — AssistantMessageData(outputTokens=100.0).outputTokens == 100
• test_fractional_float_consistent_summary_and_detail — end-to-end: both session summary and detail view show 0 for outputTokens: 1.5

All existing tests continue to pass. make check passes cleanly (lint, typecheck, security, unit + e2e tests, 99% coverage).

Generated by Issue Implementer · ● 11.1M · ◷

[Copilot]

Pull request overview

This PR fixes a divergence between the parser “fast path” and the Pydantic model path when handling fractional (non-whole) float outputTokens values, ensuring session summaries and detail timelines report consistent token counts.

Changes:

• Clamp non-whole float outputTokens (e.g. 1.5) to 0 in AssistantMessageData._sanitize_non_numeric_tokens, and coerce whole positive floats (e.g. 100.0) to int.
• Update _extract_output_tokens docstring to match the unified “positive whole-number only” behavior.
• Extend tests to cover fractional/whole float cases and verify summary/detail consistency.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
src/copilot_usage/models.py	Updates outputTokens validator to reject fractional floats and coerce whole floats, aligning with parser behavior.
src/copilot_usage/parser.py	Docstring updated to reflect that both token-extraction paths now agree without validation caveats.
tests/copilot_usage/test_models.py	Adds unit tests for fractional float clamping and whole-float coercion.
tests/copilot_usage/test_parser.py	Simplifies equivalence test (no longer expects ValidationError) and adds an end-to-end consistency regression test.

[github-actions]

Quality Gate: APPROVED (Medium impact, good code quality)

What was evaluated:

• _sanitize_non_numeric_tokens validator in models.py now correctly handles non-whole floats (e.g. 1.5 → 0), aligning with the existing _extract_output_tokens fast path in parser.py
• Docstring updates in parser.py reflect the tightened contract
• 4 new tests: 3 unit tests for the validator edge cases + 1 end-to-end regression test verifying both code paths agree
• Existing cross-check test simplified (try/except removed since ValidationError is no longer raised for non-whole floats)

Impact: Medium — touches a Pydantic field validator in the data model, but the change is a narrow bugfix for a specific edge case (non-integer float outputTokens). No API contract changes, no new dependencies. All 8 CI checks pass.

Auto-approving for merge.