Skip to content

feat(llmcost): alias request and response token types#3953

Merged
tothandras merged 3 commits into
mainfrom
feat/llm-cost-flexibiloty
Mar 17, 2026
Merged

feat(llmcost): alias request and response token types#3953
tothandras merged 3 commits into
mainfrom
feat/llm-cost-flexibiloty

Conversation

@hekike
Copy link
Copy Markdown
Contributor

@hekike hekike commented Mar 17, 2026

Summary by CodeRabbit

  • New Features
    • Support for "request" (alias for "input") and "response" (alias for "output") token-type names in LLM cost configuration, preserving existing pricing behavior.
  • Validation
    • Validation messaging updated to include the new token-type aliases.
  • Tests
    • Added tests to verify the new token-type aliases correctly map to existing input/output pricing.

Validation

{
	"data": [
		{
			"cost": "0.27726",
			"currency": "USD",
			"dimensions": {
				"model": "gpt-4-turbo",
				"type": "response"
			},
			"from": "2026-03-17T03:02:00Z",
			"to": "2026-03-17T03:04:00Z",
			"usage": "9242"
		},
		{
			"cost": "0.088",
			"currency": "USD",
			"dimensions": {
				"model": "gpt-4o",
				"type": "response"
			},
			"from": "2026-03-17T03:02:00Z",
			"to": "2026-03-17T03:04:00Z",
			"usage": "8800"
		},
		{
			"cost": "0.08401",
			"currency": "USD",
			"dimensions": {
				"model": "gpt-4-turbo",
				"type": "request"
			},
			"from": "2026-03-17T03:02:00Z",
			"to": "2026-03-17T03:05:00Z",
			"usage": "8401"
		},
		{
			"cost": "0.0249425",
			"currency": "USD",
			"dimensions": {
				"model": "gpt-4o",
				"type": "request"
			},
			"from": "2026-03-17T03:02:00Z",
			"to": "2026-03-17T03:05:00Z",
			"usage": "9977"
		}
	]
}

@hekike hekike requested a review from a team as a code owner March 17, 2026 02:10
@hekike hekike added the release-note/feature Release note: Exciting New Features label Mar 17, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 17, 2026

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

💤 Files selected but had no reviewable changes (1)
  • api/api.gen.go
⛔ Files ignored due to path filters (5)
  • api/client/go/client.gen.go is excluded by !api/client/**
  • api/client/javascript/src/client/schemas.ts is excluded by !api/client/**
  • api/client/javascript/src/zod/index.ts is excluded by !api/client/**
  • api/openapi.cloud.yaml is excluded by !**/openapi.cloud.yaml
  • api/openapi.yaml is excluded by !**/openapi.yaml
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e795ba78-78d9-4bf3-9829-4f747f680389

📥 Commits

Reviewing files that changed from the base of the PR and between 6b3dcde and b4d3c54.

⛔ Files ignored due to path filters (5)
  • api/client/go/client.gen.go is excluded by !api/client/**
  • api/client/javascript/src/client/schemas.ts is excluded by !api/client/**
  • api/client/javascript/src/zod/index.ts is excluded by !api/client/**
  • api/openapi.cloud.yaml is excluded by !**/openapi.cloud.yaml
  • api/openapi.yaml is excluded by !**/openapi.yaml
📒 Files selected for processing (1)
  • api/api.gen.go

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds "request" and "response" as recognized LLM token-type aliases for "input" and "output" across specs, public constants, adapter pricing logic, and tests. No behavioral changes to pricing; only additional accepted token-type names and tests verifying mappings.

Changes

Cohort / File(s) Summary
TypeSpec Documentation
api/spec/packages/legacy/src/productcatalog/features.tsp
Documented new request and response tokenType values as aliases for input and output.
Token Type Constants & Validation
openmeter/productcatalog/feature/unitcost.go
Added LLMTokenTypeRequest and LLMTokenTypeResponse constants and included them in token-type validation and error messages.
Cost Adapter Logic & Tests
openmeter/cost/adapter/compute.go, openmeter/cost/adapter/compute_test.go
Mapped request→input pricing and response→output pricing in costPerTokenForType; added tests ensuring these mappings return expected per-token costs.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested labels

release-note/misc

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: adding request and response as aliases for existing token types in the LLM cost feature.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/llm-cost-flexibiloty
📝 Coding Plan
  • Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
openmeter/productcatalog/feature/unitcost.go (1)

125-133: Consider centralizing token-type validity to avoid drift.

Small maintainability nit: Line 126-Line 133 hard-codes the valid token set locally, while token handling is also encoded elsewhere (adapter switch). A shared helper keeps future additions safer.

♻️ Refactor sketch
-		if u.LLM.TokenType != "" {
-			validTypes := map[LLMTokenType]bool{
-				LLMTokenTypeInput: true, LLMTokenTypeOutput: true,
-				LLMTokenTypeCacheRead: true, LLMTokenTypeReasoning: true,
-				LLMTokenTypeCacheWrite: true,
-				LLMTokenTypeRequest: true, LLMTokenTypeResponse: true,
-			}
-			if !validTypes[LLMTokenType(u.LLM.TokenType)] {
-				errs = append(errs, fmt.Errorf("invalid token_type %q: expected one of input, output, cache_read, reasoning, cache_write, request, response", u.LLM.TokenType))
-			}
-		}
+		if u.LLM.TokenType != "" && !LLMTokenType(u.LLM.TokenType).IsValid() {
+			errs = append(errs, fmt.Errorf("invalid token_type %q: expected one of %s", u.LLM.TokenType, ValidLLMTokenTypesCSV()))
+		}
// outside this block
func (t LLMTokenType) IsValid() bool { /* centralized switch/set */ }
func ValidLLMTokenTypesCSV() string  { /* stable, shared message source */ }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@openmeter/productcatalog/feature/unitcost.go` around lines 125 - 133, Replace
the hard-coded local valid-token set in the validation branch that checks
u.LLM.TokenType with a centralized validator method on the LLMTokenType type
(e.g., add LLMTokenType.IsValid()) and use a shared helper that returns the
canonical CSV/string of allowed values (e.g., ValidLLMTokenTypesCSV()) for the
error message; update the check to call LLMTokenType(u.LLM.TokenType).IsValid()
and when appending the error to errs use ValidLLMTokenTypesCSV() instead of the
inline list so token validity and the error text are defined in one place.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@openmeter/productcatalog/feature/unitcost.go`:
- Around line 125-133: Replace the hard-coded local valid-token set in the
validation branch that checks u.LLM.TokenType with a centralized validator
method on the LLMTokenType type (e.g., add LLMTokenType.IsValid()) and use a
shared helper that returns the canonical CSV/string of allowed values (e.g.,
ValidLLMTokenTypesCSV()) for the error message; update the check to call
LLMTokenType(u.LLM.TokenType).IsValid() and when appending the error to errs use
ValidLLMTokenTypesCSV() instead of the inline list so token validity and the
error text are defined in one place.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e403baf5-2ec6-4c2f-b39c-a4926cd3fa37

📥 Commits

Reviewing files that changed from the base of the PR and between c23b3d5 and bd4af22.

📒 Files selected for processing (4)
  • api/spec/packages/legacy/src/productcatalog/features.tsp
  • openmeter/cost/adapter/compute.go
  • openmeter/cost/adapter/compute_test.go
  • openmeter/productcatalog/feature/unitcost.go

@tothandras tothandras merged commit 6eb43ca into main Mar 17, 2026
25 checks passed
@tothandras tothandras deleted the feat/llm-cost-flexibiloty branch March 17, 2026 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-note/feature Release note: Exciting New Features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants