Skip to content

Refactor AI billing config and align credits cost handling#590

Merged
jfrench9 merged 3 commits intomainfrom
refactor/credits-cost-alignment
Apr 5, 2026
Merged

Refactor AI billing config and align credits cost handling#590
jfrench9 merged 3 commits intomainfrom
refactor/credits-cost-alignment

Conversation

@jfrench9
Copy link
Copy Markdown
Member

@jfrench9 jfrench9 commented Apr 5, 2026

Summary

Simplifies and streamlines the AI billing configuration, agent token cost handling, and credits system by removing unnecessary abstractions, consolidating logic, and eliminating the bulk export configuration. This refactor reduces the codebase by ~243 lines while maintaining equivalent functionality.

Key Accomplishments

  • Simplified AI billing configuration (config/billing/ai.py): Significantly reduced complexity by removing redundant configuration layers and flattening the billing model structure.
  • Streamlined agent token cost handling (config/agents.py, operations/agents/orchestrator.py): Removed the dedicated agents config module's excess abstractions, simplifying how agent token costs are resolved and passed through the orchestrator.
  • Consolidated credits logic (config/credits.py, operations/graph/credit_service.py): Reduced indirection in the credit service by simplifying cost lookups and removing unnecessary intermediate mappings.
  • Removed bulk export configuration (adapters/sec/manifest.py): Cleaned up the SEC adapter manifest by removing a bulk export config entry that is no longer needed.
  • Updated service offerings router (routers/offering.py): Aligned the offerings endpoint with the simplified billing and credits structures.
  • Updated documentation (config/README.md): Rewrote and condensed the configuration README to accurately reflect the simplified architecture.
  • Pruned and updated tests: Reduced test complexity to match the simplified production code, removing tests for deleted abstractions while maintaining coverage of core behavior.

Breaking Changes

  • Billing configuration shape changed: Any external consumers or scripts that depend on the previous AI billing config structure (e.g., nested model/provider cost maps) will need to be updated to align with the flattened configuration.
  • Removed bulk export config: The SEC manifest no longer includes bulk export configuration. Any workflows relying on this entry point should be verified.
  • Agent config API surface reduced: Code that previously imported granular agent cost configuration helpers from config/agents.py may need to be updated.

Testing Notes

  • Existing tests in tests/config/billing/test_ai.py, tests/config/billing/test_core.py, and tests/config/test_agents.py have been updated to reflect the simplified interfaces.
  • Verify that credit deduction and billing calculations produce consistent results in staging before merging—especially around agent orchestration flows and AI model cost resolution.
  • Confirm that the offerings endpoint returns correct data with the updated billing structures.

Infrastructure Considerations

  • Configuration deployment: Ensure any environment-specific billing or credits configuration files/overrides are updated to match the new simplified schema before deploying this change.
  • Downstream service compatibility: If any downstream services or dashboards consume billing configuration or offerings data, they should be validated against the new response shapes.
  • No new dependencies: This is a pure simplification refactor with no added dependencies or infrastructure requirements.

🤖 Generated with Claude Code

Branch Info:

  • Source: refactor/credits-cost-alignment
  • Target: main
  • Type: refactor

Co-Authored-By: Claude noreply@anthropic.com

jfrench9 added 2 commits April 4, 2026 20:48
- Removed the TOKEN_COSTS dictionary and related methods from AgentConfig, simplifying the agent configuration.
- Updated billing configuration to centralize token pricing for AI operations in billing/ai.py, with clear definitions for input and output token costs.
- Adjusted README and documentation to reflect changes in credit consumption and billing structure.
- Enhanced tests to validate the new token pricing model and ensure proper functionality across configurations.
- Adjusted monthly price and infrastructure details for the LadybugDB Standard offering to reflect new pricing and resource allocation.
- Removed the bulk export configuration from the SEC_MANIFEST, streamlining the manifest structure.
@jfrench9
Copy link
Copy Markdown
Member Author

jfrench9 commented Apr 5, 2026

@claude please review this PR

@claude
Copy link
Copy Markdown

claude Bot commented Apr 5, 2026

Claude finished @jfrench9's task in 1m 35s —— View job


PR Review: Refactor AI billing config and align credits cost handling

Todo:

  • Gather context and understand request
  • Read changed source files
  • Read changed test files
  • Analyze refactoring correctness and coverage
  • Post review feedback

Overall this is a clean, well-motivated refactor. The net reduction of ~243 lines is real simplification, not just deletion of useful code. A few things worth discussing:

Bug Fix (Important)

robosystems/operations/agents/orchestrator.py:641 — This is the most important change in the PR. The old fallback pricing was:

{"input": Decimal("0.003"), "output": Decimal("0.015")}

These are dollar amounts, not credit amounts. The correct values are 3 and 15 credits per 1K tokens, which is what TOKEN_PRICING contains. The old fallback would have undercharged by 1000x on any unrecognized model in the orchestrator's pre-flight credit check. Good catch.


Product Change Embedded in Refactor

robosystems/routers/offering.py:74,77,80 — The ladybug-standard plan changes here go well beyond a billing refactor:

  • Price: $50.00$99.00
  • Infrastructure: "Multi-tenant (shared r7g.large/xlarge)""Dedicated m7g.large (2 vCPU, 8 GB RAM)"
  • Features: "5M node limit" removed, "Priority support" added

This is a significant pricing and positioning change. It may be intentional, but it deserves its own explicit review/sign-off rather than being bundled into a billing config cleanup. Were these changes meant to be in this PR, or are they left over from another branch? Fix this →


Weak Test Replacement

tests/config/test_agents.py:140 — The removed assertion assert summary["token_cost_models"] == len(AgentConfig.TOKEN_COSTS) was replaced with:

assert "agent_capabilities" in summary

This is redundant — summary["agent_capabilities"] was already accessed and equality-checked on line 139. A more useful replacement would be:

assert "token_cost_models" not in summary

...which actually documents the intentional removal of that key. Fix this →


Narrowed Model Pricing (Deliberate, but Note the Tradeoff)

Removing Opus and OpenAI entries from TOKEN_PRICING is a valid simplification given the current constraint to Sonnet-only. The fallback in credit_service.py (warning + Sonnet pricing) handles unknown models gracefully. One thing to be aware of: if a new Bedrock model is added and nobody updates model_pricing_map, it will silently bill at Sonnet rates. The warning log is the only signal. This is acceptable risk for a controlled system, but worth noting in a follow-up CLAUDE.md note or ADR if this billing system grows.


Minor Observations

  • adapters/sec/manifest.pybulk_export removal is clean: confirmed no other references in the codebase.
  • config/credits.py docstring — Good tightening; the old docstring described "included operations" in detail that belongs with the code, not the module doc.
  • config/agents.py validation — Removing the TOKEN_COSTS validation check is correct since that dict no longer exists. The validate_configuration method is still useful for model/profile/capability counts.

Summary

Area Assessment
Core billing logic ✅ Correct and simplified
Bug fix (orchestrator fallback) ✅ Critical fix included
Test coverage ⚠️ One redundant assertion
Offering price change ⚠️ Needs explicit sign-off
Breaking changes (Opus/OpenAI removed) ✅ Acknowledged in PR description

The code is production-ready pending review of the offering price change. The billing math is consistent across all three touch points (AIBillingConfig, credit_service, orchestrator) after this PR.

- Modified the test to assert that "token_cost_models" is not present in the summary, ensuring accurate validation of the agent configuration output.
@jfrench9
Copy link
Copy Markdown
Member Author

jfrench9 commented Apr 5, 2026

@claude please review this PR

@claude
Copy link
Copy Markdown

claude Bot commented Apr 5, 2026

Claude finished @jfrench9's task in 2m 26s —— View job


Todo:

  • Gather context and understand request
  • Read changed source files
  • Read changed test files
  • Analyze refactoring correctness and coverage
  • Post review feedback

PR Review: Refactor AI billing config and align credits cost handling

This is a clean, well-scoped refactor. The ~243 line reduction is genuine simplification. A few things worth calling out:


Bug Fix (Important) ✅

robosystems/operations/agents/orchestrator.py:639-641 — The corrected fallback pricing is the most important change here. The old values:

{"input": Decimal("0.003"), "output": Decimal("0.015")}

...are dollar amounts, not credits. The new fallback:

{"input": Decimal("3"), "output": Decimal("15")}

...is correct (3 and 15 credits per 1K tokens, matching AIBillingConfig.TOKEN_PRICING). The old fallback would have undercharged by 1000x on any unrecognized model during the orchestrator's pre-flight credit check. Good catch.


Offering.py Example — Clarification on Prior Review

robosystems/routers/offering.py:74-80 — The example in the OpenAPI docstring changed from $50.00/"Multi-tenant (shared r7g.large/xlarge)" to $99.00/"Dedicated m7g.large (2 vCPU, 8 GB RAM)". This is not a product pricing changebilling/core.py already has base_price_cents: 9900 ($99) with "Dedicated m7g.large" as the infrastructure description. The router's inline doc example was stale and is now corrected to match the actual config. No sign-off needed here.


CREDIT_COSTS Partial Subset (Minor)

robosystems/operations/graph/credit_service.py:1175-1184 — The module-level CREDIT_COSTS dict maps only 8 operations from CreditConfig.OPERATION_COSTS, while CreditConfig covers ~20. Since all values are Decimal("0") and unknown operations default to 0 anyway, this is functionally equivalent. But this dict now looks like a meaningful subset when it isn't — any operation not listed still returns 0. Consider either removing the dict entirely and calling CreditConfig.get_operation_cost() directly, or documenting that it's illustrative-only. Fix this →


Test Assertion at Line 140 ✅

tests/config/test_agents.py:140 — The current code has assert "token_cost_models" not in summary, which is the right check. This correctly documents the intentional removal of token_cost_models from the validation summary. (Looks like this was already addressed in the latest commit 4c5c876.)


consume_ai_tokens Model Mapping — Worth Noting

robosystems/operations/graph/credit_service.py:1122-1130 — The model_pricing_map handles all current Bedrock model ID variants and correctly falls back to "anthropic_claude_4_sonnet" pricing. Since all models now map to the same pricing key anyway, the fallback is always correct. The warning log on unknown models (logger.warning(f"No pricing found for model {model}...")) is the right signal if a new model gets added without updating this map.


Summary

Area Assessment
Core billing logic ✅ Correct and simplified
Bug fix (orchestrator fallback 1000x undercharge) ✅ Critical fix
Offering.py example ✅ Correcting stale docstring, not a price change
CREDIT_COSTS partial subset ⚠️ Minor: slightly misleading, functionally correct
Test coverage ✅ Solid, test_agents.py:140 is correct
Removed Opus/OpenAI pricing ✅ Acknowledged, graceful fallback in place

Code is production-ready. The partial CREDIT_COSTS dict is the only nit worth cleaning up, but it's not a blocker.

@jfrench9 jfrench9 merged commit 8c9b103 into main Apr 5, 2026
7 checks passed
@jfrench9 jfrench9 deleted the refactor/credits-cost-alignment branch April 5, 2026 03:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant