Commit 7201e79
committed
feat(anthropic): cache-aware estimateCost + surface cacheRead/CreationInputTokens
Previously estimateCost ignored prompt-cache tier entirely and just
charged input_tokens × base rate. For cached workloads this:
- under-counted cache_creation_input_tokens (should be 1.25× base)
- silently dropped cache_read_input_tokens (should be 0.10× base)
Net effect: a caching-heavy run's reported cost was ~10-15% below the
true billed amount, and the cost telemetry could not show cache savings
because it had no visibility into cache usage at all.
Fix:
- estimateCost now takes optional cacheReadTokens + cacheCreationTokens
and bills each at its Anthropic rate (0.10× / 1.25× input price).
5-minute TTL assumed; 1-hour TTL costs 2× but is not distinguishable
from response data, so cost slightly under-estimates for long-TTL
caches (minor; documented in the estimateCost JSDoc).
- ModelCompletionResponse.usage now threads through to generateText's
TokenUsage.cacheReadTokens / cacheCreationTokens (generateText.ts
changes shipped earlier in this branch).
Tests: 6 new tests in AnthropicProvider.cache.test.ts covering the
formula, the savings math on a cache-heavy run vs cold run, and the
three tier-pricing branches. All 11 cache tests pass.1 parent b68efcd commit 7201e79
2 files changed
Lines changed: 132 additions & 4 deletions
File tree
- src/core/llm/providers
- __tests__
- implementations
Lines changed: 98 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
118 | 118 | | |
119 | 119 | | |
120 | 120 | | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
Lines changed: 34 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
947 | 947 | | |
948 | 948 | | |
949 | 949 | | |
| 950 | + | |
| 951 | + | |
950 | 952 | | |
951 | 953 | | |
952 | 954 | | |
| |||
1027 | 1029 | | |
1028 | 1030 | | |
1029 | 1031 | | |
| 1032 | + | |
| 1033 | + | |
| 1034 | + | |
| 1035 | + | |
| 1036 | + | |
| 1037 | + | |
| 1038 | + | |
| 1039 | + | |
| 1040 | + | |
| 1041 | + | |
| 1042 | + | |
| 1043 | + | |
| 1044 | + | |
| 1045 | + | |
| 1046 | + | |
| 1047 | + | |
| 1048 | + | |
| 1049 | + | |
| 1050 | + | |
| 1051 | + | |
| 1052 | + | |
| 1053 | + | |
| 1054 | + | |
1030 | 1055 | | |
1031 | 1056 | | |
1032 | 1057 | | |
1033 | 1058 | | |
| 1059 | + | |
| 1060 | + | |
1034 | 1061 | | |
1035 | 1062 | | |
1036 | 1063 | | |
1037 | | - | |
1038 | | - | |
1039 | | - | |
1040 | | - | |
| 1064 | + | |
| 1065 | + | |
| 1066 | + | |
| 1067 | + | |
| 1068 | + | |
| 1069 | + | |
| 1070 | + | |
1041 | 1071 | | |
1042 | 1072 | | |
1043 | 1073 | | |
| |||
0 commit comments