Heavy use of input tokens

[JuanSombrero23]

Hi,

I have the feeling that I am 'burning tokens'. It might be because of improper context usage, I am not sure...

The reason I feel that way, is

• the pace at which I see my credits vanish on the one side (USD 1 - 2 for a relative simple action)
• the token limits I hit on the other side (see screenshot)

I have 3 items selected in the TOM explorer, 2 simple measures and a calculation group. (confirmed, see screenshot)
I use Antrophic API.
I have explicitely mentioned them as well in my prompt...yet I run into this 30k/min token limit for such a 'simple question'.

The reason I put this question here, in TE3 AI assistant, and not in any other LLM harness, is because I feel like the answer I get here should be more correct/optimal than the average answer. Feel free to comment if anything in my approach is wrong (maybe the AI assistant is not the right place to go with such questions? / maybe I do smth wrong with the context? )

Thx
Jan

[mlonsk]

Hi @JuanSombrero23
Thank you for the detailed report, and for sharing the screenshot.

You are right to be puzzled. You did everything "correctly": you selected 3 objects in the TOM Explorer and referenced them explicitly in your prompt. So let me explain what is likely happening, and why it still consumed so many tokens.

Why is token usage still high with only 3 objects selected?

When the AI Assistant starts a turn, it always sends a system prompt to the API. This includes general instructions about how to interact with Tabular Editor. We try to keep it light and focused, but it is part of the token cost. On top of that, even with a selection in TOM Explorer, the assistant may still issue tool calls to retrieve additional context. In your example, it could need to look up the full metadata of the _Sales Breakdown calculation group, including all calculation items and their DAX expressions, before it can reason about your question. In your case, the question also involved comparing three different design approaches, which likely triggered a broader metadata retrieval to reason correctly about the tradeoffs.
If you could send us your SessionID (Help -> About Tabular Editor) referencing this GitHub discussion to support@tabulareditor.com, we can investigate which tools were being called.

The 30K input token rate limit is part of Anthropic's Tier 1 for Claude Sonnet, and as you have experienced, a single complex turn can exhaust it instantly.

On the cost side ($1–2 per interaction)

This is unfortunately not unusual for a question of this nature. A part of this is down to how conversation history works. Every time you send a message, the full conversation history is resent to the API as part of the new request. So the longer the conversation, the more tokens each turn consumes, even if the new message itself is short.

Additionally, Anthropic's API does not currently support automatic prompt caching the way OpenAI's does. With OpenAI, repeated content (like the system prompt or earlier conversation turns) can be served from cache at a fraction of the cost. With Anthropic, every token is billed in full on every turn. This is a known limitation, and we are actively working on how to implement prompt caching for Anthropic-backed sessions in the AI Assistant, which should meaningfully reduce both costs and the likelihood of hitting rate limits mid-conversation. We hope to have an update out in the coming week or two to help with this.

Here are a few suggestions on what to do in the meantime:

• Start a new conversation for each distinct task. Because history is re-sent in full every turn, this is one of the most effective ways to keep costs down right now.
• Select specific objects in the TOM Explorer and keep prompts focused. You are already doing this, which is great, but breaking a multi-part question (like comparing 3 design approaches) into separate, focused turns can further reduce the metadata retrieved per call.
• Try Claude Haiku for exploratory or design questions. It has a higher Tier 1 ITPM limit (50K vs 30K for Sonnet) and is significantly cheaper per token.
• Upgrade to Anthropic Tier 2; this requires $40 in cumulative API spend and gives you 15× the input token budget (450K ITPM), which makes these rate limit errors essentially disappear for typical usage.

Is the AI Assistant the right place for this kind of question?

Yes, absolutely! Questions about model design, calculation groups, and DAX patterns are exactly what it is built for. You are not doing anything wrong. The token costs are a real limitation of where we are today with the Public Preview, and we appreciate your patience as we work through them.

We are also working on a Tabular Editor CLI that will make it easier to use TE with agent harnesses like Claude Code against your existing Claude subscription. More on that here: https://tabulareditor.com/how-tabular-editor-supports-agentic-development-for-semantic-models

Do let me know if you have further questions.

Best,
Morten

[JuanSombrero23]

Hi Morten,

that is understood, thx for clarifying that. Upgrading could do the trick, but wouldn't solve the issue that every single question is rather expensive, especially if you already pay for Claude Code max. Looking forward to the CLI and the integration into Claude Code!

One small sidenote I discovered about the UI of the AI assistant:

There is this red cross at the top of the screen. Intuitively, I clicked it a few times to 'close the window', whereas it actually deletes the conversation, and hence, I loose the conversation for good (if I am not mistaking), which was often not the desired action. I realize it's due to wrong manipulation of my side, but it feels somewhat counter intuitive.

About sending the session ID, I'm afraid I lost the session, due to the 'red cross' ;-) but if I run into it again, I will definitely share.

Keep up the great work, looking forward to further developments!

Regards,
Jan

[otykier]

Hi @JuanSombrero23
We have just released 3.26.1 which includes several optimizations for token consumption. Could you give it a go and let us know if you are still hitting rate limits?

Regarding the red "X"; feedback noted. Thanks!