Very low rate limit

[davlhd]

I love this script but my account has a rate limit of 1M tokens per day. That makes this tool useless very quickly.
Am I missing something? Even 5M seems low?

[envisean]

@davlhd I went through the exact problem. You're probably adding a lot of input context as time goes along (that's the beauty of this thing). I added $100, then $200 and Claude bumps up your tiers. Then I spoke with sales and I'm on t4 now. It only takes a day or so to get unleashed again.

I also highly recommend that you save your chat, review the md file and optimize the context that flows back in. In earlier versions of this repo, it didn't remove files from its input context as it picked up more work, so I think there's definitely some context-management that needs to be added.

@Doriandarko is killing it though with this repo, it's insane. Hope that helps!

[PierrunoYT]

You would need OpenRouter

[ovachiever]

I have OpenRouter working through a new API menu but tools not working well due to request/response differences. Going to look into it more and/or just have it route main and not tools through OR as an option when that endpoint is selected.

Another option, OP, is to set the tools usage to the haiku model, that'll reduce usage quite a bit and haiku seems to work well for tools.

[ff781]

How long do they take to respond for your request to upgrade tiers?

[PierrunoYT]

How long do they take to respond for your request to upgrade tiers?

They dont upgrade Tiers. I also asked for it.

[ff781]

Well rip, what do we do then? How are people using this tool, self-hosted? LLAMA 405B?

[ovachiever]

Anthropic sales/support is extremely lackluster. That being said, usage over time will raise your Tier level; I believe at Tier 3 I got to 5mm requests per day for 3.5; don't think it changed at Tier 4 which I'm now at. So keep using it and at least it'll rise over time.

[ff781]

Oh, so the "solution" is to just max out the usage everyday, go take a break, then come back and repeat tomorrow and it goes up eventually?

[ff781]

Another option, OP, is to set the tools usage to the haiku model, that'll reduce usage quite a bit and haiku seems to work well for tools.

@ovachiever

also, haiku apparently has a very low output limit?

Error in tool response: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'max_tokens: 8000 > 4096, which is the maximum allowed number of │
│ output tokens for claude-3-haiku-20240307'}}

[ovachiever]

Oh, so the "solution" is to just max out the usage everyday, go take a break, then come back and repeat tomorrow and it goes up eventually?

Correct, here are the tiers: https://docs.anthropic.com/en/api/rate-limits

They finally got back to me from contacting a couple weeks ago and bumped me to Tier 5 50,0000,000, a 10x increase. I'd min/max usage AND reach out with a plea and what you're working on and see if they don't perhaps kick you up a few tiers. May help to find a few Benjamins in account as well, to show you're serious?

[ovachiever]

Another option, OP, is to set the tools usage to the haiku model, that'll reduce usage quite a bit and haiku seems to work well for tools.

@ovachiever

also, haiku apparently has a very low output limit?

Error in tool response: Error code: 400 - {'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'max_tokens: 8000 > 4096, which is the maximum allowed number of │

│ output tokens for claude-3-haiku-20240307'}}

Ya you'd need to exit the script and bump tool output tokens to 4096, when using Haiku.

[unnir]

please add OpenRouter