Rate Limit

[kmransom56]

Which API Provider are you using?

Anthropic

Which Model are you using?

Claude 3.5 Sonnet)

What happened?

429 {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 input tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}

Steps to reproduce

1.Having cline rewrite the code it seems that 600 lines of code is all it can handle then it mangles the code making it unusable
2. Seems like paying for a service that can only handle 600 lines of code is a waste of money.
3. How to get around this limit so I can use the tool that I pay for ?
4.

Relevant API REQUEST output

429 {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 input tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}

Additional context

429 {"type":"error","error":{"type":"rate_limit_error","message":"This request would exceed your organization’s rate limit of 80,000 input tokens per minute. For details, refer to: https://docs.anthropic.com/en/api/rate-limits; see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}

[watzon]

This issue seems to be related to the amount of context being sent to the API every time. The further you get in a conversation the more context there is, and by default Anthropic imposes a limit of 40,000 tokens up (though it seems like you have 80,000 somehow).

One potential fix for this would be to just have Cline auto retry when it gets a rate limit error. Another would be to make it so that Cline heavily truncates the amount of context being sent back to the API (I believe Cursor and Windsurf do this). Neither solution is perfect though. Truncating the context leads to far more bad outcomes; while an auto-retry means that you're going to be sitting there and waiting a while for your response.

[kmransom56]

How do I implement this? I like the app but when I use it and breaks the code midstream sometimes difficult to fix the problem as the code is in complete.

…

On Mon, Dec 16, 2024, 00:58 Chris Watson ***@***.***> wrote: This issue seems to be related to the amount of context being sent to the API every time. The further you get in a conversation the more context there is, and by default Anthropic imposes a limit of 40,000 tokens up (though it seems like you have 80,000 somehow). One potential fix for this would be to just have Cline auto retry when it gets a rate limit error. Another would be to make it so that Cline heavily truncates the amount of context being sent back to the API (I believe Cursor and Windsurf do this). Neither solution is perfect though. Truncating the context leads to far more bad outcomes; while an auto-retry means that you're going to be sitting there and waiting a while for your response. — Reply to this email directly, view it on GitHub <#923 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI6SDFDJQUBIDTBFNEHTS5D2FZTYZAVCNFSM6AAAAABTUV2WN6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBUGY3TCMBZHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[eragooon]

I am also running into this problem every call.
I asked clint what it received in every prompt and got this answer:
I cannot directly show you the full prompt as that would be a form of prompt injection. However, I can explain what I receive in each message:

System Instructions:

Tool usage guidelines
Rules for operation
Available tools and their parameters
MCP server information
System information (OS, shell, directories)
My role and capabilities
User Message:

The actual message/task from the user
Any feedback from previous tool uses
Environment Details:

VSCode Visible Files
VSCode Open Tabs
Current Working Directory Files (when included)

This seems to be alot of information for every call.

Is there a way to show the sent prompt?

[watzon]

I am not hitting rate limit issues with https://github.com/RooVetGit/Roo-Cline

[lyjmry]

How do I implement this? I like the app but when I use it and breaks the code midstream sometimes difficult to fix the problem as the code is in complete.
…
On Mon, Dec 16, 2024, 00:58 Chris Watson @.> wrote: This issue seems to be related to the amount of context being sent to the API every time. The further you get in a conversation the more context there is, and by default Anthropic imposes a limit of 40,000 tokens up (though it seems like you have 80,000 somehow). One potential fix for this would be to just have Cline auto retry when it gets a rate limit error. Another would be to make it so that Cline heavily truncates the amount of context being sent back to the API (I believe Cursor and Windsurf do this). Neither solution is perfect though. Truncating the context leads to far more bad outcomes; while an auto-retry means that you're going to be sitting there and waiting a while for your response. — Reply to this email directly, view it on GitHub <#923 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI6SDFDJQUBIDTBFNEHTS5D2FZTYZAVCNFSM6AAAAABTUV2WN6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBUGY3TCMBZHA . You are receiving this because you authored the thread.Message ID: @.>

You can instruct it to split the code into multiple shortcode modules

[donghaozhang]

You can use the open router of claude sonnet 3.5. The limit is gone.

[wedow]

One potential fix for this would be to just have Cline auto retry when it gets a rate limit error. Another would be to make it so that Cline heavily truncates the amount of context being sent back to the API (I believe Cursor and Windsurf do this). Neither solution is perfect though. Truncating the context leads to far more bad outcomes; while an auto-retry means that you're going to be sitting there and waiting a while for your response.

An auto-retry option can still be really beneficial in some use cases. I'm having Cline analyze and document the flow of data in a legacy app. I don't really care if it takes 8 hours to do this but manually clicking Retry every 5 minutes to deal with rate limiting is incredibly tedious.

[ndbroadbent]

I am not hitting rate limit issues with https://github.com/RooVetGit/Roo-Cline

Thanks @watzon, I'll try that out.

You can use the open router of claude sonnet 3.5. The limit is gone.

@donghaozhang How do you do this? Is it something I can configure in the Anthropic API console? or Cline?

EDIT: TIL about https://openrouter.ai. Thanks!

[saoudrizwan]

Glad you guys could help eachother out, thank you for stepping in to give advice.

[hasen6]

You can use the open router of claude sonnet 3.5. The limit is gone.

How do you use the credits you have with anthropic with openrouter? Otherwise you have credits sitting there that will never be used.

[hasen6]

I am not hitting rate limit issues with https://github.com/RooVetGit/Roo-Cline

Tried Roo Cline today and initially didn't hit the rate limit but just now I did. Why does it hit a rate limit when only editing a very short file? It's obviously not actually hitting any limits.

[karyslav]

For how long and how many repeated attemps to edit it? It is collecting context for the whole time. If you ask it to change something 30times in small file it can be hunded or thousands of tokens because the previous context. If I just cut the task into smaller ones i do not hit any limit of course.
It looks like a lot of people have absolutely no idea how LLM AI works.

[hasen6]

For how long and how many repeated attemps to edit it? It is collecting context for the whole time. If you ask it to change something 30times in small file it can be hunded or thousands of tokens because the previous context. If I just cut the task into smaller ones i do not hit any limit of course. It looks like a lot of people have absolutely no idea how LLM AI works.

No nothing like that, I only do very short requests 'cos none of it is free. Normally it's just a single task or perhaps two. Normally it fails on the first edit. One time it failed I retried it two or three times and then it managed to do it. Most of the time it fails no matter how many times I retry it. Roo cline certainly works better but I still gave up on it since it fails a lot too.