You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ChainForge currently does rate-limiting but within batches of requests to model providers. If one calls queryLLM lots of times for single calls, the rate limiting won't be enforced. To fix this, we should rate-limit at the level of the API request itself ---i.e., in call_chatgpt or call_anthropic, using a rate-limiting library that works on requests per minute (on time).
Rate limits should also be updated to reflect current standards.
The text was updated successfully, but these errors were encountered:
ChainForge currently does rate-limiting but within batches of requests to model providers. If one calls
queryLLM
lots of times for single calls, the rate limiting won't be enforced. To fix this, we should rate-limit at the level of the API request itself ---i.e., incall_chatgpt
orcall_anthropic
, using a rate-limiting library that works on requests per minute (on time).Rate limits should also be updated to reflect current standards.
The text was updated successfully, but these errors were encountered: