Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenAIClientV1 does not implement retries #721

Open
mlevtov opened this issue Apr 18, 2024 · 0 comments
Open

OpenAIClientV1 does not implement retries #721

mlevtov opened this issue Apr 18, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@mlevtov
Copy link

mlevtov commented Apr 18, 2024

Describe the bug
Retries for transient errors are implemented for OpenAIClientV0, but not for OpenAIClientV1

To Reproduce
Steps to reproduce the behavior:

import guardrails as gd
import openai
from multiprocessing.pool import ThreadPool as Pool

text_chunk = <Chunk of text that’s ~4k tokens>

def call_gd(text_chunk):
  guard = gd.Guard(prompt="summarize this chunk of text: ${chunk}", num_reasks=5)
  result = guard(
      openai.chat.completions.create,
      prompt_params={"chunk": text_chunk},
      max_tokens=1024,
      temperature=0.1
  )

with Pool(16) as p:
  results = list(p.imap_unordered(call_gd, text_chunk))

Expected behavior
Calls should retry when receiving a RateLimitError.

Library version:
guardrails-ai==0.4.1
openai==1.13.3

Additional context
See this conversation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant