OpenAI client with client timeout and parallel processing

• Retries with exponential backoof and connection reuse.
• Settings for connect/read client timeouts.
• Use multiple connections to parallelize the processing of DataFrames.

Quick Install

pip install openai-async-client

🤔 What is this?

This library is aimed at assisting with OpenAI API usage by:

Support for client side timeouts with retry and backoff for completions.

Support for parallel processing of pandas DataFrames.

Chat Completion

Example of chat completion with client timeout of 1 second to connect and 10 seconds to read with a maximum of 3 retries.

import os

from httpx import Timeout

from openai_async_client import AsyncCreate, Message, ChatCompletionRequest

create = AsyncCreate(api_key=os.environ["OPENAI_API_KEY"])
messages = [
    Message(
        role="user",
        content=f"Hello ChatGPT, Give a brief overview of the Pride and Prejudice by Jane Austen.",
    )
]
response = create.completion(ChatCompletionRequest(prompt=messages), client_timeout=Timeout(1.0, read=10.0), retries=3)

Text Completion.

create = AsyncCreate()
response = create.completion(TextCompletionRequest(prompt=f"Hello DaVinci, Give a brief overview of Moby Dick by  Herman Melville."))

DataFrame processing

Example of parallel chat completions for a DataFrame with concurrent connections.

import pandas as pd

# Define a mapping function from row to prompt
def chat_prompt_fn(r: pd.Series) -> ChatCompletionRequest:
   message = Message(
       role="user",
       content=f"Hello ChatGPT, Give a brief overview of the book {r.book_name}.",
   )
   # NOTE: Key Dict is mandatory since results are not returned in order
   key = {"author_id": r.author_id, "book_id": r.book_id}
   return ChatCompletionRequest(
       key=key,
       prompt=[message],
       system=SystemMessage(content="Assistant is providing book reviews"),
       params=OpenAIParams(model="gpt-3.5-turbo")
   )

# parallel process the DataFrame making up to max_connections concurrent requests to OpenAI endpoint
result_df = create.completions(df=input_df, request_fn=chat_prompt_fn,config=EndpointConfig.CHAT, max_connections=4)
print(result_df.head())

author_id book_id                                        book_name                               openai_id  openai_created                                  openai_completion  openai_prompt_tokens  openai_completion_tokens  openai_total_tokens api_error
0          0  76ec99  the open society and its enemies by Karl Popper  chatcmpl-6xsNdBiOHn6z7E5VAt6Smjt4sew0K      1679728409  The Open Society and Its Enemies is a landmark...                    41                       278                  319      <NA>
1          1  e39f69                         Das Capital by Karl Marx  chatcmpl-6xsNdI8PYQxClW1wn4hGbPbuxwAwV      1679728409  Of course, Das Kapital (Capital: A Critique of...                    36                       262                  298      <NA>
2          2  316382               Pride and Prejudice by Jane Austen  chatcmpl-6xsNdviulO1c1UlbaNDtH8qjBRH2w      1679728409  Sure, Pride and Prejudice by Jane Austen is a ...                    40                       181                  221      <NA>
3          3  249216                     Frankenstein by Mary Shelley  chatcmpl-6xsNd9GWaT1w7x2L3XyNFCtTLklL7      1679728409  Sure! Mary Shelley's "Frankenstein" is a gothi...                    36                       139                  175      <NA>
4          4  1d5541                    Moby Dick by  Herman Melville  chatcmpl-6xsNl3TbwsZHTagogGF65Iu6HItuS      1679728417  Certainly, Moby-Dick is a novel written by Her...                    39                       233                  272      <NA>

# result_df columns are the input_df columns plus:
# "openai_completion" - the completion/s (str/list),
# "openai_id",
# "openai_created",
# "openai_prompt_tokens",
# "openai_completion_tokens",
# "openai_total_tokens",
# "api_error" - http, openai, api error or pd.NA for successful completions.

Disclaimer

This repository has no connection to OpenAI.