How to adjust timeout for contextual completions?

[Just-enki]

Im moving my pgpt requests to a python api but unlike typescript I cannot change the default timeout setting, whereas in Typescript I could simply pass const requestOptions = { timeoutInSeconds: 600, maxRetries: 1 }; with my request. This doesn't seem to be an option here.

This has worked for me when working in typescript

async function getChatResult(chatMessages: ChatMessage[], requestOptions?: ContextualCompletions.RequestOptions): Promise<ChatResult> {
  const pgptResponse = await pgptApiClient.contextualCompletions.chatCompletion(
    {
      messages: chatMessages,
      includeSources: true,
      useContext: true,
    },
    requestOptions
  );
  return pgptResponse;
}

Now ive switched over to Python and seemingly the only way to change the values is to enter the docker container and edit resources/contextual_completions/client.py manually.

So far ive tried:
1.

client = PrivateGPTApi(base_url="http://private-gpt:8001/", timeout=600)

httpx_client = httpx.Client(timeout=httpx.Timeout(600.0, connect=600.0, read=600.0, write=600.0))

client = PrivateGPTApi(
    base_url="http://private-gpt:8001/",
    httpx_client=httpx_client 

client = PrivateGPTApi(
    base_url="http://private-gpt:8001/",
    timeout=httpx.Timeout(600.0, connect=600.0, read=600.0, write=600.0)

@app.post("/gpt/chat")
async def gptChat(chat_request: ChatRequest):
    try: 
        chat_result = client.contextual_completions.chat_completion(
            messages=chat_request.messages, 
            use_context=True,
            include_sources=True,
            timeout=600,
        )
        return chat_result.choices[0].message.content

all with no success. Is there something im missing?

[kparnis3]

Any updates on this?, running into this issue too.

[Just-enki]

@kparnis3
sorry for late reply, im running my api in docker so the workaround that has worked for me was to copy the original client.py, edit it and mount it as a volume in docker-compose.yaml to override the default settings. Unfortunately i couldn't get anything else to work.

it looks like this inside my docker-compose
- ./client.py:/usr/local/lib/python3.12/site-packages/pgpt_python/resources/contextual_completions/client.py