File descriptor leak

[erees1]

I'm guessing this is just not recommended to init the Client on each api call like this but if you are to do something like the below you get an ever growing number of open files as indicated by lsof until I hit an OSError.

from src import anthropic

import os


def send_something():
    prompt = "\n\nHuman: This is a test prompt. Return some randome text\n\nAssistant:"

    client =  anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
    resp = client.completions.create(
        prompt=prompt,
        stop_sequences=[anthropic.HUMAN_PROMPT],
        model="claude-v1",
        max_tokens_to_sample=100,
    )
    return resp.completion

if __name__ == "__main__":
    client = None
    for i in range(1000):
        print(send_something())

I can fix this by calling client._client.close() but seems like it would be nicer to expose a close() method or enable use of the client as a context manager. Suggestion here: #83

Or if this is just totally the wrong way to use the client maybe this could be mentioned in the Readme.

Thanks!

[thejaminator]

i think this would be pretty good since the anthropic client wraps a httpx client and usually we want to use it like a context manager too

with httpx.Client() as client:
    ...

[rattrayalex]

Thanks for filing! You're correct that the library is intended to be instantiated once, not on every request, but I expect we'll add .close() and support for a context manager shortly.

[rattrayalex]

Out of curiosity, can you share why you needed/wanted to instantiate the client in a loop like this?

[rattrayalex]

Sorry for the delay here, this is out in https://github.com/anthropics/anthropic-sdk-python/releases/tag/v0.3.7 and will be published to pypi soon.

[deter3]

I have tested anthropic 0.3.11 and 0.2.9 , both might have memory leak .

code 1 :
`@retry(stop=stop_after_attempt(5), wait=wait_exponential(max=10), retry=retry_if_exception_type((ConnectionResetError,anthropic.InternalServerError) ))

def send_something():
prompt = "\n\nHuman: This is a test prompt. Return some randome text\n\nAssistant":
client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
resp = client.completions.create(
prompt=prompt,
stop_sequences=[anthropic.HUMAN_PROMPT],
model="claude-v1",
max_tokens_to_sample=100,
)
client.close()
return resp.completion

if name == "main":
client = None
for i in range(10000):
print(send_something())`

code 2 :

`@retry(stop=stop_after_attempt(5), wait=wait_exponential(max=10), retry=retry_if_exception_type((ConnectionResetError,anthropic.InternalServerError) ))

def send_something():
prompt = "\n\nHuman: This is a test prompt. Return some randome text\n\nAssistant:"
resp = client.completions.create(
prompt=prompt,
stop_sequences=[anthropic.HUMAN_PROMPT],
model="claude-v1",
max_tokens_to_sample=100,
)
return resp.completion

if name == "main":
client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
for i in range(10000):
print(send_something())`

Both code led to File descriptor leak eventually after 10000-20000 loops . I ever tried to remove the retry and the same File descriptor leak happened . When changed to use

headers = {"x-api-key": f'{api_key}', "Content-Type": 'application/json'} data = {"prompt": f'\n\nHuman: {prompt1}\n\nAssistant: ',"model": 'claude-instant-1.2',"max_tokens_to_sample": 10000,"temperature":0} r = requests.post('https://api.anthropic.com/v1/complete', headers=headers, json=data,timeout=500)

everything is fine . no File descriptor leak at all .