Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python: op.write() seems have memory leak #2455

Closed
Xuanwo opened this issue Jun 13, 2023 · 3 comments
Closed

python: op.write() seems have memory leak #2455

Xuanwo opened this issue Jun 13, 2023 · 3 comments
Labels
bindings/python bug Something isn't working

Comments

@Xuanwo
Copy link
Member

Xuanwo commented Jun 13, 2023

Write a 2GiB file will need 6GiB memory.

import opendal
import asyncio
import gc

op = opendal.AsyncOperator("s3", xxx)
print(f"operator: {op}")

async def w2minio(f):
    fd = open(f,'rb')
    await op.write(f, fd.read())
    fd.close()
    gc.collect()
    print("end")

for i in range(3,7):
    f=f"x{i}"
    print(f)
    asyncio.run(w2minio(f))

img_v2_e9a084bf-6ee5-4d7d-a8f0-240369fe285g

img_v2_20147e34-5a24-4667-be13-d8717ff03c8g

@Xuanwo Xuanwo added bug Something isn't working bindings/python labels Jun 13, 2023
@wubx
Copy link

wubx commented Jun 13, 2023

image

@messense
Copy link
Member

Probably caused by PyO3/pyo3#1056

@Xuanwo
Copy link
Member Author

Xuanwo commented Dec 6, 2023

We have File support now. Let's close.

@Xuanwo Xuanwo closed this as completed Dec 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bindings/python bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants