Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AIOFile context manager looses data #79

Closed
termim opened this issue Mar 14, 2023 · 3 comments
Closed

AIOFile context manager looses data #79

termim opened this issue Mar 14, 2023 · 3 comments
Labels

Comments

@termim
Copy link

termim commented Mar 14, 2023

When AIOFile object is used as a context manager then some data is lost. Below is slightly modified example from README.rst:

import asyncio
from aiofile import AIOFile, LineReader, async_open


async def main():

    print("\nAIOFile:")
    async with AIOFile("/tmp/hello.txt", 'w') as afp:

        for i in range(10):
            await afp.write("%d Hello World\n" % i)

        await afp.write("Tail-less string\n")

    async with AIOFile("/tmp/hello.txt", 'r') as afp:
        async for line in LineReader(afp):
            print(line[:-1])

    print("\nasync_open:")
    async with async_open("/tmp/hello.txt", 'w') as afp:

        for i in range(10):
            await afp.write("%d Hello World\n" % i)

        await afp.write("Tail-less string\n")

    async with AIOFile("/tmp/hello.txt", 'r') as afp:
        async for line in LineReader(afp):
            print(line[:-1])


loop = asyncio.get_event_loop()
loop.run_until_complete(main())

The output is:

Python 3.8.16 (default, Dec 10 2022, 12:00:00) 
[GCC 7.5.0] on localhost
>>> 
AIOFile:
Tail-less string

async_open:
0 Hello World
1 Hello World
2 Hello World
3 Hello World
4 Hello World
5 Hello World
6 Hello World
7 Hello World
8 Hello World
9 Hello World
Tail-less string
>>> 

So it seems that async_open worked fine while AIOFile context manager lost what was written in the for loop.

@mosquito
Copy link
Owner

mosquito commented Mar 15, 2023

This code works as it should. The fact is that AIOFile is a low-level interface for asynchronous file operations, and the read and write methods accept an offset=0 in bytes at which the operation will be performed.

This allows you to do many independent IO operations on an once opened file without moving the virtual carriage.

For example, you may made 10 concurrent HTTP requests by specifying the Range header, and asynchronously write one opened file, while the offsets must either be calculated manually, or use 10 instances of Writer with specified initial offsets.

In order to provide sequential reading and writing, there is Writer, Reader and LineReader. async_open is not the same as AIOFile, it provides a similar interface for file operations, it simulates methods like read or write as it is implemented in a built-in open.

@termim
Copy link
Author

termim commented Mar 15, 2023

Thank you for the explanation!

@termim termim closed this as completed Mar 15, 2023
@mosquito
Copy link
Owner

@termim added this explanation to https://github.com/mosquito/aiofile#low-level-api

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants