Support for concurrent dataframe operations with async await

[scheung38]

Support for concurrent Panda Dataframe GET access

Does FastAPI support concurrent Dataframe read/GET access to various different columns for validation checks, where I am not expecting any dependencies between the columns.

i.e after df is created, instead of sequentially calling

check_for_NA
check_account_types
check_duplicates

how to async trigger these above and collect results are ready and perform a AND function

Environment

• OS: macOS: 11.1 Big Sur
• FastAPI Version fastapi 0.63.0:

(.venv) pyth3 --version
Python 3.9.1

[Mause]

Firstly, Fastapi does not have any integration with pandas.

To answer your question though, you could run those operations in a thread pool, and perform the AND once they all complete? I would be surprised if this actually improved performance though

[scheung38]

@Mause thanks I know there is no integration with Pandas, but based on the async await nature that FastApi handles I thought I'd ask to see if there are benefits, if so how to approach it.

[Mause]

Pandas doesn't have async support, so your only option here is to use threads, though as I said, it's not guaranteed that this will give a performance improvement

[hellocoldworld]

This is not related in any way to fastapi, but it's an interesting question. I would encourage you to post this question on a site like StackOverflow

[raphaelauv]

@scheung38 if you agree with the answers , could you close the issue , thank you

[tiangolo]

Thanks for the help here everyone! 👏 🙇

If that solves the original problem, then you can close this issue @scheung38 ✔️

Sorry for the long delay! 🙈 I wanted to personally address each issue/PR and they piled up through time, but now I'm checking each one in order.

[github-actions]

Assuming the original need was handled, this will be automatically closed now. But feel free to add more comments or create new issues or PRs.