Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upset best practice? #420

Closed
Jflick58 opened this issue Mar 22, 2019 · 3 comments
Closed

Upset best practice? #420

Jflick58 opened this issue Mar 22, 2019 · 3 comments

Comments

@Jflick58
Copy link

Hi,

I'm currently upserting around 2.5 million rows to Postgres. I replaced my previous use of Sqlalchemy + psycopg2 with Asyncpg, but I'm not seeing the performance gains I expected. I have a complex query statement that utilizes ON CONFLICT to upset. SQLalchemy has an ORM method for this, but I don't see anything equivalent in the Asyncpg docs. I currently use executemany and pass a list of tuples that were converted from a Pandas data frame. I don't think I would be able to perform my upset function using copy_records_to_table, can you advise on how to approach this with Asyncpg?

@elprans
Copy link
Member

elprans commented Mar 22, 2019

#295 should boost the executemany() manyfold. I'll try to find time to get it merged soon.

@pkeeper
Copy link

pkeeper commented Jul 19, 2019

Please do.
We've chosen asyncpg for it's performance, and we have a lot of upserts, that are working hacky now.

@victoraugustolls victoraugustolls mentioned this issue Jan 8, 2020
2 tasks
@elprans
Copy link
Member

elprans commented Nov 26, 2020

#295 has been merged, executemany performance should be much better in the next release.

@elprans elprans closed this as completed Nov 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants