-
Notifications
You must be signed in to change notification settings - Fork 13.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migrations for 0.11 hangs with posgresql #1260
Comments
Was there any follow-up to this issue? I'm experiencing the same challenge, upgrading the database from eca4694defa7 to 3b626e2a6783 hangs... Postgres Version 9.5.3. Has anyone else been able to successfully upgrade the DB? |
What I don't understand is that the builds on Postgres go through the migrations without any issues. Maybe it's because the tables are empty while they run on our side? A Postgres version issue? It may help to find which exact line of the migration script hangs and research that specifically. |
I've done a bit more fiddling with this issue this AM and found one interesting tidbit: If the migration is run one-step at a time in sequence, the upgrade succeeds without hanging. Perhaps there's something wrong with the branch/merge? My starting point was 'eca4694defa7' However, running each upgrade individually, one-after-the-other worked just fine: This isn't so much a fix as it is a workaround as I can't really explain what is going wrong with the straight upgrade... but it solved my immediate issue. |
I succeeded in upgrading by using @woel0007 's method of manual upgrades. However at multiple points I had to kill 'idle in transaction' postgres PIDs using select pg_terminate_backend(PID) as per #238 (I hope a better way is found). |
I got the behavior of #238 in 0.12 when doing a fresh install. The first migration produced the deadlock. All following migrations did succeed. |
Did anyone every make superset work with postgres without manually killing transactions? |
nope, had to kill the queries |
if it can help, I am experiencing the same problem. starting from blank database, had to kill a couple of connections from time to time. using postgres 9.4.7 |
anyone look into this? And the query is:
|
tldr; adding I have also experienced this bug. I think it happens because:
The final call is executing a query against the database, and because of the Python dbapi transaction semantics we now have an open, idle transaction which blocks the It's not clear what the best place to fix it is.
|
@rhunwicks thanks for the analysis! Can we add the session cleanup in a overriden
|
Is there any way we can get some attention or love on this issue? It essentially limits our ability to use superset in any serious way. I'd be happy to send a PR but it's not at all clear where to solve the issue |
@jquense whatever fix the get merged in whatever project that fix installation for postgresql is good :) |
Currently using MySQL so it's hard to reproduce or justify doing the work on my side. Sounded like it's a FAB bug so it should be addressed there. I'm a committer on that project so I can review on that repo too. |
Quick note: Personally encountered hanging with postgresql 9.2. However everything went smoothly with 9.6/10.1. |
Encountered hang with postgres 9.4, with postgres 9.6 it works perfectly. |
hangs with mysql v 8.0.11 on osx 10.13 |
hangs with mysql v 5.7.23 on osx 10.14 |
confirmed hangs with mysql v8.0. |
Great investigation. These are essentially my findings also. Ideally, reusing the connection would be nice but difficult to implement. I implemented an alternative #7417 to using sm.get_session.close() bu leveraging an app context that closes when going out of scope. |
OS Debian GNU/Linux 8
Postgresql 9.5
Caravel version 0.11.0
To reproduce :
pip install caravel --upgrade
caravel db upgrade
Additional information
Postgres activity will migration hangs :
Migrations logs :
Edit :
First query to hangs is :
But even after killing it, migration still hangs at the same point
The text was updated successfully, but these errors were encountered: