New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Autogenerate detected removed index, for constraints #157
Comments
Michael Bayer (@zzzeek) wrote: we tried really hard to get this right and there is a significant, explicit effort to make sure PG's uniques/indexes line up, including tests for this specific pattern (see https://bitbucket.org/zzzeek/alembic/src/85a21ed6cc637efc2e9705a30d397a194acb9742/tests/test_autogenerate.py?at=master#cl-818, https://bitbucket.org/zzzeek/alembic/src/85a21ed6cc637efc2e9705a30d397a194acb9742/tests/test_autogenerate.py?at=master#cl-911). so knowing what PG version and what SQLAlchemy version here (assuming you're on the latest Alembic) would be helpful. |
Michael Bayer (@zzzeek) wrote: I'm testing over here and I'm not seeing it. Using a model like this:
I run autogenerate for rev 1 and as expected we get this:
run that upgrade. So now, we have the unique constraints in the DB (which PG sees as indexes) and we have the unique=True in the model, which I also made sure I applied on an FK column as you have above. run autogenerate again, I get:
So need all the data here - PG/SQLA/alembic versions, and also SQL output of what you get when you run |
Michael Bayer (@zzzeek) wrote: just to check, as I notice your constraint appears to be named on the PG side, I gave it a name in my test:
the constraint remains unnamed when I use "unique=True". but a run against that still had the right answer, as we compare the indexes/constraints on the names of the columns within, if either side has no name. |
Michael Bayer (@zzzeek) wrote: specifically, if you run this for a particular table with a unique, you should see results for both indexes and unique constraints:
otherwise it is sounding like those constraints were created on the DB side as "unique index" alone - the fact that they are named on the DB side and not in your Python side suggests a mismatch between your model and what's actually in the DB. if you add "index=True" to those columns, then you'll get a unique index without the UNIQUE constraint in the Python model. If I run a test like this, e.g. first I create the schema using "unique=True, index=True" on the column. then I remove the "index=True" part, then I run another autogenerate, I get:
which in fact is correct - by removing "index=True", it means I no longer want an index, i want just the unique constraint. which PG then creates as a pair of "unique constraint" / "index" as it does, but that's fine. the schema and model are remaining consistent with each other. please confirm these are in fact just unique indexes in your PG database and that there's not an actual unique constraint, e.g. that adding index=True resolves, and we can close this, thanks ! |
John Kida (@jkida) wrote: I am running SQLAlchemy 0.8.0 , PostgreSQL 9.1.10 I tried running the test you mentioned, but it looks like i dont have a get_unique_contraints() method in SA 0.8.0
PG sees this as type = index, so that looks correct to me. This column was always ONLY a unique contraint, it was never a index=True. Im not sure i understand exactly what I should confirm, I believe these are unique constraints.. the DB shows
They should be just unique constraints, im just not sure why autogenerate is appearing to want to remove them, or thinks it needs to drop an index. Sorry if this wasnt helpful i can try to do some testing to reproduce it, from scratch. |
Michael Bayer (@zzzeek) wrote: OK the first thing to understand is that when a "UNIQUE CONSTRAINT" is created on postgresql, PG will automatically create an additional UNIQUE INDEX at the same time. So generating a schema with "unique=True" means that PG will report on two constraints - one is "UNIQUE CONSTRAINT" and the other "UNIQUE INDEX". so that's the confusion that the new index comparison feature has to deal with. Next, I didn't realize yesterday that we didn't even port the "unique constraint" feature to 0.8 at all, and there's no good reason for that so this is now backported to 0.8.4. So for the moment, your issue should resolve if you try out 0.8.4, which I can release later in the week if needed. you can get current 0.8 at https://bitbucket.org/zzzeek/sqlalchemy/get/rel_0_8.zip - let me know if the issue is resolved if you get the chance to try it out. |
John Kida (@jkida) wrote: Mike, Thanks for the great work! |
Michael Bayer (@zzzeek) wrote: hi john - this would normally be very early to push 0.8.4 but since this issue is going to hit everyone using alembic I'll try to get it out by the weekend.
|
rachidb (@rach) wrote: This ticket seem to be focus on Postgres. |
Michael Bayer (@zzzeek) wrote: not sure, any chance you want to try it out for me? |
Zeyi Fan (@fanzeyi) wrote: Same issue here. Not PSQL, I use MySQL. |
Zeyi Fan (@fanzeyi) wrote: I have a issue with MySQL that is similar to this issue, so I report it here. Before start: I'm new to alembic, and I'm not familiar with some concepts in database. Library version: Python 2.7.5
MySQL 5.6.10 All codes below runs inside virtualenv and uses InnoDB as database engine. Two parts: I. Database Structures:
Problem: When I run alembic auto-generate first time, It seems normal. But when I run it again (just after upgrading the database, nothing changed), it will get I solved it by removing the II. Database Structures:
Problem: Like part I, generate it, upgrade it and re-generate it. I got |
Michael Bayer (@zzzeek) wrote: I've identified issues with this logic in the current master due to a related fix, but the issue is not in 0.6.1. if you've removed "index=True", that would be why you're getting a drop_constraint() - that flag is why there is an "ix_user_name" in the first place. put the index=True back in the Column. I will add additional tests to ensure no net change is detected. |
Michael Bayer (@zzzeek) wrote: confirmed that the dedupe logic which takes place for PG has to be completely reversed for MySQL, which instead of reporting uniques as indexes, reports indexes as uniques. I'm rewriting the whole index autogenerate feature right now. |
Changes by Michael Bayer (@zzzeek):
|
Michael Bayer (@zzzeek) wrote: if everyone mentioned in this ticket can please test against master today, I can get the release out ASAP. |
AlexK (@Osleg) wrote: Still happens for me, getting drop_index in migration script even if drop is not intended in models. |
Michael Bayer (@zzzeek) wrote: did you restore that |
Michael Bayer (@zzzeek) wrote: just tried it here (not doing the Model #1:
autogen #1:
model #2:
autogen #2:
model #3 - no change, just run revision --autogenerate again autogen #3:
|
Michael Bayer (@zzzeek) wrote: also the above is against MySQL 5.5.29. |
AlexK (@Osleg) wrote: We didn't talk about it, it was someone else :)
And here if I remove (for example, though it happens for any change) |
AlexK (@Osleg) wrote: The above is against psql (PostgreSQL) 9.3.2 |
Michael Bayer (@zzzeek) wrote: Whoops! OK before I saw your Postgresql note, just tried it on MySQL, no issues. Using Postgresql. Model 1:
autogenerate #1:
Model 2 - no change. Autogenerate #2, no changes:
Model 3 - remove
autogenerate #3: we get just the column that was changed, no
so please make sure you are A. at least on SQLAlchemy 0.8.4 and B. using the git master of Alembic as these fixes aren't released yet. |
Michael Bayer (@zzzeek) wrote: although I have identified that we now are generating the UNIQUE constraint twice in the initial autogenerate. need to tweak that. |
AlexK (@Osleg) wrote: First of all sorry, my bad, i'm on MySQL here, psql is on production.
autogenerate #1
Model no change:
Model change:
And again sorry for confusion the database is |
Michael Bayer (@zzzeek) wrote: work is ongoing |
Changes by Michael Bayer (@zzzeek):
|
AlexK (@Osleg) wrote: Upgraded:
First autogenerate
No changes:
Changed nullable=False to True
Worked perfectly i guess. |
Michael Bayer (@zzzeek) wrote: OK I'm pretty confident we've at least improved the situation greatly here |
Changes by Michael Bayer (@zzzeek):
|
Migrated issue, originally created by John Kida (@jkida)
(Postgresql)
It appears everywhere I have a constraint in my sqlalchemy code in which there is a unique constaint index alembic --autogenerate wants to remove it.
ie.
python code:
lead_id = Column(Integer, ForeignKey('crm_lead.id'), unique=True)
DataBase \d:
Indexes:
"chat_pkey" PRIMARY KEY, btree (id)
"chat_lead_id_key" UNIQUE CONSTRAINT, btree (lead_id)
INFO [alembic.autogenerate.compare] Detected removed index 'chat_lead_id_key' on 'chat'
It appears alembic does not consider constraints as indexes, and psql does.
The text was updated successfully, but these errors were encountered: