Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed issue with upgrading metadata table in AWS Redshift #1231 #1256

Merged
merged 4 commits into from Apr 18, 2016

Conversation

@nathanvick
Copy link
Contributor

@nathanvick nathanvick commented Mar 24, 2016

This fixes the issue where upgrading the metadata table (from Flyway v3.2.1 to v4.0) failed, for at least some installations of AWS Redshift. Redshift does not support altering columns within a table. There are two common workarounds for this:

  • Copying data between temp columns: adding a temp column, copying data to the temp column, dropping the original column, adding the updated column, copying data back to the updated column, dropping the temp column
  • Copying data between temp tables: renaming the existing table, creating the updated table, copying data to the updated table, dropping the renamed table

This pull request changes the approach from "copying data between temp columns" to "copying data between temp tables" because the former approach requires a commit half way through the process, for some installations of Redshift.

nathanvick added 2 commits Mar 5, 2016
Flyway couldn't upgrade the metadata table, due to the following error:
"Amazon Invalid operation: cannot insert/update into table after dropping
non-nullable column"

This was caused by trying to remove the NOT NULL constraint on the version
column by adding/dropping a temp columan and copying the values. For somae
unknown reason, this works on some versions of Redshift, but not others.
The solution is just to rename the original table, create a new one, copy
the data, and drop the original table.
Redshift does not support indexes because it is a column-oriented
database. Instead data can be physically stored sorted, to improve
reading and sorting performance, based on the sort key.
@wgillett

This comment has been minimized.

Copy link

@wgillett wgillett commented on 4f0b1d8 Mar 6, 2016

Looks good to me. I just coded something very similar! I'll see if I can test your change instead.

This comment has been minimized.

Copy link

@wgillett wgillett replied Mar 6, 2016

Tried it, failed on the table rename. Apparently renaming a table with the schema specified like this doesn't work (the dot in "schema"."table" is considered a syntax error), see:

http://stackoverflow.com/questions/27787741/postgresql-how-to-rename-a-table-inside-a-schema

for reports of a similar problem. Good news is that when I removed the schema, changing the line

ALTER TABLE "${schema}"."${table}" RENAME TO "${schema}"."${table}_flyway4_upgrade";

to

ALTER TABLE "${table}" RENAME TO "${table}_flyway4_upgrade";

then the script worked. That does leave open the possibility of mistakenly renaming a similarly named table in a different schema. Not a problem in my context but don't know if it would make others unhappy.

Walter

Here's the exception I got:

org.flywaydb.core.internal.dbsupport.FlywaySqlScriptException:

Script failed

SQL State : 42601
Error Code : 500310
Message : Amazon Invalid operation: syntax error at or near ".";
Line : 17
Statement : ALTER TABLE "public"."schema_version" RENAME TO "public"."schema_version_flyway4_upgrade"

This comment has been minimized.

Copy link
Owner Author

@nathanvick nathanvick replied Mar 7, 2016

Thanks for the feedback. I'm guessing it would also be possible to keep the schema in the old object name and just remove it from the new object name:

ALTER TABLE "${schema}"."${table}" RENAME TO "${table}_flyway4_upgrade";

I pushed a new commit with just this change: 7050f3f

…o 4.0 failed

When renaming a table, the schema should not be specified in the
new table name.
@wgillett

This comment has been minimized.

Copy link

@wgillett wgillett commented on 7050f3f Mar 7, 2016

That should work fine. I don't have time to retest the full Flyway core right now, but I did confirm on our Redshift DB that while this command fails:

alter table "public"."foo" rename to "public"."fizzle";

this one works - removing the schema qualifier only on the target table name:

alter table "public"."foo" rename to "fizzle";

Thanks.

@axelfontaine axelfontaine added this to the Flyway 4.0.1 milestone Mar 24, 2016
@axelfontaine axelfontaine merged commit 5dc5b89 into flyway:master Apr 18, 2016
@axelfontaine
Copy link
Contributor

@axelfontaine axelfontaine commented Apr 18, 2016

Thanks Nathan! Merged.

@axelfontaine
Copy link
Contributor

@axelfontaine axelfontaine commented Aug 17, 2017

@nathanvick Could you please contact me by email at axel at boxfuse dot com ? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.