New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added support for non-transactional SQL migrations #1027

Closed
wants to merge 2 commits into
base: master
from

Conversation

Projects
None yet
@Omie

Omie commented May 29, 2015

fixes #1026

  • added new Resolver and Executor classes
  • NTSqlMigrationResolver works on the files prefixed with "NTV", currently hardcoded. (Looking forward for suggestions over this)
  • Added the new resolver in migrationResolvers while creating CompositeMigrationResolver instance
  • added small test for new resolver

Omie added some commits May 28, 2015

added support for non-transactional SQL migrations
- added new Resolver and Executor classes
- NTSqlMigrationResolver works on the files prefixed with "NTV",
  currently hardcoded

- added small test for new resolver
@ccouturi

This comment has been minimized.

ccouturi commented Jun 5, 2015

+1 for this feature.

@eemmiirr

This comment has been minimized.

eemmiirr commented Jul 28, 2015

+1

4 similar comments
@Nthalk

This comment has been minimized.

Nthalk commented Jul 31, 2015

+1

@0x6e6562

This comment has been minimized.

0x6e6562 commented Aug 20, 2015

+1

@baekdahl

This comment has been minimized.

baekdahl commented Aug 27, 2015

+1

@CRegenschein

This comment has been minimized.

CRegenschein commented Aug 28, 2015

+1

@CRegenschein

This comment has been minimized.

CRegenschein commented Aug 28, 2015

when will this feature be merged into flyway?

@0x6e6562

This comment has been minimized.

0x6e6562 commented Sep 10, 2015

For anybody that is looking for a quick and dirty way to mix in some non-transactional migrations into an existing SQL-based patch series, I knocked out the following shim:

public abstract class AbstractResolvedMigration implements ResolvedMigration {

  private final String version;
  private final String description;

  public AbstractResolvedMigration() {

    String className = this.getClass().getSimpleName();
    String template = "V(\\d+)__(.*)";
    Pattern pattern = Pattern.compile(template);
    Matcher matcher = pattern.matcher(className);
    if (matcher.matches()) {
      this.version = matcher.group(1);
      this.description = matcher.group(2).replaceAll("_", " ");
    } else {
      throw new RuntimeException("Bogus migration name: " + className);
    }

  }

  @Override
  public MigrationVersion getVersion() {
    return MigrationVersion.fromVersion(version);
  }

  @Override
  public String getDescription() {
    return description;
  }

  @Override
  public Integer getChecksum() {
    final CRC32 crc32 = new CRC32();
    crc32.update(getScript().getBytes());
    return (int) crc32.getValue();
  }

  @Override
  public MigrationType getType() {
    return MigrationType.CUSTOM;
  }

  @Override
  public String getPhysicalLocation() {
    return "virtual://loaction";
  }

  @Override
  public MigrationExecutor getExecutor() {
    return new MigrationExecutor() {

      @Override
      public void execute(Connection connection) {
        PreparedStatement stmt;
        try {
          stmt = connection.prepareStatement(getScript());
          stmt.execute();
        } catch (SQLException e) {
          throw new RuntimeException(e);
        }
      }

      @Override
      public boolean executeInTransaction() {
        return false;
      }

    };
  }
}

And then if I need to execute something outside of a TX, I subclass the base class to supply the patch specific SQL. For example, say I have a V50 SQL migration that requires a new type in Postgres, I create a V49 custom migration that patches the DDL in a non-TX migration in order that the subsequent V50 migration that depends on the new type can execute:

public class V49__some_meaningful_description extends AbstractResolvedMigration {

  @Override
  public String getScript() {
    return "ALTER TYPE foo ADD VALUE 'BAR' AFTER 'BAZ';";
  }

}

And then to mix the V49 migration into the execution series, you need to add a bit of resolver glue and then tell Flyway about it at runtime:

public class NonTransactionalMigrationResolver implements MigrationResolver {

  @Override
  public List<ResolvedMigration> resolveMigrations() {

    List<ResolvedMigration> resolvedMigrations = new ArrayList<>();

    resolvedMigrations.add(new V49__some_meaningful_description());
    // add any other non-TX migrations to this list

    return resolvedMigrations;
  }
}

Hope this can help somebody until such a time that there is a less hacky way to do this kind of thing.

@hendley

This comment has been minimized.

hendley commented Oct 14, 2015

+1

2 similar comments
@eemmiirr

This comment has been minimized.

eemmiirr commented Oct 26, 2015

+1

@mjallday

This comment has been minimized.

mjallday commented Nov 11, 2015

👍

@tomdcc

This comment has been minimized.

tomdcc commented Nov 11, 2015

+1

@axelfontaine Is there anything we can do to help get this PR in shape to be merged?

@akopts

This comment has been minimized.

akopts commented Dec 1, 2015

+1

5 similar comments
@mr-zhukov

This comment has been minimized.

mr-zhukov commented Dec 1, 2015

+1

@skrasovsky

This comment has been minimized.

skrasovsky commented Dec 1, 2015

+1

@skozlov

This comment has been minimized.

skozlov commented Dec 1, 2015

+1

@irwinm

This comment has been minimized.

irwinm commented Dec 1, 2015

+1

@hhware

This comment has been minimized.

hhware commented Dec 22, 2015

+1

@brckner

This comment has been minimized.

brckner commented Feb 22, 2016

When would this feature be merged into master? We have currently issues too, where we need migrations to run in an non transactional context.

@falschparker82

This comment has been minimized.

falschparker82 commented Mar 18, 2016

+1

1 similar comment
@karlvlam

This comment has been minimized.

karlvlam commented May 17, 2016

+1

@karlvlam

This comment has been minimized.

karlvlam commented May 18, 2016

any updates for this issue?

@anthonylau

This comment has been minimized.

anthonylau commented May 18, 2016

+1

2 similar comments
@VLMH

This comment has been minimized.

VLMH commented May 18, 2016

+1

@jjbubudi

This comment has been minimized.

jjbubudi commented May 18, 2016

+1

@lukasniemeier-zalando

This comment has been minimized.

lukasniemeier-zalando commented May 20, 2016

@axelfontaine dies this change has a Chance of being merged?

@morero

This comment has been minimized.

morero commented Jun 8, 2016

+1

@j-lindblom

This comment has been minimized.

j-lindblom commented Jun 8, 2016

@axelfontaine Would you consider merging this PR?

@Trekoid

This comment has been minimized.

Trekoid commented Jul 21, 2016

I wouldn't want non-transactional migrations to have a different runtime order than the regular V1_Description.sql files. I would prefer something like V53_NT__Description.sql or V53__NT_Description.sql so they would run in line with other transactional migrations.

@Omie

This comment has been minimized.

Omie commented Jul 22, 2016

@Trekoid this patch does maintain the order. V1_foo.sql, NTV2_bar.sql and V3_baz.sql files will get executed in foo, bar, baz order.

On the other note, this PR is vastly out of date, there are hundreds of commits in between and this will probably require work from scratch.

@davidvuong

This comment has been minimized.

davidvuong commented Nov 17, 2016

Hey @Omie,

Just to add to @Trekoid's comment:

My IDE orders files alphanumerically and having it named NTVxx when all other flyway migrations areVxx makes it a little hard to navigate. I understand that flyway will run these migrations in order but editors won't display them in order (which can be a bit difficult to grok if you have lots of migrations).

Just my 2 cents. Awesome PR BTW. I'd love to see this finally get merged soon.

@Trekoid

This comment has been minimized.

Trekoid commented Nov 28, 2016

@davidvuong your should be happy to hear the company I work for co-sponsored the development of non-transactional migrations with BoxFuse, so expect it soon in release 4.1. The solution @axelfontaine and the team at BoxFuse created requires no special file naming. It will auto detect the migration and run it appropriately.

@davidvuong

This comment has been minimized.

davidvuong commented Nov 29, 2016

@Trekoid That's awesome. Looking forward to it!

@brckner

This comment has been minimized.

brckner commented Nov 30, 2016

Hey, I've also worked on a non transactional resolver for flyway. It scans for sql files with suffix .ntx.sql and executes them in a nontransactional context. It works only with flyway maven plugin... so you can add it as a dependency in maven pom and add the custom resolver in the flyway-maven-plugin. if anyone is interested, i'll put it on gitlab and maven central.

@arturaz

This comment has been minimized.

arturaz commented Jan 30, 2017

Anything?

@axelfontaine

This comment has been minimized.

Contributor

axelfontaine commented Feb 7, 2017

This has been implemented differently using auto-detection (PostgreSQL only for now). Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment