basePackage implementation is required

[svok]

We plan to use cassandra migrations in different packages. Right now this tool does not allow that due to the following:

1. This package has no basePackage option. Here in MigrationRepository

 Enumeration<URL> scriptResources = getClass().getClassLoader().getResources(scriptPath);

while in flyway

    @Override
    public List<ResolvedMigration> resolveMigrations() {
        List<ResolvedMigration> migrations = new ArrayList<>();

        for (Location location : locations) {
            if (!location.isClassPath()) {
                continue;
            }
            resolveMigrationsForSingleLocation(location, migrations);
        }

        Collections.sort(migrations, new ResolvedMigrationComparator());
        return migrations;
    }

    private void resolveMigrationsForSingleLocation(Location location, List<ResolvedMigration> migrations) {
        try {
            Class<?>[] classes = scanner.scanForClasses(location, getImplementedInterface());
            for (Class<?> clazz : classes) {
                M migration = ClassUtils.instantiate(clazz.getName(), scanner.getClassLoader());
                ConfigUtils.injectFlywayConfiguration(migration, configuration);

                ResolvedMigrationImpl migrationInfo = extractMigrationInfo(migration);
                migrationInfo.setPhysicalLocation(ClassUtils.getLocationOnDisk(clazz));
                migrationInfo.setExecutor(createExecutor(migration));

                migrations.add(migrationInfo);
            }
        } catch (Exception e) {
            throw new FlywayException("Unable to resolve " + getMigrationTypeStr() + " Java migrations in location " + location + " : " + e.getMessage(), e);
        }
    }

2. Migrations from different packages conflict one another.

For example, PackageA has migrations 10, 20, while PackageB has 10, 30. So migrations from PackageA won't be applied since their vertions are lower then from PackageB.

[patka]

Hello,

I am not sure I get your request. So, you want to be able to specify multiple locations for migration scripts? Is that the requirement?
And regarding your second point: You say that the migrations of different locations are not merged but just the location with the highest version is considered?

Cheers
Patrick

[svok]

Both points are about one thing. A rather complicated project may include different packages that have their own migrations.
First point states that currently this project allows fetch only migrations from the main project.
The second point states that different migration sets are not correctly handled.

For example.
Following is in one keyspace

PackageA: com.example.table1.jar
Migrations: 10, 20 -> table1

PackageB: com.example.table2.jar
Migrations: 10, 25, 30 -> table2

PackageMain: com.example.main.jar includs both com.example.table1.jar and com.example.table2.jar

In this scenario I get two problems.

1. cassandra-migration doesn't see the migrations in the subpackages. So, I cannot use com.example.table1.jar!/migration/cassandra/.cql and com.example.table2.jar!/migration/cassandra/.cql files. I can use only com.example.main.jar!/migration/cassandra/*.cql files. If there are no such files cassandra-migration throws an exception "No such directory /migration/cassandra".

2. This problem is related to the first one. Both subpackages have their own version sets which do not interfere each other. But cassandra-migration now have no possibility to separate those version sets as different.
   Suppose we issue a new version of our project with new migrations:
   PackageA 25, PackageB 40. In current code of cassandra-migration PackageA 25 will never be applied since cassandra-migration considers only one version set and PackageA 25 is compared with PackageB 30.

[patka]

Ok, to be honest it was never build with such a scenario in mind. The underlying philosophy was that one application maintains its own set of tables (like for example Ruby on Rails does). Now, your scenario suggests that you compile one big application from multiple more or less independent "modules" that happen to share the same database.

I have to check how it could be done but I guess the migration table needs some sort of information for which context or location the change is happening. While that is not so complicated to achieve, problems will start to rise if a refactoring would move the migrations from one place to another.

I will see that I get back to you in terms of feasibility within the next couple of days but I cannot give you any guarantees at the moment on when I have time to implement it as it is very likely that I will move to a new place in the next couple of weeks and I am about to start a new job beginning of October.

[patka]

I had another look on your request and also on the things you referenced from the flyway project. In my opinion there is a conceptional missunderstanding here or I still don't get the problem ;-)

For each keyspace there is exactly one migration table for the whole keyspace and that keyspace has a version. If you have migration scripts in different modules that manage the same keyspace, those scripts should be merged and then all scripts that have a higher version number than the keyspace indicates will be applied. If you however have the same version twice (as stated in your example above) this leads to a conflict because every version has to be unique per keyspace.

What your example indicates is that every table should be versioned which does not make sense in my opinion. If you really require this you should think about putting those tables in different keyspaces and then use different Database instances where you apply different MigrationTasks.

As far as I can see the behavior is the same in flyway as you can see in the CompositeMigrationResolver. It throws an exception as well if two migrations with the same version are given.

I will however have a look on the issue that you cannot import scripts from different jars. Might be that this merging of scripts from different locations does not work as expected.