Add ability to provide variables in migrations

[alniks]

For example ${mysql_db}:

CREATE TABLE users (id UInt64, name String, status UInt8) ENGINE = MySQL('${mysql_address}', '${mysql_db}', 'users', 'root', 'secret');

The variables are then taken from app_config files depending on the environment. So we are able to provide different values for a example for development, test, and jenkins environment.

However, there should be a way to escape those variables for INSERT/UPDATE statement (maybe others as well):

INSERT INTO var_samples (id, sample, text) VALUES (1, '${var}', 'Please, use ${var} in your settings')

The above should go into DB without any replacements, even if there are some vars in the environment.

[alniks]

One more consideration is not to break some old migrations which might use similar syntax in statements and were not going to use replacements.

[ipolevoy]

Proposing to add an additional parameter: mergeValues, where it will point to a file with substitution values. If you want to use different values in different environments, you can create multiple executions of the plugin and provide different mergeValues for different executions.

<plugin>
    <groupId>org.javalite</groupId>
    <artifactId>db-migrator-maven-plugin</artifactId>
    <version>${javalite.version}</version>
    <configuration>
        <environments>testenv</environments>
        <migrationsPath>./src/main/db/migrations</migrationsPath>
        <configFile>${project.basedir}/src/main/resources/database.properties</configFile>
        <encoding>${project.build.sourceEncoding}</encoding>
        <mergeProperties>${project.basedir}/src/main/resources/merge-testenv.properties</mergeProperties> <!-- new parameter proposed -->
    </configuration>
    <executions>
        <execution>
            <id>dev_migrations</id>
            <phase>validate</phase>
            <goals>
                <goal>migrate</goal>
            </goals>
        </execution>
    </executions>
</plugin>

[ipolevoy]

Also, proposing to use a Mustache format:

{{mysql_address}}

inside the property file.

[ipolevoy]

need to add documentation to the site

[ipolevoy]

docs are here: https://javalite.io/database_migrations#variable-substitutions-in-migrations

[ipolevoy]

apparently, there is no way to specify different files for environments, we need to add that

[ipolevoy]

This broke another test

Caused by: java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.putVal (ConcurrentHashMap.java:1011)
at java.util.concurrent.ConcurrentHashMap.put (ConcurrentHashMap.java:1006)
at java.util.Properties.put (Properties.java:1334)
at java.util.Properties.setProperty (Properties.java:229)
at org.javalite.db_migrator.maven.AbstractDbMigrationMojo.prepareCurrentMergeProperties (AbstractDbMigrationMojo.java:131)
at org.javalite.db_migrator.maven.AbstractDbMigrationMojo.execute (AbstractDbMigrationMojo.java:102)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)

to reproduce:

 cd db-migrator-integration-test/src/test/project/cassandra-test-project
 mvn clean install

this is the reason that docker has a port allocated. Cassandra starts but never shuts down

[ipolevoy]

@yanchevsky please, see the comment above