Skip to content

Add ability to provide variables in migrations #1104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
alniks opened this issue May 13, 2021 · 8 comments
Closed

Add ability to provide variables in migrations #1104

alniks opened this issue May 13, 2021 · 8 comments
Assignees

Comments

@alniks
Copy link

alniks commented May 13, 2021

For example ${mysql_db}:

CREATE TABLE users (id UInt64, name String, status UInt8) ENGINE = MySQL('${mysql_address}', '${mysql_db}', 'users', 'root', 'secret');

The variables are then taken from app_config files depending on the environment. So we are able to provide different values for a example for development, test, and jenkins environment.

However, there should be a way to escape those variables for INSERT/UPDATE statement (maybe others as well):

INSERT INTO var_samples (id, sample, text) VALUES (1, '${var}', 'Please, use ${var} in your settings')

The above should go into DB without any replacements, even if there are some vars in the environment.

@alniks
Copy link
Author

alniks commented May 13, 2021

One more consideration is not to break some old migrations which might use similar syntax in statements and were not going to use replacements.

@ipolevoy ipolevoy reopened this May 13, 2021
@ipolevoy
Copy link
Member

ipolevoy commented May 14, 2021

Proposing to add an additional parameter: mergeValues, where it will point to a file with substitution values. If you want to use different values in different environments, you can create multiple executions of the plugin and provide different mergeValues for different executions.

<plugin>
    <groupId>org.javalite</groupId>
    <artifactId>db-migrator-maven-plugin</artifactId>
    <version>${javalite.version}</version>
    <configuration>
        <environments>testenv</environments>
        <migrationsPath>./src/main/db/migrations</migrationsPath>
        <configFile>${project.basedir}/src/main/resources/database.properties</configFile>
        <encoding>${project.build.sourceEncoding}</encoding>
        <mergeProperties>${project.basedir}/src/main/resources/merge-testenv.properties</mergeProperties> <!-- new parameter proposed -->
    </configuration>
    <executions>
        <execution>
            <id>dev_migrations</id>
            <phase>validate</phase>
            <goals>
                <goal>migrate</goal>
            </goals>
        </execution>
    </executions>
</plugin>

@ipolevoy
Copy link
Member

Also, proposing to use a Mustache format:

{{mysql_address}}

inside the property file.

@ipolevoy
Copy link
Member

need to add documentation to the site

@ipolevoy
Copy link
Member

@ipolevoy
Copy link
Member

apparently, there is no way to specify different files for environments, we need to add that

@ipolevoy ipolevoy reopened this Aug 23, 2021
ipolevoy added a commit that referenced this issue Aug 25, 2021
#1104 - Add ability to provide variables in migrations
@ipolevoy
Copy link
Member

ipolevoy commented Aug 26, 2021

This broke another test

Caused by: java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.putVal (ConcurrentHashMap.java:1011)
at java.util.concurrent.ConcurrentHashMap.put (ConcurrentHashMap.java:1006)
at java.util.Properties.put (Properties.java:1334)
at java.util.Properties.setProperty (Properties.java:229)
at org.javalite.db_migrator.maven.AbstractDbMigrationMojo.prepareCurrentMergeProperties (AbstractDbMigrationMojo.java:131)
at org.javalite.db_migrator.maven.AbstractDbMigrationMojo.execute (AbstractDbMigrationMojo.java:102)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)

to reproduce:

 cd db-migrator-integration-test/src/test/project/cassandra-test-project
 mvn clean install

this is the reason that docker has a port allocated. Cassandra starts but never shuts down

@ipolevoy
Copy link
Member

@yanchevsky please, see the comment above

ipolevoy added a commit that referenced this issue Aug 26, 2021
#1104 - Add ability to provide variables in migrations - fix null poi…
ipolevoy added a commit that referenced this issue Aug 27, 2021
#1104 - Add ability to provide variables in migrations - fix reset mojo
ipolevoy added a commit that referenced this issue Nov 12, 2024
…gration-mojo

#1104 - Add ability to provide variables in migrations - fix merge properties
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants