Bump version to flyway-9.18.0

Bump maven version to address CVE-2021-29425 Implement adjusting block depth in postgres parser to support begin atomic function definitions Closes #3665: Only warn if reports are unable to write to the default location
flyway · May 15, 2023 · d8fbcb6 · d8fbcb6
1 parent 60a5ad1
commit d8fbcb6
Show file tree

Hide file tree

Showing 39 changed files with 224 additions and 87 deletions.
diff --git a/.mvn/wrapper/maven-wrapper.properties b/.mvn/wrapper/maven-wrapper.properties
@@ -14,5 +14,5 @@
 # limitations under the License.
 #
 
-distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.8.7/apache-maven-3.8.7-bin.zip
+distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.2/apache-maven-3.9.2-bin.zip
 wrapperUrl=https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.1.1/maven-wrapper-3.1.1.jar
diff --git a/.snyk b/.snyk
diff --git a/...on/Flyway CLI and API/Getting Started/First Steps/First Steps - Command-line.md b/...on/Flyway CLI and API/Getting Started/First Steps/First Steps - Command-line.md
@@ -5,32 +5,39 @@ redirect_from: /Getting Started/First Steps/First Steps - Command-line.md/
 
 # First Steps: Command-line
 
-This brief tutorial will teach **how to get up and running with the Flyway Command-line tool**. It will take you through the
+This brief tutorial will teach **how to get up and running with the Flyway Command-line through Docker**. It will take you through the
 steps on how to configure it and how to write and execute your first few database migrations.
 
 This tutorial should take you about **5 minutes** to complete.
 
 ## Prerequisites
 
-Start by [downloading the Flyway Command-line Tool](https://flywaydb.org/download) for your platform and extract it.
+Start by downloading Flyway Teams and Enterprise Docker image to your Docker environment. 
+
+```
+docker pull redgate/flyway
+```
 
 ## Configuring Flyway
 
-Let's now jump into our new directory created from downloading Flyway:
+The easiest way to get started is simply to test the default image by running
 
-<pre class="console"><span>&gt;</span> cd flyway-{{ site.flywayVersion }}</pre>
+```
+docker run --rm redgate/flyway
+```
+
+This will give you Flyway Command-line's usage instructions.
 
-Configure Flyway by editing `/conf/flyway.conf`, like this:
+Next, configure Flyway by creating a file named `flyway.conf`, like this:
 
 ```properties
-flyway.url=jdbc:h2:file:./foobardb
-flyway.user=SA
-flyway.password=
+flyway.url=jdbc:sqlite:/flyway/db/test_db.sqlite3
+flyway.user=sa
 ```
 
 ## Creating the first migration
 
-Now create your first migration in the `/sql` directory called `V1__Create_person_table.sql`:
+Now create your first migration sql called `V1__Create_person_table.sql`:
 
 ```sql
 create table PERSON (
@@ -41,44 +48,88 @@ create table PERSON (
 
 ## Migrating the database
 
-It's now time to execute Flyway to migrate your database:
+It's now time to execute Flyway to migrate your database.
+
+- Replace {absolute path to folder to store SQLite db file} with the path to where you would like to store your SQLite db file.
 
-<pre class="console"><span>flyway-{{ site.flywayVersion }}&gt;</span> flyway <strong>migrate</strong></pre>
+- Replace {absolute path to folder containing sql migrations} with the path to your sql file directory.
+
+- Replace {absolute path to folder containing conf file} with the path to your conf file directory.
+
+```
+docker run --rm -v "{absolute path to folder to store SQLite db file}:/flyway/db" -v "{absolute path to folder containing sql migrations}:/flyway/sql" -v "{absolute path to folder containing conf file}:/flyway/conf" redgate/flyway migrate
+```
 
 If all went well, you should see the following output:
 
-<pre class="console">Database: jdbc:h2:file:./foobardb (H2 1.4)
-Successfully validated 1 migration (execution time 00:00.008s)
-Creating Schema History table: "PUBLIC"."flyway_schema_history"
-Current version of schema "PUBLIC": << Empty Schema >>
-Migrating schema "PUBLIC" to version 1 - Create person table
-Successfully applied 1 migration to schema "PUBLIC" (execution time 00:00.033s)</pre>
+<pre class="console">Database: jdbc:sqlite:/flyway/db/test_db.sqlite3 (SQLite 3.34)
+Schema history table "main"."flyway_schema_history" does not exist yet
+Successfully validated 1 migration (execution time 00:00.009s)
+Creating Schema History table "main"."flyway_schema_history" ...
+Current version of schema "main": << Empty Schema >>
+Migrating schema "main" to version "1 - Create person table"
+Successfully applied 1 migration to schema "main", now at version v1 (execution time 00:00.034s)</pre>
 
 ## Adding a second migration
 
-If you now add a second migration to the `/sql` directory called `V2__Add_people.sql`:
+If you now add a second migration to your sql directory called `V2__Add_people.sql`:
 
-<pre class="prettyprint">insert into PERSON (ID, NAME) values (1, 'Axel');
+```sql
+insert into PERSON (ID, NAME) values (1, 'Axel');
 insert into PERSON (ID, NAME) values (2, 'Mr. Foo');
-insert into PERSON (ID, NAME) values (3, 'Ms. Bar');</pre>
+insert into PERSON (ID, NAME) values (3, 'Ms. Bar');
+```
 
 and execute it by issuing:
 
-<pre class="console"><span>flyway-{{ site.flywayVersion }}&gt;</span> flyway <strong>migrate</strong></pre>
+```
+docker run --rm -v "{absolute path to folder to store SQLite db file}:/flyway/db" -v "{absolute path to folder containing sql migrations}:/flyway/sql" -v "{absolute path to folder containing conf file}:/flyway/conf" redgate/flyway migrate
+```
 
 You now get:
 
-<pre class="console">Database: jdbc:h2:file:./foobardb (H2 1.4)
-Successfully validated 2 migrations (execution time 00:00.018s)
-Current version of schema "PUBLIC": 1
-Migrating schema "PUBLIC" to version 2 - Add people
-Successfully applied 1 migration to schema "PUBLIC" (execution time 00:00.016s)</pre>
+<pre class="console">Database: jdbc:sqlite:/flyway/db/test_db.sqlite3 (SQLite 3.34)
+Successfully validated 2 migrations (execution time 00:00.015s)
+Current version of schema "main": 1
+Migrating schema "main" to version "2 - Add people"
+Successfully applied 1 migration to schema "main", now at version v2 (execution time 00:00.029s)</pre>
+
+## Verification
+
+You can use `info` command to check what Flyway did.  
+
+```
+docker run --rm -v "{absolute path to folder to store SQLite db file}:/flyway/db" -v "{absolute path to folder containing sql migrations}:/flyway/sql" -v "{absolute path to folder containing conf file}:/flyway/conf" redgate/flyway info
+```
+
+This will give output as below: 
+
+<pre class="console">
+
+Database: jdbc:sqlite:/flyway/db/test_db.sqlite3 (SQLite 3.34)
+Schema version: 2
+
++-----------+---------+---------------------+------+---------------------+---------+----------+
+| Category  | Version | Description         | Type | Installed On        | State   | Undoable |
++-----------+---------+---------------------+------+---------------------+---------+----------+
+| Versioned | 1       | Create person table | SQL  | 2023-04-26 13:01:02 | Success | No       |
+| Versioned | 2       | Add people          | SQL  | 2023-04-26 13:05:04 | Success | No       |
++-----------+---------+---------------------+------+---------------------+---------+----------+
+
+</pre>
+
+If you run the `migrate` command again, Flyway will give below output: 
+
+<pre class="console">Database: jdbc:sqlite:/flyway/db/test_db.sqlite3 (SQLite 3.34)
+Successfully validated 2 migrations (execution time 00:00.016s)
+Current version of schema "main": 2
+Schema "main" is up to date. No migration necessary.</pre>
 
 ## Summary
 
 In this brief tutorial we saw how to:
-- install the Flyway Command-line tool
-- configure it so it can talk to our database
+- set up the Flyway Command-line environment through Docker
+- configure it so it can talk to your local database
 - write our first couple of migrations
 
 These migrations were then successfully found and executed.
diff --git a/documentation/Flyway CLI and API/Reports/Reports - Change.md b/documentation/Flyway CLI and API/Reports/Reports - Change.md
@@ -10,6 +10,8 @@ You might want this to:
 * Pre-deployment - check the effect of your pending changes (DBA review)
 * Post-deployment -capture a history of changes for retrospective auditing or reporting
 
+![Change report](assets/change_report_screenshot.png)
+
 ## Learn more
 * [Check concept](Concepts/Check Concept) - how does it work ?
 * [Check command](Commands/Check) - What is the command structure ?

diff --git a/documentation/Flyway CLI and API/Reports/Reports - Code Analysis.md b/documentation/Flyway CLI and API/Reports/Reports - Code Analysis.md
@@ -7,6 +7,8 @@ The code analysis report shows you the results of the static analysis tool(s) th
 * Teams customers can configure how their pipeline can be halted based on policy around rule violations
 * Enterprise customers also get an easy mechanism to create and use custom rules based on regex pattern matching
 
+![Code report](assets/code_report_screenshot.png)
+
 ## Learn more
 
 * [Check concept](Concepts/Check Concept) - how does it work ?

diff --git a/documentation/Flyway CLI and API/Reports/Reports - Drift.md b/documentation/Flyway CLI and API/Reports/Reports - Drift.md
@@ -9,6 +9,8 @@ You might want this to:
 * Check if changes have been made to your database outside of the Flyway migration control mechanism
 * The target database is in the state you expect it t be in prior to applying new migrations
 
+![Drift report](assets/drift_report_screenshot.png)
+
 ## Learn more
 
 * [Check concept](Concepts/Check Concept) - how does it work ?

diff --git a/documentation/Flyway CLI and API/Reports/Reports - Dry Run.MD b/documentation/Flyway CLI and API/Reports/Reports - Dry Run.MD
@@ -11,6 +11,8 @@ You might want this to:
 * submit the SQL statements for review to a DBA before applying them
 * use Flyway to determine what needs updating, yet use a different tool to apply the actual database changes 
 
+![Dryrun report](assets/dryrun_report_screenshot.png)
+
 ## Learn More
 
 * [Dry Run Concept](Concepts/Dry Runs) - how does it work ?

diff --git a/documentation/Flyway CLI and API/Reports/Reports - Info.md b/documentation/Flyway CLI and API/Reports/Reports - Info.md
@@ -10,6 +10,8 @@ You might want this to:
 * Check the current migration state of your target DB
 * Understand what is yet to be deployed in a more digestible format than the raw filenames
 
+![Info report](assets/info_report_screenshot.png)
+
 ## Learn More
 
 * [Info](Commands/info)

diff --git a/documentation/Flyway CLI and API/Reports/Reports - Migration.md b/documentation/Flyway CLI and API/Reports/Reports - Migration.md
@@ -9,6 +9,8 @@ subtitle: Migration Report
  * Have a record of what happened when for audit or reporting purposes
  * Understand and be able to communicate failures in a more richly descriptive manner 
 
+![Migrate report](assets/migrate_report_screenshot.png)
+
 ## Learn More
 
  * [Migrate Command](Commands/Migrate) - how does it work ?

diff --git a/documentation/Flyway CLI and API/Supported Databases/Snowflake.md b/documentation/Flyway CLI and API/Supported Databases/Snowflake.md
@@ -101,3 +101,4 @@ location of the private key:
 ## Limitations
 
 - Parallel migrations as described [here](Learn More/Frequently Asked Questions#parallel) are unavailable in Snowflake. You can track the status of this feature in our GitHub issues [here](https://github.com/flyway/flyway/issues/3305).
+- Users using Java 16 or above, which includes the JRE shipped within Java Command Line, will need to add the following JVM argument to JAVA_ARGS `--add-opens java.base/java.lang=ALL-UNNAMED`. This can be done via the [command line or environment variables](Usage/Command-Line) This is due to a change in the Java 16 runtime which causes an error within the Snowflake JDBC driver.
diff --git a/documentation/Release notes and older versions/Release Notes for Flyway Engine.html b/documentation/Release notes and older versions/Release Notes for Flyway Engine.html
@@ -25,6 +25,41 @@
 </div>
 <div class="col-md-9">
 
+<div class="release">
+        <h2 id="9.18.0">Flyway 9.18.0 (2023-05-15)</h2>
+
+        <h3>Bug fixes</h3>
+        <ul>
+            <li>Avoid hanging indefinitely during 'check' comparisons when running on Windows.</li>
+            <li>Running `check -changes` or `check -drift` now handles schema creation rows in the schema history table.</li>
+        </ul>
+
+        <h3>Changes</h3>
+        <ul>
+            <li>All undo migrations will no longer have their own rows in the ASCII-table output. This seemed redundant given that there's a separate `Undoable` column to provide this information. Also, there was some inconsistency whereby undo migrations in the AVAILABLE state were already filtered out but others were not.</li>
+            <li>Baseline migrations now have the 'baseline' category in info output</li>
+            <li>
+                <a href="https://github.com/flyway/flyway/issues/3665">Issue 3665</a>
+                Only warn if reports are unable to write to the default location rather than print an error
+            </li>
+        </ul>
+
+        <h3>New features</h3>
+        <ul>
+            <li>Flyway now handles defining PostgreSQL functions using `BEGIN ATOMIC`.</li>
+        </ul>
+
+        <h3>Java compatibility</h3>
+        <ul>
+            <li>Bump maven version to address CVE-2021-29425</li>
+            <li>Update documentation to provide solution to Snowflake JDBC JRE16+ issues</li>
+        </ul>
+
+        <p>
+            Thanks to sseide for reporting these issues.
+        </p>
+    </div>
+
 <div class="release">
         <h2 id="9.17.0">Flyway 9.17.0 (2023-04-27)</h2>
 

diff --git a/documentation/_config.yml b/documentation/_config.yml
@@ -14,7 +14,7 @@
 # limitations under the License.
 #
 
-flywayVersion: 9.17.0
+flywayVersion: 9.18.0
 enterpriseUrl: https://download.red-gate.com/maven/release/org/flywaydb/enterprise
 kramdown:
   smart_quotes: ["apos", "apos", "quot", "quot"]

diff --git a/flyway-commandline/pom.xml b/flyway-commandline/pom.xml
@@ -21,7 +21,7 @@
     <parent>
         <groupId>org.flywaydb</groupId>
         <artifactId>flyway-parent</artifactId>
-        <version>9.17.0</version>
+        <version>9.18.0</version>
     </parent>
     <artifactId>flyway-commandline</artifactId>
     <packaging>jar</packaging>
@@ -30,7 +30,7 @@
 
 
 
-        <flyway-gcp-spanner.version>9.17.0-beta</flyway-gcp-spanner.version>
+        <flyway-gcp-spanner.version>9.18.0-beta</flyway-gcp-spanner.version>
         <jre.version>17.0.6</jre.version>
     </properties>
     <dependencies>

diff --git a/flyway-commandline/src/main/java/org/flywaydb/commandline/CommandLineArguments.java b/flyway-commandline/src/main/java/org/flywaydb/commandline/CommandLineArguments.java
@@ -147,6 +147,7 @@ private static List<String> getConfigFilesFromArgs(String[] args) {
     private static Map<String, String> getConfigurationFromArgs(String[] args) {
         return Arrays.stream(args)
                      .filter(CommandLineArguments::isConfigurationArg)
+                     .filter(arg -> !arg.startsWith("-" + CONFIG_FILES + "="))
                      .filter(f -> !isConfigurationOptionCommandlineOnly(getConfigurationOptionNameFromArg(f)))
                      .collect(Collectors.toMap(p -> (Arrays.stream((EnvironmentModel.class).getDeclaredFields()).anyMatch(x -> x.getName().equals(getConfigurationOptionNameFromArg(p)))
                                                        ? "environments." + ClassicConfiguration.TEMP_ENVIRONMENT_NAME + "."

diff --git a/flyway-commandline/src/main/java/org/flywaydb/commandline/Main.java b/flyway-commandline/src/main/java/org/flywaydb/commandline/Main.java
@@ -22,6 +22,7 @@
 import org.flywaydb.commandline.logging.console.ConsoleLog.Level;
 import org.flywaydb.commandline.logging.console.ConsoleLogCreator;
 import org.flywaydb.commandline.logging.file.FileLogCreator;
+import org.flywaydb.commandline.utils.OperationsReportUtils;
 import org.flywaydb.core.Flyway;
 import org.flywaydb.core.FlywayTelemetryManager;
 import org.flywaydb.core.api.*;
@@ -65,6 +66,10 @@
 import java.util.stream.Collectors;
 import java.util.stream.Stream;
 
+import static org.flywaydb.commandline.utils.OperationsReportUtils.DEFAULT_REPORT_FILENAME;
+import static org.flywaydb.commandline.utils.OperationsReportUtils.HTML_REPORT_EXTENSION;
+import static org.flywaydb.commandline.utils.OperationsReportUtils.HTM_REPORT_EXTENSION;
+import static org.flywaydb.commandline.utils.OperationsReportUtils.JSON_REPORT_EXTENSION;
 import static org.flywaydb.commandline.utils.OperationsReportUtils.createHtmlReport;
 import static org.flywaydb.commandline.utils.OperationsReportUtils.createJsonReport;
 import static org.flywaydb.commandline.utils.OperationsReportUtils.filterHtmlResults;
@@ -170,13 +175,24 @@ public static void main(String[] args) throws Exception {
                     String reportFilename = configuration.getReportFilename();
                     String baseReportFilename = getBaseFilename(reportFilename);
 
-                    String tmpJsonReportFilename = baseReportFilename + ".json";
-                    String tmpHtmlReportFilename = baseReportFilename + (reportFilename.endsWith(".htm") ? ".htm" : ".html");
+                    String tmpJsonReportFilename = baseReportFilename + JSON_REPORT_EXTENSION;
+                    String tmpHtmlReportFilename = baseReportFilename + (reportFilename.endsWith(HTM_REPORT_EXTENSION) ? HTM_REPORT_EXTENSION : HTML_REPORT_EXTENSION);
 
                     htmlCompositeResult = JsonUtils.appendIfExists(tmpJsonReportFilename, htmlCompositeResult, new CompositeResultDeserializer(configuration.getPluginRegister()));
 
-                    jsonReportFilename = createJsonReport(htmlCompositeResult, tmpJsonReportFilename);
-                    htmlReportFilename = createHtmlReport(configuration, htmlCompositeResult, tmpHtmlReportFilename);
+                    try {
+                        jsonReportFilename = createJsonReport(htmlCompositeResult, tmpJsonReportFilename);
+                        htmlReportFilename = createHtmlReport(configuration, htmlCompositeResult, tmpHtmlReportFilename);
+                    } catch (FlywayException e) {
+                        if(DEFAULT_REPORT_FILENAME.equals(configuration.getReportFilename())) {
+                            LOG.warn("Unable to create default report files.");
+                            if(LOG.isDebugEnabled()) {
+                                e.printStackTrace(System.out);
+                            }
+                        } else {
+                            LOG.error("Unable to create report files", e);
+                        }
+                    }
 
                     if (htmlReportFilename != null) {
                         LOG.info("A Flyway report has been generated here: " + htmlReportFilename);