Add ignoreMissingMigrations flag

[pik0]

What version of Flyway are you using?

4.0.3

What database are you using (type & version)?

MySQL 5.1.60

What operating system are you using?

Centos 6.5

What did you do?

(Please include the content causing the issue, any relevant configuration settings, and the command you ran)

Upgraded from flyway 3.2.1 to 4.0.3, now receiving this error when trying to test.

In our scenario, we have multiple projects that all feed in to various customer's databases. Depending on what module they license, one or more of these modules needs to be processed through flyway and applied. Not all migrations previously processed will be present in all module migrations. This works fine at 3.2.1, but fails at 4.0+.

What did you expect to see?

Existing migrations should have been accepted and any deprecated or missing migrations ignored. Or, at least provide a flag to disable this error.

What did you see instead?

Validate failed: Detected applied migration not resolved locally

Also, the schema for the schema_version table appears to have changed and the checksum column is calculated differently. This makes it impossible to revert to the older version of flyway without restoring the schema_version table. Fortunately, we're scrupulous when it comes to database backups...

[AndreiArion]

Hi pik0, can you test with validateOnMigrate=false ( e.g -Dflyway.validateOnMigrate=false)?
It should disable the validation step and will ignore the missing files.

[pik0]

But we don't want to disable validation. We can continue to run the current release until this can either be corrected or a better solution advised.

[AndreiArion]

Ok, we are somehow in the same situation, maybe we need a flag that when activated it validates the checksums for the existing files while ignoring missing migrations.

axelfontaine: is this feature on the roadmap? can we help with a pull request ? do you see any problems implementing this (e.g. for someone who never looked at the source code :) )?
Thanks

[cgmcintyr]

+1 bumped into same issue when testing

[pik0]

Thanks Axel for the milestone marker. What is the event horizon for 4.1?

[davidkarlsen]

Would very much like this as well - we want validations of DDLs still hanging around - but can accept the fact that some have been removed. A flag ignoreMissing=true to do this would be preferable.

[asaarnak]

1 solution is to just delete the not used migration from the database.
delete from schema_version where version = '60'
For example if version 60 is missing from migration files but exists in database.

[davidkarlsen]

@asaarnak sure. But it's kinda hackish. We like fully automated processes

[maxsoelalberton]

Hi, I created this shell to remove missing scripts. I hope that helps:

#!/usr/bin/env bash

param=${1} #Flyway parameters like: -configFile=conf/database.properties

for line in $(./flyway info ${param} | grep Missing | cut -d '|' -f2 | awk NF); do 
    missingScripts=$missingScripts$line','
done

if [ -z "$missingScripts" ]; then
    echo "None missing script"
    exit 1
fi
test $? -eq 1 && exit 0

echo 'DELETE FROM "schema_version" WHERE "version" IN ('${missingScripts::-1}');' > sql/beforeValidate.sql

./flyway validate ${param}

rm sql/beforeValidate.sql

[pik0]

Any update on the 4.1 milestone plan? We really need to fixed before we can "migrate" to flyway 4.x... 😃

[pik0]

Yay! Can't wait to try. Have the 4.1 milestone features been locked in? We can test once there is a release.