Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration breaks due to java.io.IOException: Mark invalid #2875

Closed
ankos opened this issue Jul 9, 2020 · 5 comments
Closed

Migration breaks due to java.io.IOException: Mark invalid #2875

ankos opened this issue Jul 9, 2020 · 5 comments
Assignees
Milestone

Comments

@ankos
Copy link

ankos commented Jul 9, 2020

Which version and edition of Flyway are you using?

6.5.0

If this is not the latest version, can you reproduce the issue with the latest one as well?

(Many bugs are fixed in newer releases and upgrading will often resolve the issue)

Which client are you using? (Command-line, Java API, Maven plugin, Gradle plugin)

Java API

Which database are you using (type & version)?

Oracle 12c

Which operating system are you using?

Windows (locally) / Linux (CI Build with Jenkins)

What did you do?

Before running integration tests with jUnit we create the database schema from scratch. There are 912 script files at the moment.
I use the Java API to set placeholders and start the migration.

What did you expect to see?

All migrations are applied and the tests are started.

This works absolutely fine for the previous versions I used:

  • 6.4.1
  • 6.4.4
What did you see instead?

When using version 6.5.0 the migration always breaks at script number 46 with the following error message:

org.flywaydb.core.api.FlywayException: Unable to parse statement in db/migration/V1.000046__<some-name>.sql at line 2 col 1. See https://flywaydb.org/documentation/knownparserlimitations for more information: Mark invalid at org.flywaydb.core.internal.parser.Parser.getNextStatement(Parser.java:294) at org.flywaydb.core.internal.parser.Parser.access$000(Parser.java:41) at org.flywaydb.core.internal.parser.Parser$ParserSqlStatementIterator.next(Parser.java:733) at org.flywaydb.core.internal.parser.Parser$ParserSqlStatementIterator.next(Parser.java:698) at org.flywaydb.core.internal.sqlscript.ParserSqlScript.parse(ParserSqlScript.java:82) at org.flywaydb.core.internal.sqlscript.ParserSqlScript.validate(ParserSqlScript.java:127) at org.flywaydb.core.internal.sqlscript.ParserSqlScript.executeInTransaction(ParserSqlScript.java:196) at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.canExecuteInTransaction(SqlMigrationExecutor.java:93) at org.flywaydb.core.internal.command.DbMigrate.isExecuteGroupInTransaction(DbMigrate.java:312) at org.flywaydb.core.internal.command.DbMigrate.applyMigrations(DbMigrate.java:275) at org.flywaydb.core.internal.command.DbMigrate.migrateGroup(DbMigrate.java:244) at org.flywaydb.core.internal.command.DbMigrate.access$100(DbMigrate.java:54) at org.flywaydb.core.internal.command.DbMigrate$2.call(DbMigrate.java:162) at org.flywaydb.core.internal.command.DbMigrate$2.call(DbMigrate.java:159) at org.flywaydb.core.internal.jdbc.TableLockingExecutionTemplate$1.call(TableLockingExecutionTemplate.java:38) at org.flywaydb.core.internal.jdbc.TransactionalExecutionTemplate.execute(TransactionalExecutionTemplate.java:66) at org.flywaydb.core.internal.jdbc.TableLockingExecutionTemplate.execute(TableLockingExecutionTemplate.java:33) at org.flywaydb.core.internal.database.base.Connection.lock(Connection.java:129) at org.flywaydb.core.internal.schemahistory.JdbcTableSchemaHistory.lock(JdbcTableSchemaHistory.java:140) at org.flywaydb.core.internal.command.DbMigrate.migrateAll(DbMigrate.java:159) at org.flywaydb.core.internal.command.DbMigrate.migrate(DbMigrate.java:137) at org.flywaydb.core.Flyway$1.execute(Flyway.java:206) at org.flywaydb.core.Flyway$1.execute(Flyway.java:158) at org.flywaydb.core.Flyway.execute(Flyway.java:527) at org.flywaydb.core.Flyway.migrate(Flyway.java:158) at finance.egp.reporting.merger.ReportingMergerTest.lambda$static$0(ReportingMergerTest.java:166) at finance.egp.framework.fusion.test.FusionApplicationTestStage.populateSchema(FusionApplicationTestStage.java:101) at finance.egp.framework.fusion.test.FusionApplicationTestStage$ByteBuddy$hNCI3voh.populateSchema$accessor$8sqzLu5M(Unknown Source) at finance.egp.framework.fusion.test.FusionApplicationTestStage$ByteBuddy$hNCI3voh$auxiliary$5wCia11p.call(Unknown Source) at com.tngtech.jgiven.impl.intercept.ByteBuddyMethodInterceptor$1.proceed(ByteBuddyMethodInterceptor.java:43) at com.tngtech.jgiven.impl.intercept.StepInterceptorImpl.doIntercept(StepInterceptorImpl.java:117) at com.tngtech.jgiven.impl.intercept.StepInterceptorImpl.intercept(StepInterceptorImpl.java:84) at com.tngtech.jgiven.impl.intercept.ByteBuddyMethodInterceptor.interceptSuper(ByteBuddyMethodInterceptor.java:47) at finance.egp.framework.fusion.test.FusionApplicationTestStage$ByteBuddy$hNCI3voh.populateSchema(Unknown Source) at finance.egp.framework.fusion.test.FusionApplicationTestStage.unloading_tenant(FusionApplicationTestStage.java:172) at finance.egp.framework.fusion.test.FusionApplicationTestStage$ByteBuddy$hNCI3voh.unloading_tenant$accessor$8sqzLu5M(Unknown Source) at finance.egp.framework.fusion.test.FusionApplicationTestStage$ByteBuddy$hNCI3voh$auxiliary$9PDNIpXs.call(Unknown Source) at com.tngtech.jgiven.impl.intercept.ByteBuddyMethodInterceptor$1.proceed(ByteBuddyMethodInterceptor.java:43) at com.tngtech.jgiven.impl.intercept.StepInterceptorImpl.doIntercept(StepInterceptorImpl.java:117) at com.tngtech.jgiven.impl.intercept.StepInterceptorImpl.intercept(StepInterceptorImpl.java:84) at com.tngtech.jgiven.impl.intercept.ByteBuddyMethodInterceptor.interceptSuper(ByteBuddyMethodInterceptor.java:47) at finance.egp.framework.fusion.test.FusionApplicationTestStage$ByteBuddy$hNCI3voh.unloading_tenant(Unknown Source) at finance.egp.reporting.merger.ReportingMergerTest.GIVEN_the_analytics_datamodel_WHEN_merged_THEN_all_necessary_data_is_copied_correctly(ReportingMergerTest.java:100) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.springframework.test.context.junit4.statements.RunBeforeTestExecutionCallbacks.evaluate(RunBeforeTestExecutionCallbacks.java:73) at org.springframework.test.context.junit4.statements.RunAfterTestExecutionCallbacks.evaluate(RunAfterTestExecutionCallbacks.java:83) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75) at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86) at com.tngtech.jgiven.junit.JGivenMethodRule$1.evaluate(JGivenMethodRule.java:73) at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:251) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:97) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61) at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33) at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230) at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)

This seems to be the interesting part:

Caused by: java.io.IOException: Mark invalid at java.io.BufferedReader.reset(BufferedReader.java:512) at java.io.FilterReader.reset(FilterReader.java:117) at java.io.FilterReader.reset(FilterReader.java:117) at org.flywaydb.core.internal.parser.PlaceholderReplacingReader.reset(PlaceholderReplacingReader.java:177) at java.io.FilterReader.reset(FilterReader.java:117) at org.flywaydb.core.internal.parser.PositionTrackingReader.reset(PositionTrackingReader.java:56) at java.io.FilterReader.reset(FilterReader.java:117) at org.flywaydb.core.internal.parser.RecordingReader.reset(RecordingReader.java:48) at java.io.FilterReader.reset(FilterReader.java:117) at org.flywaydb.core.internal.parser.PeekingReader.refillPeekBuffer(PeekingReader.java:80) at org.flywaydb.core.internal.parser.PeekingReader.peek(PeekingReader.java:183) at org.flywaydb.core.internal.parser.PeekingReader.peek(PeekingReader.java:165) at org.flywaydb.core.internal.parser.Parser.readToken(Parser.java:483) at org.flywaydb.core.internal.parser.Parser.getNextStatement(Parser.java:161) ... 76 more

@MikielAgutu
Copy link

MikielAgutu commented Jul 9, 2020

Can you share the problematic script?

@ankos
Copy link
Author

ankos commented Jul 9, 2020

The script itself is not problematic. The execution with versions < 6.5.0 work just fine, all 900+ scripts are executed. As soon as I switch to version 6.5.0 in my pom.xml the error occurs. I have tried it several times.

I know that the version has changed, because flyway prints some log statements which include the version number

@JonLally
Copy link

JonLally commented Jul 10, 2020

Can confirm this issue is in version 6.5.0 and believe it is related to the fix for #2853 . Was able to reproduce the issue with a small sql for postgres with placeholders

CREATE TABLE people (
    id SERIAL primary key,
    first_name CHARACTER VARYING(50),
    surname CHARACTER VARYING(50),
    country CHARACTER VARYING(50)
);

INSERT INTO people (first_name, surname, country) VALUES ('${first_person_first_name}', '${first_person_surname}', '${first_person_country}');

When I provide the placeholder values 'John', 'Doe' and 'Ireland' respectively I get the exception reported above. I think this is because those placeholder values make the script 245 characters long, 55 less than its original 300. The PeekingReader.refillPeekBuffer() method keeps adding to the peekBuffer after 245 characters is reached. When reset() is called the BufferedReader throws the exception because markedChar is now -2. If I remove the check !supportsPeekingMultipleLines inside PeekingReader.refillPeekBuffer() the script gets migrated successfully.

@Lyeeedar Lyeeedar self-assigned this Jul 13, 2020
@Lyeeedar Lyeeedar added this to the Flyway 6.5.1 milestone Jul 13, 2020
@petarov
Copy link

petarov commented Jul 15, 2020

I had the same java.io.IOException: Mark invalid issue yesterday. Downgrading to 6.4.4 as suggested seems to have fixed it.

@juliahayward
Copy link
Member

juliahayward commented Jul 16, 2020

Fixed in 6.5.2

juliahayward added a commit that referenced this issue Jul 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants