Ant task and CLI produce different changeset digests

[MaxAller]

Description
If you run <liquibase:updateDatabase .../> using Ant, and subsequently run liquibase update using the CLI, the CLI command will fail ("table already exists") since it'll try to run changesets that have already been applied.

To Reproduce

I've been using the changeset from the tutorial here and running it against an Aurora PostgreSQL database from a Linux host.

On the Ant side, I have something like this:

<property name="db.changelog" location="schema/db.changelog-master.xml" />

<taskdef resource="liquibase/integration/ant/antlib.xml" uri="antlib:liquibase.integration.ant">
    <classpath><snip/></classpath>
</taskdef>

<liquibase:database id="db"
                    driver="org.postgresql.Driver"
                    url="jdbc:postgresql://localhost:5432/<db>"
                    user="master"
                    password="${db.password}"/>

<target name="db:update" description="Runs all pending database migrations.">
    <liquibase:updateDatabase changeLogFile="${db.changelog}" databaseref="db" />
</target>

Then on the CLI side, this:

java -cp "..." liquibase.integration.commandline.Main \
    --classpath "..." \
    --defaultsFile="devscripts/liquibase.properties" \
    "$@"

liquibase.properties is

driver: org.postgresql.Driver
url: jdbc:postgresql://localhost:5432/<db>
changeLogFile: schema/db.changelog-master.xml

Version is LiquiBase 3.8.8.

Expected behavior
If I have an unapplied changeset, I would expect that running the Ant update task or the liquibase CLI update task in sequence, in either order, to work without throwing errors.

Screenshots
N/A

Additional context
Realistically I don't expect to be using the CLI for updates some of the time and Ant for updates the rest of the time. But it does call into question whether the CLI's reflective operations will be able to correctly identify which changesets have run. We might use the CLI sometimes because the functionality it offers is a superset of what the Ant tasks offer, so I'd like to be assured that it'll actually work when we need it.

[SteveDonie]

Would you mind adding one more thing to this? After running an update with Ant, show the results of running select * from DATABASECHANGELOG, and then do the same thing after running an update from the command line. I suspect that the issue is that the filename column in each will be slightly different, and that is what the underlying issue is.

[MaxAller]

I was close to resolving this issue because I couldn't repro anymore, but actually I can now after rolling back some of my changes.

If you have a db.changelog-master.xml file that includes another file, that works in both cases. However, if all your changesets are instead direcly in db.changelog-master.xml, that doesn't work. (With the tools consuming db.changelog-master.xml in both cases, obviously).

---

Now for what you asked for, when only using a single changelog file:

State when running with Ant first:

   id    | author |                                                 filename                                                  |        dateexecuted        | orderexecuted | exectype |               md5sum               |                                                 description                                                 | comments | tag | liquibase | contexts | labels | deployment_id 
---------+--------+-----------------------------------------------------------------------------------------------------------+----------------------------+---------------+----------+------------------------------------+-------------------------------------------------------------------------------------------------------------+----------+-----+-----------+----------+--------+---------------
 initial | aller  | /workplace/<full path to my project>/schema/db.changelog-master.xml | 2020-03-23 16:32:06.647243 |             1 | EXECUTED | 8:4af49915b417de20e16855073c2569f0 | createTable tableName=<snip>; createIndex indexName=<snip>, tableName=<snip> |          |     | 3.8.8     |          |        | 4981126588
(1 row)

CLI:

   id    | author |            filename            |        dateexecuted        | orderexecuted | exectype |               md5sum               |                                                 description                                                 | comments | tag | liquibase | contexts | labels | deployment_id 
---------+--------+--------------------------------+----------------------------+---------------+----------+------------------------------------+-------------------------------------------------------------------------------------------------------------+----------+-----+-----------+----------+--------+---------------
 initial | aller  | schema/db.changelog-master.xml | 2020-03-23 16:33:50.881025 |             1 | EXECUTED | 8:4af49915b417de20e16855073c2569f0 | createTable tableName=<snip>; createIndex indexName=<snip>, tableName=<snip> |          |     | 3.8.8     |          |        | 4981230827

---

Indeed, your guess was correct. In my Ant invocation, the path is an absolute path. In every other case, it's a relative path.

I was thinking LiquiBase should always convert the path to absolute, but that wouldn't be portable across different systems, right? So...is this anything more than a convoluted user error?

[MaxAller]

Er, actually I'd assumed that I was the one passing an absolute path into Ant, but in fact either Ant or the LiquiBase task is converting my relative path into an absolute path.

The workaround is not to put changesets directly in that file.

[SteveDonie]

Yeah, I think that is something we should look into with Ant. Seems to me like it should be using the relative path.
The other "workaround" is to actually specify the changelog path in the changelog itself.

There is an attribute you can put at the root of a changelog file named logicalFilePath that you can set. You might set yours to schema/db.changelog-master.xml and then it should work in both cases.

[MaxAller]

Cool, I'll give that logicalFilePath thing a try.

As for next steps, it's up to you. As a user, this is surprising and frustrating behavior 😅

[molivasdat]

Hi @MaxAller. Did Steve's suggestion of using logicalFilePath work for you?

[mattbertolini]

I recreated the issue described and solved it by changing the following line:

<property name="db.changelog" location="schema/db.changelog-master.xml" />

to:

<property name="db.changelog" value="schema/db.changelog-master.xml" />

This is because the location attribute of <property> will make an absolute path based on the ant project base directory (defined in the basedir attribute of the <project>. It sounds like the changeLogFile attribute is designed to be a relative path so to keep consistency with the CLI, making it a standard value will do that.

It might be a good idea to document that the ant task changeLogFile attribute should be a relative path so that others don't encounter this issue. If they need full paths they should leverage the changeLogDirectory attribute as well.

Hope this helps.

[molivasdat]

Hi @MaxAller Did @mattbertolini 's suggestion work for you? If so, I'll make sure that the documentation gets updated appropriately. @mccormickc

[ro-rah]

We should close this issue once we have a doc ticket.