junit tests and rename handling breaks with mercurial 3.4

[tarzanek]

Get https://mercurial.selenic.com/release/mercurial-3.4.tar.gz
use this hg instead of your common working hg (seems to be 3.1.2 on travis ubuntu and on jessie)
run junit tests - they will break on renamed files, history won't contain proper log entries, since hg 3.4 doesn't give you proper output anymore

[tarzanek]

(fwiw even 3.5-rc is affected)

[tarzanek]

also note when Solaris 11.3 will be released it will ship this hg, so this bug is kinda hot

[vladak]

Using the test repository from https://github.com/OpenGrok/OpenGrok/tree/master/testdata/repositories/mercurial (renamed hg to .hg) I see this:

hg 3.2.3:

$ hg log -f -r 'reverse(4:)' --template {rev}:{file_copies}\\n /var/tmp/mercurial/novel.txt
8:
7:bar.txt (foo.txt)novel.txt (book.txt)
6:
5:book.txt (plaintext.txt)
4:

hg 3.4:

$ hg log -f -r 'reverse(4:)' --template {rev}:{file_copies}\\n /var/tmp/mercurial/novel.txt 
8:
7:bar.txt (foo.txt)novel.txt (book.txt)

so 3.4 is missing 3 revisions.

[vladak]

The problem seems to be in the revision specifier. If I omit the -r option for hg 3.4, then full list of revisions is displayed.

[vladak]

Filed bug against Mercurial: https://bz.mercurial-scm.org/show_bug.cgi?id=4959

[vladak]

AFAIK this bug is only problem for displaying diffs of revisions across renames, it should not affect history index.

[cdgz]

I follow a bit mercurial, in 3.4 release notes: https://www.mercurial-scm.org/wiki/WhatsNew#Mercurial_3.4_.282015-05-01.29

log: make -fr show complete history from the given revs (BC)

flag BC means change sensitive towards Backwards Compatibility. There was also https://bz.mercurial-scm.org/show_bug.cgi?id=4921 and https://www.selenic.com/pipermail/mercurial/2015-October/048831.html. As said in 4921, technically it's a regression, but previous behavior (when both -r and -r supplied) is considered as misleading/dangerous.

Citing comment to breaking change:

old behavior is A) rare, B) easy to emulate (& ::.), and C) currently undefined

so probably small revset hack indicated by B) should fix our tests (sorry didn't go in details there..)

[vladak]

Thanks for the pointers. If I understand the change done to the log command correctly in 3.4, all it does is changes hg log -f -r REV to hg log -f -r 'reverse(::REV)'. This is done so that it saves the typing when one wants to display history of given revision.

The use case in OpenGrok is a bit different as the -r option is used to trim the list of revisions for given file. When MercurialRepository.java#findOriginalName() searches for original name of given file in given revision, we do not want Mercurial to fetch all the revisions of given file, we are interested only in history down to given revision. This is done to hopefully save some I/O and CPU as history of some files can have hundreds of changesets and usually the users are interested in the revisions near the top.

We can either remove lines 292 + 293:

283        /*
284         * Get the list of file renames for given file to the specified
285         * revision. We need to get them from the newest to the oldest
286         * (hence the reverse()) so that we can follow the renames down
287         * to the revision we are after.
288         */
289        argv.add(ensureCommand(CMD_PROPERTY_KEY, CMD_FALLBACK));
290        argv.add("log");
291        argv.add("-f");
292        argv.add("-r");
293        argv.add("reverse(" + rev_to_find + ":)");
294        argv.add("--template");
295        argv.add("{rev}:{file_copies}\\n");
296        argv.add(fullpath);

and hope the extra I/O and CPU load will not cause too big load.

Or, figure out what is meant by option B) and use the workaround - assuming it will be the same for Mercurial 3.4 and below. Certainly we do not want to have special cases in OpenGrok code for different Mercurial versions.

[vladak]

It seems that the Mercurial bug I filed has got some traction so it might be fixed in future Mercurial version. Until then I am thinking about commenting out lines 292+293. Will keep this bug open.