Reports show git commits in the wrong order

[ali1234]

Describe the bug
When running wily on a git repo, and then generating a report, the report lists commits in the wrong order, and even contains commits that aren't part of the history.

To Reproduce
Steps to reproduce the behavior:

1. git clone https://github.com/ali1234/32blit-tools
2. git checkout wily-test
3. wily build src/ttblit
4. wily report src/ttblit
5. Compare report to git log

Expected behavior
Wily reports should list git commits in the order they appear in the git history, otherwise it looks like the complexity is constantly changing, when in fact the commit only introduced a few small changes.

Running Pycharm's terminal, Ubuntu 20.04, Python version 3.8.5, wily version 1.19.0

[devdanzin]

Confirmed. Easier to reproduce by using wily index and git log --oneline. History is linear and commits are in chronological sequence, so the cause isn't either of these. Investigating.

Output for wily index:

--------Configuration---------
Path: /home/danzin/projects/32blit-tools
Archiver: git
Operators: {'raw', 'cyclomatic', 'maintainability', 'halstead'}

-----------History------------
╒════════════╤═════════════════╤════════════╕
│ Revision   │ Author          │ Date       │
╞════════════╪═════════════════╪════════════╡
│ 22e1016    │ Alistair Buxton │ 2021-01-12 │
├────────────┼─────────────────┼────────────┤
│ 1ca573a    │ Alistair Buxton │ 2021-01-12 │
├────────────┼─────────────────┼────────────┤
│ 56ca29a    │ Alistair Buxton │ 2021-01-12 │
├────────────┼─────────────────┼────────────┤
│ 0b4d767    │ Alistair Buxton │ 2021-01-11 │
├────────────┼─────────────────┼────────────┤
│ 9e03d14    │ Alistair Buxton │ 2021-01-11 │
├────────────┼─────────────────┼────────────┤
│ 5a6befe    │ Alistair Buxton │ 2021-01-11 │
├────────────┼─────────────────┼────────────┤
│ c9c4207    │ Alistair Buxton │ 2021-01-11 │
├────────────┼─────────────────┼────────────┤
│ 7996aca    │ Alistair Buxton │ 2021-01-11 │
├────────────┼─────────────────┼────────────┤
│ 36f9cf9    │ Alistair Buxton │ 2021-01-11 │
├────────────┼─────────────────┼────────────┤
│ 6c3f523    │ Alistair Buxton │ 2021-01-11 │
├────────────┼─────────────────┼────────────┤
│ 69ae620    │ Alistair Buxton │ 2021-01-11 │
├────────────┼─────────────────┼────────────┤
│ 28701b5    │ Alistair Buxton │ 2021-01-11 │
├────────────┼─────────────────┼────────────┤
│ f24ab84    │ Alistair Buxton │ 2021-01-11 │
├────────────┼─────────────────┼────────────┤
│ e0bbe19    │ Philip Howard   │ 2021-01-11 │
├────────────┼─────────────────┼────────────┤
│ c1b8aaa    │ Alistair Buxton │ 2021-01-05 │
├────────────┼─────────────────┼────────────┤
│ 48ed205    │ Alistair Buxton │ 2021-01-02 │
├────────────┼─────────────────┼────────────┤
│ a228875    │ Alistair Buxton │ 2021-01-02 │
├────────────┼─────────────────┼────────────┤
│ fade5c9    │ Alistair Buxton │ 2021-01-02 │
├────────────┼─────────────────┼────────────┤
│ 47bcf8e    │ Alistair Buxton │ 2021-01-02 │
├────────────┼─────────────────┼────────────┤
│ 756dfff    │ Alistair Buxton │ 2021-01-02 │
├────────────┼─────────────────┼────────────┤
│ 48a54ac    │ Alistair Buxton │ 2021-01-02 │
├────────────┼─────────────────┼────────────┤

Output for git log --oneline:

56ca29a (HEAD -> wily-test, origin/wily-test) Remove unused code
1ca573a Only check we have a config once
22e1016 Remove working_path from packer
0b4d767 Move Tool.prepare() to AssetTool
f24ab84 Remove build() and prepare_options() methods of AssetTool
28701b5 Make packer only use AssetBuilders, not AssetTools
69ae620 Make AssetSource into a single static function
6c3f523 Make Tools call AssetBuilders directly
36f9cf9 Split AssetBuilder into two classes
7996aca Auto-register all Tools and load them dynamically
c9c4207 Split out RLE code into functions and re-implement with a generator
5a6befe Fix transparent colour palette update
9e03d14 Put the right path into the ImageAsset
e0bbe19 Merge pull request #62 from ali1234/asset-refactor
c1b8aaa Use importlib.import_module() to load implementations
addda07 Rename OutputFormat to AssetFormatter
468ec10 Only handle lists in join()
4d11268 Implement AssetWriter
d2db808 Make all formatters have components
cc7d72d Make output formatters instances of OutputFormat
48a54ac Clean-up formatters

[devdanzin]

This happens because we sort commits by date in:

wily/src/wily/cache.py

Line 167 in 72c4bb8

index = sorted(index, key=lambda k: k["date"], reverse=True)

But somehow all commits for a given day have the same date (trying to figure out why) and the sort ends up shuffling commits for a given day. It seems that we could actually simply not sort and index would be in the correct order.

[devdanzin]

The commits have the exact same date because they were committed on the exact same second. What wily uses is commit.committed_date, not commit.authored_date (which is what appears in normal git log). If we run git log --format=fuller we can see that the CommitDate is the same for several commits.

That causes the sort mentioned in my previous message to reorder commits with the same date. So we can either use commit.authored_date instead (which may be what users expect, as it's what git log shows), avoid sorting altogether, or live with the fact that, for commits that share the same CommitDate, results will be in wrong order.

Edit: it seems this "same commit time" issue arises from amending/rebasing commits.

[Wurstnase]

Looks like when you do not reverse the iter_commits here and do not sorted here all commits have the correct order.

However, sometimes changes in metrics are connected to the wrong commit.