How to compare two different commits in benchmark?

[alaershov]

I tried using two scenarios like this:

incremental_build_before {
    tasks = ["assembleDebug"]
    apply-abi-change-to = "app/src/main/java/com/example/profiledapplication/MainActivity.kt"
    git-checkout {
        build = "8e8fdcbb6a93f465cfbd48882c27b53a8297d469"
    }
}

incremental_build_after {
    tasks = ["assembleDebug"]
    apply-abi-change-to = "app/src/main/java/com/example/profiledapplication/MainActivity.kt"
    git-checkout {
        build = "c7afcd0a92921d3e56c8730a31ba089fd391cf04"
    }
}

But after running the first scenario I get
error: Your local changes to the following files would be overwritten by checkout: app/src/main/java/com/example/profiledapplication/MainActivity.kt Please commit your changes or stash them before you switch branches.

[alaershov]

It works fine without apply-abi-change-to, is there any way to make them work together?

[alaershov]

Seems that ApplyAbiChangeToSourceFileMutator executes before GitCheckoutMutator. It probably should go before all apply change mutators.

[gabrielfeo]

What I've been doing to work around this is I benchmark the two scenarios separately and then merge the result HTMLs with this script: merge-gradle-profiler-results.py