Should go-git diff use modification times?

[hansbogert]

Plain git uses modification times heavily when doing git status for example. I noticed go-git simply always rehashes. However, for large repositories this is multiple times slower.
Is my observation correct? If so, do we want to implement the merkletrie with optional caching using the git index which holds the modification times?