adjust timestamps when building from git-clone

[IzzySoft]

git clone sets the timestamps of files to now() (i.e. the time of the clone), which does not reflect their creation or commit times. That way, building a module ZIP from a repo that has not seen any commits for 2 years would result in a module being announced as "brand new". To avoid that, timestamps should be adjusted after cloning and before zipping.

To speed up implementation for that, you can find matching Python code here. Quoting for your convenience:

#!/usr/bin/env python
# Bare-bones version. Current directory must be top-level of work tree.
# Usage: git-restore-mtime-bare [pathspecs...]
# By default update all files
# Example: to only update only the README and files in ./doc:
# git-restore-mtime-bare README doc

import subprocess, shlex
import sys, os.path

filelist = set()
for path in (sys.argv[1:] or [os.path.curdir]):
    if os.path.isfile(path) or os.path.islink(path):
        filelist.add(os.path.relpath(path))
    elif os.path.isdir(path):
        for root, subdirs, files in os.walk(path):
            if '.git' in subdirs:
                subdirs.remove('.git')
            for file in files:
                filelist.add(os.path.relpath(os.path.join(root, file)))

mtime = 0
gitobj = subprocess.Popen(shlex.split('git whatchanged --pretty=%at'),
                          stdout=subprocess.PIPE)
for line in gitobj.stdout:
    line = line.strip()
    if not line: continue

    if line.startswith(':'):
        file = line.split('\t')[-1]
        if file in filelist:
            filelist.remove(file)
            #print mtime, file
            os.utime(file, (mtime, mtime))
    else:
        mtime = long(line)

    # All files done?
    if not filelist:
        break

No worries about performance impact, if the author is to be believed. They state it took 0.27s for the entire Bash repo, and still less than a minute for the entire Linux repo even. So it should not even be noticable for a single Magisk module 🙈

[SanmerDev]

https://github.com/ya0211/magisk-modules-repo-util/blob/6d7e36a1b9216c2851f0ad786b93d1fc057a4a85/sync/File.py#L109

The timestamp currently used for git clone is the last commit 😃.

[IzzySoft]

OK, that's a compromise. I just got irritated by all files having the same timestamp (but then, some devs set it to 1980-01-01 for all their files when zipping the module, which is much worse IMHO).

Maybe filing it as lower-prio nice-to-have then? Having all the details in modules.json (#7) is what I feel most important at the moment (see the index.json next to my modules.json for what I've cobbled together currently for easier access in different places). That said, I'd be curious what your road-map says 😉

[SanmerDev]

In order to save traffic, git clone uses depth=1, which causes us to not get the last committed of each file, so we can only set it as the current commit.

[IzzySoft]

Makes sense, thanks!