Speed up re-signing via per-file hash

[laurentsimon]

Certain models like huggingface repositories currently do not support versioning. This means that signing multiple times with small diffs can be expensive. It may be useful to have a format listing per-file hashes, in order to speed up re-hashing. Example:

path/a/b ...hash1...
file2       ...hash2...

This file would be kept under the model folder or in another configuration folder (probably the latter?). When signing, the caller would give us a list of files that need to be re-computed (from git diff), and our library would only re-hash the necessary files. We'd also pass back the recomputed file to the caller, so that they can store it where they want. The user would need a git hook that does all this for them.

This file could be the final file we sign, or a helper file that is only stored on a dev machine to speed up signing when making changes to the repo. (probably the latter?)

The main danger (?) is that:

1. If someone edits the file (by mistake), we'd be re-computing the hash wrong.
2. Folks start sharing this file. It would only take one bad actor to backdoor the model for multiple users (?)

@McPatate any thoughts?

[McPatate]

Certain models like huggingface repositories currently do not support versioning.

Do you mean per-file versioning? You have access to a git log, which can be interpreted as a versioning system since you know if a file changed across commits & the commit id being the version.

Regarding 1., we could encode the file in a binary format that is not human readable.

Regarding 2., could we not sign the file each time it's changed?

[laurentsimon]

Certain models like huggingface repositories currently do not support versioning.

Do you mean per-file versioning?

No, just repository versioning.

You have access to a git log, which can be interpreted as a versioning system since you know if a file changed across commits & the commit id being the version.

Correct. I thought the huggingface API did not support git versioning and it could only download at head. I was wrong. from_pretrained supports git tag, called revision (https://huggingface.co/docs/transformers/main_classes/model). So effectively users can tag upon release. I don't think the feature is commonly used by users today.. but probably not a problem for us.

Regarding 1., we could encode the file in a binary format that is not human readable.

Regarding 2., could we not sign the file each time it's changed?

+1, I had a similar idea. I think that should work.

@McPatate do you think we need this optimization at all? The motivation was to improve signing speed because we thought that model versioning was not supported by the huggingface API, but that assumption was wrong. Do you think it'd be enough to:

1. Nudge users to version their models (git tag)
2. Provide a CLI they can run when they push a tag. (the CLI in this repo when it's v1 ready)

If users version their models, they'd only need to sign when the model actually changes.. which means they need to re-sign large files and cannot use the per-file hash anyway... Wdut?

[laurentsimon]

Closing in favor of #111