[Feature] Compare downloaded hash of nightly with installed version's hash

[se35710]

Feature Request

Is your feature request related to a problem? Please describe.

Nightly versions are not always updated, eg it's the same hash of the downloaded file as the one that is installed. With the new update_nightly Boolean set to true, this just creates copy after copy of the same files.

Describe the solution you'd like

A simple check of the downloaded hash, and the installed hash. For this to work, either add the hash attribute and value to the manifest file (manifest.json) in the installation directory, or to the install.json file with a if nightly read hash from install.json check.

Describe alternatives you've considered

None comes to mind.

[niheaven]

But the file is downloaded to compute the hash, right? So why we need compare these hashes and decide to not copy the file?

Could you give me some usages? I couldn't understand the request here.

[HUMORCE]

the feature you want in fact: the Excavator checks the nightly manifest and updates version and hash value of the manifest by hash value changes instead of checkver.

from:

"version": "nightly-20221010",
"nightly": True,
"url": "http://example.org/foo.zip",
"hash": "100"

to:

"version": "nightly-20221111",
"nightly": True,
"url": "http://example.org/foo.zip",
"hash": "999"

but the nightly is designed to skip them, nightly manifest usually does not have hash value.

[se35710]

With the new update_nightly Boolean set to true, this just creates copy after copy of the same files.

So even if the file changes on another intervall than daily, a new date folder is created and symlinked.

Maybe I am just trying to use latest as nightly, but it is convenient for downloads that doesn't contain a unique identifier in the url for each version.

[se35710]

the feature you want in fact: the Excavator checks the nightly manifest and updates version and hash value of the manifest by hash value changes instead of checkver.

No, I this is not about excavator, but instead the installer logic. We don't want to put hashes in nightly manifest files.

[HUMORCE]

So even if the file changes on another intervall than daily, a new date folder is created and symlinked.

I think it already exists, just not perfect.

❯ scoop update keepass-plugin-keetraytotp --force

scoop\apps\keepass-plugin-keetraytotp
❯ ls

    Directory: C:\Users\HUMOR\scoop\apps\keepass-plugin-keetraytotp

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d----           11/2/2021  3:01 PM                _0.108.0.old
d----          11/27/2022  5:21 PM                _0.108.0.old(1)
d----          11/27/2022  5:25 PM                0.108.0
l-r--          11/27/2022  5:25 PM                current -> C:\Users\HUMOR\scoop\apps\keepass-
                                                  plugin-keetraytotp\0.108.0

[HUMORCE]

It works well if it is not forced to be updated several times in same day.

scoop\apps\fiddler-classic-addon-jsformat
❯ ls

    Directory: C:\Users\HUMOR\scoop\apps\fiddler-classic-addon-jsformat

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
l-r--          11/27/2022  5:34 PM                current ->
                                                  C:\Users\HUMOR\scoop\apps\fiddler-classic-addon-jsformat\nightly-20221127
d----          11/27/2022  5:34 PM                nightly-20221126
d----          11/27/2022  5:34 PM                nightly-20221127

❯ scoop reset fiddler-classic-addon-jsformat@nightly-20221126
Resetting fiddler-classic-addon-jsformat (nightly-20221126).
Linking ~\scoop\apps\fiddler-classic-addon-jsformat\current => ~\scoop\apps\fiddler-classic-addon-jsformat\nightly-20221126

p.s.

emmm, I executed scoop cleanup * by mistake, a few minutes ago.
seem I doing similar thing: I rename specific version as prod|dev for some apps, but scoop cleanup * destroy all of them.

version control? or manage of manifests/apps:

we can:

• remove all old versions: scoop cleanup <specific_app>|*
• reset to specific version (installed) scoop reset <app>
• install specific version when available

cannot be done with scoop command:

• remove|rename specific version (of specific app)
• hold(?) make specific version of app cannot be deleted via scoop cleanup *,
• record unique identifier for nightly installation to avoid installs duplicate version of nightly apps (this issue)

[se35710]

@niheaven

Could you give me some usages? I couldn't understand the request here.

a.json: nightly cron job builds the artifact, and the manifest url is https://example.com/nightly/a.7z
b.json: manually triggered job, can be anything from daily updates, but sometimes monthly, even yearly. Manifest url is https://example.com/latest/b.7z

Every time scoop update * is run, since update_nightly is set, a.7z and b.7z are downloaded and installed in their nightly-<date> folder. This is true, even if b hasn't been updated. This feature requests would still download the file, but not create the nightly-<date> folder and change current.

Now, as @HUMORCE points out, scoop cleanup b would cleanup all versions for b, but I still think it would be nicer if update_nightly only updated when b.7z changed.

[niheaven]

So what you need is not recreate the junction? Or you just need a persist version folder?

[se35710]

If we could save the hash after a successful install phase of a nightly manifest, we could compare it with the hash calculated after the download phase the next time scoop update <manifest|*> is run. If the hashes are equal, abort the update for that manifest, leaving junction and old folder in place.