Calculating hash as a way to determine version

[casualshammy]

Hello.

I guess it is really difficult to determine version of addons on third-party websites, so thank you for your hard work.

As a 'last solution', could you consider using any hash algoritm? Something like option "Use hashing algoritm to determine addon version". It straitforward way to handle cases like Easy Scrap - two different versions of this addon have the same Version tag in .toc file. I think you just have not time to ask every addon author to keep information actual.

There is a problem in hash calculation - you need to download the latest file, unzip it and then do hashing on folder(s). It may be network- (and sometimes CPU-) intensive, but it worth its purpose.

[mlablah]

Hi casualshammy.

Thanks for your suggestion to use hashing as a kind of fingerprinting to detect version changes independent of actual version reported by local addon and the api's we query.

This prompted us to discuss it internally (@casperstorm and I) and we sort of broke it down into a couple of scenarios.

Scenario 1a: To handle addons that don't have a version in toc

• calculate hash of local addon
• lookup addon in api
• if it has the same reported version, download immediately
• calculate hash of downloaded addon
• compare to determine if update is needed

We do not think this would be a good idea, since it would mean initiating a lot of downloads (some addons are huge) without the user expecting it. Another issue with that is that we currently use the absence of a version in the toc file to indicate the addon should not be displayed as it is a "sub-addon". For example: Bagnon's "BagBrother" folder.

Scenario 1b: To handle addons that report wrong version in toc (always update-able)

• calculate hash of local addon
• lookup addon in api
• if it has a different reported version, download immediately
• calculate hash of downloaded addon
• compare to determine if update is needed

For the same reason as 1a we don't want to initiate any downloads without user consent. And it would be a lot of effort to fix what is essentially a mistake on behalf of the addon author who forgot to bump the version in the toc file.

Scenario 2: Use api provided fingerprint and compare with computed fingerprint of local addon

• The Curseforge api provides a fingerprint value for each download, which seems to be a custom hash of the main toc file in the download. We could in theory perform the same calculation for local toc files (eg. https://github.com/camas/grunt/blob/master/src/murmur2.rs ) and compare with api.

Doing this could potentially detect rare instances were Ajour wrongly thinks the local version is the same as the remote version. But it would rely on a reverse-enginereed hash algorithm and be very api dependent.

But we appreciate the suggestion, and perhaps there are other ways in which we could use hashing to improve Ajour, that we haven't thought of!

EDIT: As a workaround to addons that are "always update-able" due to wrong version specified in toc file, I can recommend the ignore function in Ajour's settings. It will hide the addon from the list and exclude it from the update all action.

[casualshammy]

Ah, I forgot about subfolders... But what if

1. Get list of all folders in AddOns folder
2. Look for corresponding addons in api (BTW, any plans for supporting wowace? DeathNote wants to be in Ajour! :) )
3. Download all addons. I understand that it is not 1 MB of data, but even raider.io is just about 70 MB. Anyway, this "Use hashing algoritm to determine addon version" may be optional.
4. Unpack all addons. Now we know all subfolders per addon. Here we do hash comparison. After that we have the list of 'good' addons, their subfolders and update status.
5. Finally, we have list of 'orphan' addons. We can theat them as we wish. 'Unsupported', 'Abandoned', 'Not found', etc. We even can just place this addons in the bottom of the list in Ajour.

Why I think you can't rely on Version tag? Because, for example, developer of TipTac doesn't know that he shouldn't place Version tag in .toc files in subfolders, and haven't set Dependencies tag. It is normal for WoW client, it is normal for Twitch client, but it breaks Ajour.

[mlablah]

BTW, any plans for supporting wowace? DeathNote wants to be in Ajour!

Looks like you've discovered another edge case we weren't aware of! :)

Death Note is also on Curseforge but it looks like the title in the toc (DeathNote) is different from the title in the api (Death Note) so we don't find it when we search for it. I think this would also be a problem using wowace, because if I search for DeathNote there I get no results.

Download all addons. I understand that it is not 1 MB of data, but even raider.io is just about 70 MB. Anyway, this "Use hashing algoritm to determine addon version" may be optional.
Unpack all addons. Now we know all subfolders per addon. Here we do hash comparison. After that we have the list of 'good' addons, their subfolders and update status.

We would really (really) like to avoid maintaining a "state" between executions of ajour, though the ignore/unignore feature does violate that principle slightly.
Our goal has been to be able to create a "state" quickly, and just by parsing the addons folder whenever the application is started. Without relying on api's that introduce extra dependencies we aren't in control of.

So we came up with logic to (try to) achieve that without actually knowing anything about what the original download contained.

And this is how it looks at the moment:
https://github.com/casperstorm/ajour/blob/master/src/addon.rs#L174-L192

It roughly goes like this:
Show something if it has a version, unless it's dependent on (or something depends on it) which has a title that is a substring of its own title.

The gui then does some extra grouping of addons if they have the same id (so that downloads like "Deadly Boss Mods (DBM) - Legion" don't show up as 7 addons).

It's not perfect, but it works surprisingly well.

Of the 99 most popular addons on curseforge, 91 are displayed correctly, 4 are hidden due to missing version in toc, and 4 are hidden because they are considered sub addons (HandyNotes and Atlas extensions)

Finally, we have list of 'orphan' addons. We can theat them as we wish. 'Unsupported', 'Abandoned', 'Not found', etc. We even can just place this addons in the bottom of the list in Ajour.

I think this is a good idea for a future enhancement! We should consider showing (somewhere) a list of addons that are not considered dependencies of any shown addon, and perhaps allow the user to force an update of those.

[casualshammy]

We would really (really) like to avoid maintaining a "state" between executions of ajour, though the ignore/unignore feature does violate that principle slightly.
Our goal has been to be able to create a "state" quickly, and just by parsing the addons folder whenever the application is started. Without relying on api's that introduce extra dependencies we aren't in control of.

So we came up with logic to (try to) achieve that without actually knowing anything about what the original download contained.

Maybe I explained my idea wrong (you don't need to save any information between sessions), but I can understand your focus on app performance. I just personally prefer to make it slower, but more reliable :) Unfortunately, I don't know Rust, so I cannot break something in your program (ha-ha).
Thank you very much for detailed answers!