Selector shows wrong checksums

[Akira25]

The selector shows wrong checksums for the images. In fact, it shows the checksums of only one image-type, even if we select the other one. So at least for half of the images, there get correct checksums displayed.

when this bug is fixed, we can drop e7fe664

[khanku]

I started looking at this a while back but was thrown off by the missing (freifunk-berlin specific) installation/usage instructions.
Maybe https://github.com/freifunk-berlin/falter-firmware-selector#installation could be updated? Or maybe you could explain how collect.py is run here @Akira25?

[mwarning]

@Akira25 since I wrote the collect.py script, is there something I can help with?

[Akira25]

@mwarning maybe you can. Really nice, that you provide us with in that issue. :)

@khanku I just added some hacky shell-scripts, that where on the firmware-selector-machine, but no added in the versioning-system.

The whole process is a bit hacky and messy. Probably the collect.py-script can do it all alone with just minor adjustments, but I didn't have had the time to read it all through now.

First thing to notice, we kind of "misuse" the version-pulldown menue to provide different flavors of the firmware too.

For deployment we currently do at that way:

1. checkout the falter_stable branch on deployment-machine into vhost-root.
2. link the files from subdir www/ into webserver-root (current directory)
3. run ./fetch_release.sh 1.2.1, which will call get_profiles_local.py. That script will get the json-files we aim for through collept.py and place them in a subdir 1.2.1/ into our vhost-root. In that subdir, theres a dir for tunneldigger and notunnel
4. afterwards I adjust the config.js by hand to have an entry for the tunneldigger-version and notunnel-version.

Everything quite hacky, I know. I'm sorry, that I didn't had the time to make it proper.

[Akira25]

The whole stuff with linking files into another direction comes from a time, where I hadn't root access to the machine, but only to that specific directory. So I was not able to adjust the webservers config accordingly. That will be better in the near future. (freifunk-berlin/ansible@3bb08c1 line 39 and after).

[khanku]

First thing to notice, we kind of "misuse" the version-pulldown menue to provide different flavors of the firmware too.

I think that is the crux of the issue. IIRC the checksum that's shown is always the one from the first flavour. The scripts you added should help troubleshooting.

[khanku]

It's weird: it works on my machine(tm)!
Here is what I did:

./get_profiles_from.py https://firmware.berlin.freifunk.net/stable/1.2.1/
jq -r .images[0].sha256 <1.2.1/tunneldigger/ipq40xx/mikrotik/mikrotik_sxtsq-5-ac.json
6de12efeebbc411ef4a4c12a3a2e2cb8e8e610db7842415012e8e713ceff83ce  #correct
jq -r .images[0].sha256 <1.2.1/notunnel/ipq40xx/mikrotik/mikrotik_sxtsq-5-ac.json
cbf96b3026ab5bb3f0fbedc9cabf65fd7937eae02d273d14d8e27fefacd68d7a  # correct
# also works in my browser, locally

Those are the correct checksums that can be found at https://firmware.berlin.freifunk.net/stable/1.2.1/tunneldigger/ipq40xx/mikrotik/sha256sums and https://firmware.berlin.freifunk.net/stable/1.2.1/notunnel/ipq40xx/mikrotik/sha256sums respectively.

Now if I go to https://selector.berlin.freifunk.net/ I can see that both https://selector.berlin.freifunk.net/1.2.1/tunneldigger/ipq40xx/mikrotik/mikrotik_sxtsq-5-ac.json and https://selector.berlin.freifunk.net/1.2.1/notunnel/ipq40xx/mikrotik/mikrotik_sxtsq-5-ac.json have the same checksum. And the same build timestamp (up to the second) which makes me think they're the same file.

Since the differences between get_profiles_from.py and get_profiles_local.py are minimal I'm going to guess that the data in /usr/local/src/www/htdocs/buildbot/stable/$VERSION/ is the source of the error.
@Akira25 you seem to have access to that machine. Could you check the files and confirm/infirm?

[Akira25]

you can find the original data freely readable beneath the images at http://firmware.berlin.freifunk.net/, which maps to /usr/local/src/www/htdocs/buildbot/.

The copied data (fetched by collect.py) is accessable at https://selector.berlin.freifunk.net/1.2.1/, which maps to /usr/local/src/www/htdocs/firmware-selector/1.2.1/

[Akira25]

in result, the original data from /usr/local/src/www/htdocs/buildbot/stable/$VERSION/ holds the correct checksums. I assume that we maybe use collect.py in a different way than intended.

[khanku]

I believe I've found the issue: when passed a local URI collect.py will use its scan function which will always pick the first (freifunk-specific) flavour. That's what happens in get_profile_local.py and results in the checksum issue which get_profile_from.py does not exhibit.

[Akira25]

This was fixed by #4 . Thanks a lot to @khanku for your contribution.