index versions

[eightfilms]

Closes #79

Contains a workflow that should be triggered by other repositories to publish versions onto gh pages. Currently still testing this on the private dummy repo but leaving this up for visibility

Favouring a pull-based approach here, where we have a scheduled cron workflow pulling versions from other repos every 30 minutes, instead of a push-based approach.

Both solutions aren't nice - either we don't have instant availability of new releases with the pull-based approach by querying the latest tag through the GitHub API at set intervals (for now - set to 30 minutes) while ensuring that other repos are ignorant of fuelup, or we give up this ignorance of fuelup with the push-based approach by pushing the latest tag into fuelup from other repos, ensuring we have instant availability of the latest versions.

It seems that the pull-based approach is the lesser of the two evils since it might get messy trying to push data into this repo to be published, especially if we want to reorganize repos or add new components/plugins in the future to the default toolchain. Also, a 30 minute polling interval doesn't seem too long of an inconvenience/wait for new releases.

As far as I am aware - there doesn't seem to be any GitHub Action that supports subscribing to other repos for releases and executing a workflow thereafter - happy to get rid of this script in favour of something more robust otherwise!

[eightfilms]

I did most of the work on this through a .sh script and tested it on a personal repo so as to not pollute commits but you can check it here: https://github.com/binggh/fetch-versions-test/tree/gh-pages

The github yaml configuration in that repo is set to workflow_dispatch so that I can run it on demand to test, but the correct configuration should be a schedule cron job that runs every 30 minutes, querying the latest release tags from either the sway or fuel-core repo and publishing a new version if needed.

this is all within a 80-line-ish script and i'd imagine we wouldn't have to make much changes on the script (index-versions.sh) unless we make changes to the toml layout. At first was contemplating if it should be in Rust but seemed like it would be more convenient to do everything through scripting since we'd need to curl + calculate the sha256 hash (following the convention in Rust channels).

To add additional channels, eg. channel-fuel-nightly.toml and channel-fuel-stable.toml, I'd imagine we can run the same script 3 times and input the filename as an arg, with additional steps in between (for example, the conditions for updating the nightly channel is different from latest channel)

With this index we also no longer need to query the GitHub API within fuelup to fetch the latest version and build the download links manually - instead we can fetch and parse this toml for the download links and only fallback to using the github api if somehow fetching this toml fails.

Still some cleanup to do but otherwise the above should be a good gauge of what it looks like. Should be ready for review!

[JoshuaBatty]

At first was contemplating if it should be in Rust but seemed like it would be more convenient to do everything through scripting since we'd need to curl + calculate the sha256 hash (following the convention in Rust channels).

Nice work @binggh. I have to admit that my bash knowledge is pretty flaky so from a reviewing and maintenance point of view I would favor taking a Rust approach but can understand where you are coming from.

[mitchmindtree]

Lgtm - thanks for providing the demo repo to show it works too!

We are missing one big step: integration testing of the fetched versions, and falling back to the old ones if they fail. We should also keep track of version sets that are known to fail to avoid having this CI job stuck in an infinite loop of failing tests in the case that the latest set happen to fail for whatever reason. We can address all this in a future PR 👍

[eightfilms]

@JoshuaBatty was honestly considering to switch to Rust writing this midway because it'd be much more convenient writing/reading TOML files that way, but on the other hand curl and shasum are just one liners in a script. If in future we need more complex logic I think it'd be trivial to rewrite what we already have here into Rust and then add additional logic on top of that imo!