Releases: Variablize Code For Bitcoin Core Releases

[harding]

Preview (looks the same!): http://dg2.dtrt.org/en/download
Diff of HTML _site: http://dg2.dtrt.org/diff.html
(English only from master (f3580f5) to this (6428f0c) with all plugins enabled)

This pull attempts to make the Bitcoin Core releases on Bitcoin.org easier and more flexible, while also requiring less maintenance and providing built-in testing.

• Easier: every release since I started contributing has required (or threatened to require) some last-minute change to the Download page template. This moves all of those template variables into the release file, and provides inline documentation, so they can easily be changed even by someone unfamilar with the backend code.

• More flexible: it might be the case (for example during an emergency release) that some files might not be available for download. This makes optional all of the files except for the SHA256SUMS.asc. If a file isn't available, the link will be replaced with plain text. Hovering over the text will produce the following tool tip:

  

• Less maintenance: It's currently impossible to fully prepare a release ahead of time because we don't know the release date. This makes the release date optional, so it can be set hours/days after the release.

• Built-in testing: A test is added to our Makefile to download the HTTP headers for each of the download files to check that the status is "200 OK". When I get Travis CI setup next week, this should make it reasonably safe for someone to publish a release even if they don't have access to a Jekyll site-build environment. (E.g. most (all?) core devs.)

---

In this pull:

• Commit ac0185e makes all the actual changes to _plugins/releases.rb and related files.
• Commit 6428f0c does nothing but rename the old _releases/ files.

[saivann]

Testing if download files exist: Awesome!

Moving the date in the YML: Indeed, much needed.. (Although I think it shouldn't be seen as optional in case someone forgets it, the RSS file and release notes page need it)

However I feel like moving and adding variables to release notes introduces more manual intervention. Perhaps I'm mistaken, but I wished we could only keep the date here so adding a release remains super-simple and "risk-free" for contributors. Even the title and magnet link could be automated.

The main issue with filenames now seems fixed upstream, builds should have deterministic names (and if they don't, your Makefile should now catch it). One issue I see with setting filenames in the release notes is that if any build file is added or removed, we end up updating 3 files (release note, plugin, template) instead of just the template. I feel like editing the template only is the intuitive way of... editing the template :) .

Setting some download file as unavailable: FWIW I think that has never been needed yet, but there's no harm in having the feature! However since it's uncommon, maybe this would be more useful as an opt-in instead of an opt-out. For example, perhaps you could allow to set optional_disable = "win32exe win64exe ...", and only mention the feature in the README?

[saivann]

@harding According to the stats, you could stop at 0.8.2 (the old links were generated but never referenced at this point):
https://bitcoin.org/stats/ (you can sort by urls)

[harding]

Ah, nice. I'll do that. Thanks!

[harding]

@saivann thanks!

Moving the date in the YML: Indeed, much needed.. (Although I think it shouldn't be seen as optional in case someone forgets it, the RSS file and release notes page need it)

They don't need it anymore. For the releases page, a Liquid if simply doesn't display the date if it isn't set. For the RSS page, an if-else uses the site build date if the release date isn't set. This fixes the need to amend the release commit everyday, which took ~10 minutes each time because of slow complete-site build times for the preview.

I can make it print a non-error warning so anyone building locally knows that the date should be set.

However I feel like moving and adding variables to release notes introduces more manual intervention.

I think it introduces the opportunity for manual intervention, but it currently requires even less manual intervention than now. Because of the %v variable, the only three pieces of information that need to be updated for a typical release are the date (optional), the version, and the magnet link (also optional)---that's one less than needs be updated now (the title needs to be updated now).

I can move stuff around in the 0.10.0.md and README so that this is more clear.

I feel like editing the template only is the intuitive way of... editing the template :) .

I can add a comment to the template and to the release instructions to describe what other files need to be edited. The goal here is to make it as absolutely easy as possible to make a release, even if it's a non-standard release, for someone not previously familiar with the site. That way I can go on vacation without worry. To me, easy means:

• Everything is in one place, as much as possible.
• Anything that might need to be changed, is easy to change.
• Documentation is provided as close to the decision-point as possible.

I like your focus on automation, and I've tried to preserve it for typical releases. But I've also tried to provide easy overrides for non-typical situations, and to allow for some imperfection in releases (e.g. optional date) that can be fixed up later.

[harding]

Commit d9a04d6 removes the unnecessary redirections and adds warnings for missing files.

@saivann I didn't do anything else as I see you're working on your own alterations. :-)

[saivann]

@harding Yes, indeed :). Although I don't want to be blocking or disrupting your work as I may be wrong :) . Since real examples speak more than words, I wished to at least provide one you could consider: saivann/bitcoin.org@fad1a0d. (Note, poorly tested, maybe incomplete)

In this example, I thought that we could perhaps not present features that are known to not be used yet outside of the README? Too much features can be intimidating, maybe there is a balance here.

Everything is in one place, as much as possible.

I feel it's a double-edged sword. Your changes are an improvement if only the name of a build file is changed. However in the example I mentionned (adding or removing a build file, which also seems to happen occasionally), it makes things quite more difficult (editing three files including a plugin). Editing the template file directly still provides the ultimate flexibility as to what is displayed on that page.

[harding]

@saivann rather than keep debating, it's probably best to extract out the parts of this PR that we both agree on and then worry about the rest during the next release cycle. What do you think about me closing this pull and creating a new pull with just the following features?

• Makefile test to verify the download files exist.
• Only three variables (like in your commit): required_version, optional_title, and optional_date
  • Print a non-stopping debug warning for any release that doesn't have a date set
• Remove code that generates /release/*/*/*/*.html and use redirects in _config.yaml
• A README description describing the above, and mentioning that _templates/download.html needs to be edited to change any URLs

[saivann]

@harding Sure, these changes as a first step all sound good to me, thanks!