Nightly builds

[lifeforms]

Motivation

Right now, a person reports a FP, we fix it quickly, but it might take 6 months for the PR to become generally available in an easily downloadable zip.

Users could track our github but it is a more geeky action and it is discongruent with their normal integration process of using our zips.

Maybe even downstream users won't like our release schedule and they would prefer to make their CRS cuts at a different point in time than our releases (see Ubuntu versus Debian).

In general, our development branch is pretty stable, and these days we find relatively few blocking issues during RC testing.

Proposed solution

We should look into issuing nightly zip builds and publish those on our website.

People who have a successfully solved issue can quickly download an updated zip with their fix. (They should still carefully study the CHANGES file though, because we could have introduced backward incompatible changes in the meantime!)

Code changes to consider in the zip:

• These builds should contain some kind of notice (949110 and friends) so logs will easily tell us at which point in time the snapshot was taken (e.g. 20210315).
• The rules in this build could have a 'ver' action (like 3.4.0-pre or 3.4.0-nightly or 3.4.0-20210315) instead of the ver (3.4.0) that is in the rule.

Ideally this would be done with a msc_parser script.

Alternatives

Keep using Github for pre-releases and direct people to use our Git contents instead of zip files, as we do now.

Additional context

Github Actions support cron actions so we could generate a zip file daily and put this on the website. It will involve quite some advanced Github Action mastery, so we shouldn't expect it too soon.

[fzipi]

Hey @lifeforms , I've been playing a bit with this.

Take a look at https://github.com/fzipi/coreruleset/releases.

There you have some of the options that can be done automatically. My conclusion is that, unless we create a github action ourselves, the ones have limited support. In summary:

• a body can be added to the release, so the text reflects what can be expected from it (e.g. in Nightly Releases)
• another option is to add the text from the previous commits in the release, since the previous release (a.k.a. changelog)
• both will add, by default, the source code in zip and tar.gz format (this can't be changed). In this case, the source will have the latest -dev tag, without dates nor others information :/

Let me know what you think.

[dune73]

Thank you for looking into this @fzipi.

I am not quite sure I follow you. So you think the existing functionality you demonstrated at the URI linked is overly limited and you propose we write our own action.

Questions:

• Is it too limited or good enough from your perspective. From my perspective it might just be good enough.
• Can we write our own github action to run this? Complicated? How to run it?

Personally, I could live with the -dev ending even if it's not beautiful. But I think it is accepted for nightly builds.

[fzipi]

Well, we can get the files, but we will always get the additional zip and targz files of the commits in master.

So my vote will be for just publishing what we have in master for nightly.

We can write our own github action, of course, but need to review the API to see if we can do a release that doesn't include the default files or something like that. This will take additional time.

[dune73]

Hold on, when we do the nightly builds as you propose, we get a new commit with a zip blob every night?

Still not understanding this properly.

How much time for the basic setup you propose?

[fzipi]

You get one nightly every day, yes.

The basic setup, 5-10 minutes, and one merge.

[dune73]

An automatic commit with an update to an file in the main branch every night is a no-go, I think.

[fzipi]

Let's try again. I'm not explaining myself correctly 😄

• to do this, we need to run a new github action every day
• the action basically creates a new instance of a release, called "nightly"
• will take whatever has reached the "master" branch "v3.4/dev", create a new zip and tar.gz file, and add it to the release
• there will be only one pair of files in the release
• nothing is commited to the repository
• the basic setup you asked needs the new file mentioned with the github action for running this
• that's it!

[dune73]

OK. Thank you. Sounds much better than what I misunderstood.

But the nightly builds always have the same name / file names and they will clog the release / tags view with a new entry every day?

If not, it's a no-brainer, I think. Running with a stable file name for nightly builds seems pretty standard for me.

[fzipi]

Yes, they will overwrite the last one. I think you can have the last X releases around, but need to double check.

For me, this will be an way for downstream providers to have a quick curl and script their tests, instead of waiting for a release. But they can do the same using just git. It is just "nice to have".

[dune73]

Do you have a status update here @fzipi?

[fzipi]

Yes @dune73. Added PR and can be merged anytime.