confused with the latest changes - one branch?

[perexg]

It seems that this repo moved again from the branch system (one branch = one version) to all-in-one branch. It's a bit unclear to me, how I can fetch one version for packaging. And it's also unclear, how you'll tag the releases.

[lgirdwood]

@perexg sorry again, we've had a lot of discussion on how to improve this with the SOF developers, and @marc-hb has been improving how we move forward here. We have a pending Readme patch to be merged before we tag today, i.e. the binaries are all correct but we will tag once the readme update is merged today.

[perexg]

Ok, but how I can fetch only v1.8 with the current scheme? If you tag the main branch, I'll fetch both 1.7 and 1.8. And if there are 10 releases, I'll end to fetch all of them.

The solution may be to filter the files in the release assets, but they have wrong names (v1.7.tar.bz instead sof-bin-1.7.tar.gz) or return back to the release branches.

[lgirdwood]

@marc-hb please also tag the release. You have maintainer access so should be able to do this via GH or git. The known issues are here https://github.com/thesofproject/sof/milestone/15 for linking to in release.

[lgirdwood]

Ok, but how I can fetch only v1.8 with the current scheme? If you tag the main branch, I'll fetch both 1.7 and 1.8. And if there are 10 releases, I'll end to fetch all of them.

The solution may be to filter the files in the release assets, but they have wrong names (v1.7.tar.bz instead sof-bin-1.7.tar.gz).

I think @marc-hb has some flows for installation. Lets wait until he's online later today.

[marc-hb]

It seems that this repo moved again from the branch system (one branch = one version) to all-in-one branch.

That's correct.

It's a bit unclear to me, how I can fetch one version for packaging.

You can't do that anymore. The longer term fix for that is to move away from using git to release binaries. For now the entire .git/ folder is 10M.

And it's also unclear, how you'll tag the releases.

As you noted, tagging the releases in sof-bin does not really make sense anymore.

The 1.7 version in the README.md is obviously an example.

[lgirdwood]

@marc-hb thanks for your inputs here, are there any docs in sof-docs or GH feature request that describe the release process roadmap and your current plans. I assume the end goal is to have a script that can create a tar.bz2 of FW, ldcs and topolgies for someone to attach to GH releases ?

[perexg]

You can't do that anymore. The longer term fix for that is to move away from using git to release binaries. For now the entire .git/ folder is 10M.

Just a note: From the storage perspective, it does not matter, if you store different source binary files to one branch or multiple branches. Perhaps, git merging may help, too.

I'm using the auto-generated tar balls by github now (using the commit-id not your tag which was floating previously). In this case, only the necessary files are downloaded. Your move to artifacts means that you plan to use own scripts for the release, which is also fine, but the storage can be used efficiently with the branches, too like:

• main branch (without binary files, only license files and scripts)
• version branch (with binary files for the specific version + merged main branch)

In this case, you can just tag the version branch and you'll get the automatic tar ball from github.

Example of commit-id URL to get the tar ball from github for v1.7: https://github.com/thesofproject/sof-bin/archive/6bd5501e6ac3f29dde42a28ffe03a2942aa60640/sof-bin-6bd5501.tar.gz .

[perexg]

Or you may create meta branch with scripts and doc files which will be merged to main (all branches) and to separate release branches. There are multiple possibilities.

[marc-hb]

I assume the end goal is to have a script that can create a tar.bz2 of FW, ldcs and topolgies ...

Most of this script already exists and I use it many times everyday: https://github.com/thesofproject/sof/tree/main/installer

It's still missing a few features: final tarball, checksums, compilation flags,... I was hoping to get some user feedback to help prioritize further development but I'm afraid most people still use xtensa-build-all.sh + build-tools.sh + their own, private scripts on top. Cause why change? This is also the case of our internal Jenkins-based build and validation; this mismatch makes each release time-consuming and error-prone.

It's been unsurprisingly hard to get people excited about packaging and releases so the only way forward has been slow: try something (not too) different and wait a bit to see if anyone is confused or complains. There :-)

... for someone to attach to GH releases ?

Yes, my idea was to eventually just upload the output of the installer directly to https://docs.github.com/en/github/administering-a-repository/releasing-projects-on-github/managing-releases-in-a-repository and deprecate the entire sof-bin repo: everything required to perform releases would be self-contained in the main sof repo, no external dependencies and no other branches or tag besides the "real" sof ones. Hopefully simple enough not to require any Powerpoint: configure and run the installer/, upload to github releases, done. No secret sauce hidden in Jenkins or wherever.

Example: https://github.com/zephyrproject-rtos/sdk-ng/releases

I'm using the auto-generated tar balls by github now (using the commit-id not your tag which was floating previously). In this case, only the necessary files are downloaded.

First, thanks for this clarification and your expert feedback in general: even more useful considering how rare you are.

I didn't even know github had such a convenient git archive implementation, sorry I "broke" that. I understand the convenience of tagging sof-bin now, I will do today.

Once I tag sof-bin this git archive feature should not really be "broken" any more, just less efficient. Do you mind ignoring the older versions until we move to github releases? Just a few spurious megabytes every few months.

For the record I "announced" the move to a single branch in January 2021 (among many other things, some abandoned). I understand that didn't make any headline :-) thesofproject/sof#3775 (comment)

Cc: @aiChaoSONG , @keyonjie , @fredoh9 , @plbossart , @mrhpearson @cujomalainey , please forward to any other (rare!) person who might have an interest for this topic.

[plbossart]

FWIW we added recently a pointer to sof-bin in the drivers:

sound/soc/sof/loader.c:                 "you may need to download the firmware from https://github.com/thesofproject/sof-bin/\n");
sound/soc/sof/topology.c:                       "you may need to download the firmware from https://github.com/thesofproject/sof-bin/\n");

you may change directions on how you want to deliver the binaries but that site has to stay.

[marc-hb]

Good point thanks! We can have a simple "redirection" in the top-level sof-bin/README.md

As you made no other comment, I assume you agree with all the rest? :-)

[plbossart]

As long as @perexg is happy I am happy.