Discuss Starting Point

[noelmiller]

I opened this thread at the request of @xynydev

Here is what has already been discussed on Discord: https://www.answeroverflow.com/m/1195027063040643154

I want this to be an open (and constructive) conversation regarding Starting Point and how we would like to move forward.

[xynydev]

Cataloging things talked about in the discord thread (with short responses):

• Maintanence burden, having to sync frequently with upstream. (from discord user M2)
  • I don't even sync with upstream constantly myself, and try to limit the amount of changes (especially those requiring manual merge) in the template. That being said, a small maintenance burden will basically always exist with custom images.
• Over-abstraction, making it harder to build locally. Documentation for just using the classic containerfile syntax and tech like that could be preferred. (from @EyeCantCU)
  • I agree on the local building part. It should be possible pretty easily, just needs some docs.
  • I actually wrote some shallow docs for setting up custom builds with just Containerfiles, but it hasn't been merged. feat: syncing page and minimal setup page for tinkerer's guide ublue-os/website#688
  • The abstraction point will be touched on in my next comment.
• Issues of scope. Who is this for? Should Bazzite/Bluefin/co. use startingpoint? (mostly from @fiftydinar and @bsherman)
  • I'll quickly point out here that the methodology used in startingpoint doesn't need to be for everyone. The custom images that don't use it have good reasons to, and the maintainers seem happy with the state of their projects. I don't want to push this on everyone. I recognize that differing setups have different benefits, there doesn't need to be a single compromise for everyone.
  • This will also be talked about in my next comment.
• Concerns with forking startingpoint being seen as the official way to make custom images. (mostly from @bsherman but I've seen this from others too)
  • This will also be talked about in my next comment. I don't want to oversimplify this issue.
• What if we just made a UI that created Containerfiles? (from @bigpod98)
  • Actually, part of the goal of startingpoint's recipe.yml is to make it possible to make an UI for custom image configuration. I'm not saying it would be impossible to make a direct to Containerfile UI. There's just two ways I see it could go: 1. it just becomes a not-that-useful Containerfile IDE with useful snippets, or 2. it internally stores a structure similar to recipe.yml used to generate the image (at this point, why not make this configuration hand-editable and the UI optional to use).

This list is not comprehensive, and some concerns were already answered by others in the original discord thread.

Next up; some more in-depth thoughts.

[bigpod98]

One another point i made is that with starting point we just went back to the begining a bespoke files which is what treefiles are, which then went to an actual well used standard aka oci containerfiles and now we are going back to a bespoke files which work as an abstraction for containerfiles, and if we wanted to have a easy way to maintain and build an image a UI to manage containerfile would be better aka some sort of ublue image builder that at the end spits out a containerfile.

But the UI part should be in future probably after everything matures and so on

[xynydev]

I'll begin with a bit of history, just so we're all on the same page.
Back when ublue was just ublue-os/base, the same repo served as the one people forked to create custom images (similar to the purpose startingpoint serves today), and the one people rebased to when they wanted a slightly improved version of Silverblue (similar to the purpose the -main images serve today, but only for Silverblue). The repo included a post-login service that installed some Flatpaks for the user, with a Zenity window informing them of the progress. As my first contribution, I PRd in a script that gets the list of RPMs to install during the build and Flatpaks to install post-login from a file called recipe.yml. Sound familiar?

Shortly after, the ublue project was moving fast and the ublue-os/base repo was soon to be deprecated in favor of ublue-os/main. It was an overall positive change, empowering the project to build a ton of different images. The other purpose of ublue-os/base (serving as a "template" for custom image makers) was (IMO) lost in the transition, though. I used the lazy consensus to create startingpoint, and that sparked a bit of debate with core maintainers (sorry for (some of) that). But eventually I was able to convince enough people that the repo should be kept, and I started regularly maintaining it and updating it with features. The options to change the base image and image name were added to the recipe too, as I wanted to minimize the amount of files a dev would have to change to get started (Containerfile, build.yml and recipe.yml vs just recipe.yml). The goal was (and largely still is) to make this cool technology accessible to Linux nerds beyond those with pre-existing knowledge of containers, the OCI standard, GitHub Actions, and the like.

Before the rewrite, it seemed that people were extending the system in all sorts of incompatible ways, and it felt like there should be standard ways to build multiple images with a common core, and with the new modular system, extend the system with more functionality. The rewrite made me see more potential in the methodology used by startingpoint (and Vanilla's vib). The module-based approach would allow greater customization and community creation and sharing of new features. I spent time talking with some people with more ideas than technical knowledge, and some wished for a kind of "drag and drop" building block based system, where you could just pick and choose different system components to build "your own OS". I thought that with the modular system, a wide selection of base images, and some effort a similar UX could be achieved. We're closer to this than ever.

And it feels like the efforts put in have paid off. It's somewhat often I see someone being thrilled at how easy getting started with making custom images is. Some have especially pointed out the automatic repo setup tools I've made, which try to make the barrier to entry as low as possible, by simplifying the somewhat easy, but sometimes intimidating set-up process (setting up signing & such). These tools have coincidentally been the ones explicitly ruled out of scope for Ublue by core maintainers. It's not that these users don't want to learn; they often ask questions when presented with a problem they cannot solve by themselves, and dutifully listen to what we have to say. In my opinion, these tools make getting started with developing custom images easier, and thus enriches the space with developers. I just feel like the reasons presented for the exclusion has been them being out-of-scope, and that we don't want to make making custom images so easy.

I want to make making custom images so easy. I want to enrich this new native container space with good developer documentation and approachable tooling. That is how I would like to align the scope of the tooling I create. The abstraction of recipe.yml is there to be more approachable and making somewhat complicated features available to ordinary users. It might seem useless when used only for something as simple as rpm-ostree install. Then again, it's what enables someone who just wants a custom image with an obscure package layered only need to change a single configuration file. And down the line, it might enable a user to just click some buttons in a UI to build an image with just the right set of packages and other features.

---

In the Discord thread and in previous discussions, I've seen hesitancy regarding calling startingpoint & related things an official Universal Blue project or an official way to do things recommended by Ublue. This has been coming mostly from core maintainers. And I do kind of get that, this is a bit of an outlier among the Ublue projects, and it can obviously get annoying when you feel like some part of the community is trying to push on you a vision you don't really agree with.

I've experienced this disconnect too, manifesting in different ways. For example, the refusal to give the (currently bare-bones) web UI a subdomain or approve a scoped Netlify GitHub app to automate its deployment, due to not wanting to support it 'officially' (but still not wanting to push it out of the org). Another example would be some members of the community inquiring when the structure used by startingpoint would be adopted by official images, seeing the standard repository structure and configuration a preferable alternative to the bespoke repository structures, build scripts, and even flatpak managers/installers used by official images.

So, I've silently expressed these small frustrations in private conversations with community members who I've chatted with. Most recently, we discussed with @gmpinder, who has built the tool now known as BlueBuild, and announced about it to us in #156. I view that a compiler-based structure (compiling recipe.yml to Containerfile, like vib) would be a great future direction for the project, as it would allow us to solve some problems more efficiently (such as pulling in akmods, community modules from other repos than the default ones, creating layers, etc.). During the discussion, I mentioned that a "sibling" org (of ublue) might be something I would consider starting. The org could be called BlueBuild (a name I came up with) and it could slowly transition to serve all the use cases that ublue-os/startingpoint, ublue-os/bling, and the Making Your Own -docs currently serve here. (Of course, a faster transition would be possible, but might hurt downstreams.) This wouldn't be a fork of ublue, but ideally a peaceful split into differently scoped projects. All the custom image making related docs and projects I'd gladly consolidate in this other org, then.

I see another organization as the best way to realize the proposal by @noelmiller to turn this into an "unofficial project". Naturally, the tooling-oriented project could have its own website, and the uses of community platforms could be negotiated. Noel's other proposed idea could be also realized with this structuring, as a sibling org could have multiple, well-documented templates (or just documentation pages), where not much continued support is offered, but one could have a more minimal template and follow other practices than the "main offering".

I wouldn't be against Ublue making its own, more minimal "Making Your Own" guides after a hypothetical split, but I would fancy them all being available on the same site. "Forking Your Own" should probably stay at Ublue, though, as it's pretty specific to Ublue projects.

In a hypothetical restructuring like this, I would love to keep good relations, maybe have the projects point to each other in some places, keep communication channels open.

I've mentioned VanillaOS' efforts in this space above, and it's also important to consider that what they're doing with vib is very similar to what we're doing. Competition is good, though, and I think the two products could coexist in this space, or even collaborate.

---

So, that's a long one, huh? Please join this thread with your thoughts. You can present criticism, but you might not be able to change my opinions :3

[xynydev]

I find it kind of funny that this thread was started by pondering what problems users are struggling with, etc., and now it's a big amount of what devs think.

Anyway, this is my contribution for the weekend, I've got another thing I have to now get working on. Cheers!

[gmpinder]

I'm all in with maintaining BlueBuild! I think having good tooling for this will help us with attracting more users and allowing them to easily make changes to fit their needs. The idea of a UI for this is super interesting to me and something that I could see myself working on with @xynydev .

[plata]

Maybe this is a bit too general, but: Should startingpoint even be centered around uBlue or rather for random ostree based distributions?

From what I'm concerned, I've used startingpoint because it just was the easiest. Apart from that I would probably base on Kinoite directly.

[xynydev]

Should startingpoint even be centered around uBlue or rather for random ostree based distributions?

It doesn't need to be. It is currently so integrated with ublue that it is considered to, but I'd 100% just make different (& equal) docs pages for basing on Universal Blue, upstream atomic Fedora, VanillaOS, and whatever other distros support native containers in the future.

[fiftydinar]

Should startingpoint even be centered around uBlue or rather for random ostree based distributions?

It doesn't need to be. It is currently so integrated with ublue that it is considered to, but I'd 100% just make different (& equal) docs pages for basing on Universal Blue, upstream atomic Fedora, VanillaOS, and whatever other distros support native containers in the future.

I support this initiative.

Along with documentation, some rewrite of modules is needed to support images other than Universal Blue, but it is achievable.

[gmpinder]

So @xynydev what would be the first steps for this initiative?

[noelmiller]

Hello everyone!

Thank you so much for having a frank and open discussion regarding Starting Point.

I want to be clear that I only speak from my perspective with this response and not on behalf of Universal Blue as a whole.

To be blunt: I see a misalignment between Universal Blue's mission and the mission of Starting Point.

Mission of Universal Blue

In my view, Universal Blue is meant to be an extension of already well known standards for building custom images with a focus on "User First" design for our opinionated images (Bazzite or Bluefin). By utilizing these existing well known standards and design patterns, it allows us to build on the shoulders of giants and not re-invent the wheel as much as possible.

Ideal Starting Point

An ideal "Starting Point" to me is to utilize one of our base images or downstream images and understand the process of how they got there with well documented information or pointing to existing resources on the subject. Ideally, "Starting Point" should be a way to guide users into already well established patterns we have used (and agreed on) to create our downstream images from "Starting Point".

There should be no extra tooling that is not used by our core maintainers of downstream images. This allows us to be able to assist in troubleshooting more effectively since developers (or power users) of "Starting Point" are being encouraged and guided to use our existing tooling.

Where I see misalignment

• Starting Point as it currently exists is not the base that our existing images start from.
• Starting Point has extra tooling that can assist users with building their own images, but in my view, this tooling is not well supported by core maintainers of our downstream images. It is not being used to build those downstream images and there has been friction to adopt existing Starting Point as the common core.
• Starting Point contributors have the desire to keep the existing tooling that has been built and wants it to be a more general tool for building from any image (not just Universal Blue or Upstream Fedora)

Proposed Solutions

1. I think the idea of having this be a sister project to Universal Blue is a great idea. This allows Starting Point to have more freedom and still allows collaboration with Universal Blue due to us having similar aligned goals in many areas.
2. If above solution is decided on, I think Universal Blue should re-haul our existing documentation and support process around building custom images. We should focus on providing information about the processes and existing tooling we use to build our downstream images. I think it is important to explain that it is hard, but will allow new developers attracted by custom images to contribute to the common core we all depend on.

Summary

I want to express the gratitude I have to everyone for having an open and constructive conversation about this issue. The last thing I want to see is members having unnecessary friction when working together when it's a problem we can solve together as a community. I am very appreciative of everyone involved in Universal Blue (and the greater community)

If there is a split, I want it to be an amicable one everyone involved.

Cheers,

Noel Miller

[xynydev]

what would be the first steps for this initiative?

It seems that no-one has expressed an opinion against splitting off the custom image tooling portion of the project (which I'm assuming is what you mean by "This initiative"). I'd still like to see more comments, here, though. Maybe y'all can go ahead and give 👍 or 👎.

But, first of all, we'd have to gather round the people that would be interested in the new project, make some decisions together. Then we'd slowly work to move the repos (needs some high-level ppl in the ublue org present), add people to the new org, create new docs sites, etc. It'd take some effort, but it'll definetly be possible.

...And while typing this comment, Noel's response popped up. Cheers!

I agree with most of what you say. IMO the ideal startingpoint you describe is not called startingpoint and isn't a template repository. It's a thought-out and extensive guide to understanding how the processes Universal Blue builds on works. I think that that is a great initiative and look forward to seeing that. In my eyes, it's not something with the aim of making the creation of custom images as easy as possible, but something that helps it's readers understand and contribute to ublue, while educating them about things that they would benefit when forking.
It's exiciting! I've once even tried to make an infographic out of this process, but saw that it was too much information and complex processes to easily be packaged into a single picture. I could go so far as to say that I would gladly help build this side of the coin, too.