Bootstrapping Guide

[travankor]

Describe how to bootstrap from source and manage local repositories, etc.

cf.
https://github.com/the-maldridge/xbps-mini-builder
xbps-fbulk https://old.reddit.com/r/voidlinux/comments/gf6smw/what_is_the_difference_between_gentoo_and_void/fpvwhed/

[ericonr]

Hardware device support might be a more valid use-case that I forgot to mention -- There were plans some time ago to drop device-specific kernels, like usbarmory and kirkwood.

Unfortunately, 90% of ARM boards require custom vendor kernels (even if they're open-source, the vendors are apparently too lazy to upstream their changes).

God I hate vendors. Do you remember where this discussion happened? Last time kernels were mentioned on IRC the need for specific board kernels was very clear.

Regarding bootstrap: this void-linux/void-packages#21680 probably fits as well. I don't know if this is a guide that should be included here or in void-packages, though. Perhaps here, but it's not a priority, I think.

[fosslinux]

FWIW, I have been bootstrapping (starting from a busybox system) void-packages, so I would be more than happy to help with this.

I haven't been doing much automation, or using xbps-fbulk, yet, though.

[ericonr]

@fosslinux that would be awesome! I think we'd have to decide where to put said information first, of course.

[ericonr]

From IRC, as reference:

[22:27] <q66> since you need to rebuild everything at least 3 times in order to be usable at all
[22:27] <q66> i.e. like
[22:28] <q66> ./xbps-src bootstrap # create a masterdir from scratch, will also create some bare binpkgs in hostdir/binpkgs, those are useless
[22:28] <q66> rm -rf hostdir/binpkgs # drop those
[22:28] <q66> ./xbps-src pkg base-chroot # rebuild all
[22:28] <q66> rm -rf masterdir # drop initial masterdir
[22:28] <q66> ./xbps-src binary-bootstrap # new masterdir from newly built pkgs
[22:28] <q66> rm -rf hostdir/binpkgs # again
[22:28] <q66> ./xbps-src pkg base-chroot # again
[22:28] <q66> rm -rf masterdir # again
[22:28] <q66> ./xbps-src binary-bootstrap # now it should be ok

[travankor]

I think this could be split up into 3 parts:

1. 3-stage bootstrap with xbps-src

2. xbps-src automation tools/xbps-fbulk

3. local repository management (including xbps-rindex)

Basically sections 2+3 is similar to managing a source code install. Sections 1 is useful for verifying that the bootstrap tools work as intended and for obtaining a "from scratch" base result.

@ericonr

God I hate vendors. Do you remember where this discussion happened? Last time kernels were mentioned on IRC the need for specific board kernels was very clear.

Around a year ago (for the kirkland-kernel usbarmory-kernel). Admittedly, I haven't been following IRC very strictly lately. Regardless, both those kernels seem abandoned and lacking security patches for several years. I think custom kernel and custom images would fit best under an ARM section. For other arches, you can use the vanilla kernel and default dotconfig.

[ericonr]

Yeah, the usbarmory kernel is pretty bad here https://a-hel-fi.m.voidlinux.org/void-updates/void-updates.txt

Regarding the 3-stage boostrap, what do you think of https://github.com/void-linux/void-packages/blob/91d3a3cafff3ee6ee80ac0ca6a730bc505fd685c/README.md#install-bootstrap ?

I agree that mentioning automation tools somewhere is necessary, though. There's xbulk for a quick packaging, xbps-mini-builder for a more complicated setup (this one is already covered in restricted.md), and I've never actually used xbps-fbulk, though it seems to cover the same functionality that xbulk does (spoiler, it doesn't).

Usually xbps-rindex is run by the other tools, but it's useful if you don't want to depend on xdowngrade to install some random package you downloaded. Do you have other use cases?

[ericonr]

    maldridge │ xbulk, at least as far as I think you're looking at it, tries to get from one state to another
    maldridge │ fbulk builds everything
    maldridge │ or attempts it anyway
      ericonr │ including dependencies and everything?
    maldridge │ hence the "attempts"
    maldridge │ fbulk doesn't always get the ordering right
      ericonr │ would you say it basically runs xbps-src with -N ? so to speak
    maldridge │ I would not
    maldridge │ iirc it has a primitive solver and tries to construct a package graph
      ericonr │ so it sidesteps having to run with -N because it will attempt to build everything in the correct order
    maldridge │ I believe so

[the-maldridge]

I think there's 3 key parts here as @travankor pointed out above, but I disagree with the grouping.

There is the 3 stage bootstrap from nothing, which should be documented at length as that is what will be required if it were ever necessary to recover void from a loss of the mirrors.

From there, tools like fbulk and the mini builder are great for running your own repos. fbulk generally won't destroy your repos if you aren't trying to maintain the change rate we hold on the official repos. The mini builder is really singularly designed for managing a very small handful of packages that aren't officially built. Its possible to abuse it to build the whole tree, but I would not recommend it and won't support it.

The last case, repo management, is deserving of its own section, but probably not in the general docs. I'd prefer to document that in the context of InfraDocs, which should be written with the voice of managing the official repos, unstaging, and generally getting rindex to do the right thing when its convinced it won't.

[fosslinux]

I agree with @the-maldridge . I hadn't thought of the loss of the mirrors scenario, that is a good point.

RE: The newly added docs in Manual.md, they aren't very clear, IMO. I believe that the 3 stage bootstrap should be in a separate section, with numbered steps. I am happy to document that, since I have recently completed it. To me, the bootstrap section sounds very muddled (even if it is slightly clearer how to use ./xbps-src bootstrap.

Regarding running your own repos, I personally am not using anything crazy, other than fbulk and a script of my own wrapping ./xbps-src. It doesn't need to be fancy, IMO.

I believe that the repo management section, which also sounds like to me as it should be in InfraDocs, should include documentation on BuildBot as well, as that seems to be the void recognized way of building packages for the official repo.

Also, texinfo is not required for the bootstrap.