Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement architecture separation #310

Merged
merged 24 commits into from
Apr 3, 2021
Merged

Conversation

matteodelabre
Copy link
Member

@matteodelabre matteodelabre commented Mar 11, 2021

This PR improves Toltec’s support for reMarkable 2 in a few ways:

  • Prevents users from installing packages that are known to be incompatible with their device (fixes feature: prevent installation of rM2 incompatible apps on rM2 #129).
  • Automatically installs and sets up the rm2fb server when an app that uses the framebuffer is installed.
  • Stops relying on LD_PRELOAD to load the rm2fb shim into apps that need it. Instead, change their dynamic dependencies to include the shim directly. Thanks to this change, we no longer need to set LD_PRELOAD globally (fixes rm2fb: disable LD_PRELOAD for busybox services #248) and users can launch apps that need the shim without prefixing them with rm2fb-client.

It replaces the previous PR #136.

Tooling changes

More details in the updated docs/package.md.

  • Added the archs field for declaring which device(s) a recipe is compatible with. Supported values are rmall (for device-independent packages), rm1 (for reMarkable 1) and rm2 (for reMarkable 2). When multiple arch values are set for a recipe, the recipe is generated once per entry, which allows specializing builds for specific devices.
  • Replaced the depends field with an installdepends field used for declaring install-time dependencies. Previously, dependencies declared in the depends field were fulfilled both at build time and at install time, with no way of declaring install-time-only dependencies.
  • Added a replaces field that sets the Replaces value in the generated package. Used for migrating users from the rm2fb package to the display and rm2fb-client packages (see below).
  • Added a patch_rm2fb flag for recipes to instruct the build system to patch all the generated binaries that use the rM1 framebuffer with the rm2fb client shim.
  • Changed the web package listing to include download links for each available architecture of each package.
  • Packages are now stored in subdirectories named after their architecture. This is because opkg does not like seeing packages for architectures other than the ones it is configured to accept in opkg.conf. With this layout change, we can make opkg happy by only adding subdirectories for enabled architectures.

Install changes

  • The bootstrap script now adds the rmall architecture (and folder) on all installs, and the rm1 or rm2 architecture (and folder) depending on the type of the device it’s executed on. It will fail if the device is not of a recognized type (based on the value of /sys/devices/soc0/machine).
  • Running the new bootstrap script on an existing install will upgrade the configuration to use the new architecture subdirectories layout.

rm2fb changes

  • The package is split into two packages, display (which provides the rm2fb server on rM2 and is an empty package on rM1) and rm2fb-client (which provides the client shim on rM2 and a mock shim on rM1).
  • The rm2fb-server command is removed (the most common use-case being that the display server is started from the systemd unit).

Apps changes

Apps marked as reMarkable-1-only:

Apps marked as reMarkable-2-only:

  • rm2-suspend-fix

Apps updated to include the rM2 shim:

  • appmarkable
  • calculator
  • chessmarkable
  • draft
  • fingerterm
  • keywriter
  • koreader (through setting LD_PRELOAD in the existing wrapper script)
  • oxide
  • plato
  • recrossable
  • remarkable-splash
  • retris
  • rmkit
  • wikipedia
  • xochitl (through a new wrapper script at /opt/bin/xochitl)
  • zshelf

Test plan

On rM1 and rM2 (system version: 2.5).

  • Starting from a clean install of Toltec stable or testing, download the updated bootstrap script and run:
$ toltec_branch=dev/split-architectures bash bootstrap
  • This should update the /opt/etc/opkg.conf configuration file to enable the rmall architecture and, depending on the device you’re running this on, the rm1 or the rm2 architecture.
  • It should also replace the src/gz toltec line with two source lines, src/gz toltec-rmall and src/gz toltec-rm1 or src/gz toltec-rm2, pointing to subfolders of https://toltec-dev.org/dev/split-architectures.
  • Run opkg upgrade. Most installed packages should get upgraded, and if you had installed any packages that use the screen, the display and rm2fb-client packages should get installed. If you previously installed the rm2fb package, it should get removed automatically.
  • Try installing rm2-suspend-fix on a reMarkable 1, or vnsee on a reMarkable 2: both operations should fail.
  • Test all updated packages to make sure they can properly draw to the screen (both on rM1 and rM2) and that you can interact with them through the touchscreen and the pen, when applicable. It is no longer needed to prefix each app invocation with rm2fb-client on the command line on reMarkable 2.
  • In particular, make sure that apps work properly when launched from draft, oxide or remux (things to check: launching, pausing, killing apps, switching back and forth from xochitl).

Rollout plan

When this PR hits testing and stable, existing users will no longer receive updates until they run the new bootstrap script on their existing installs. They will also probably see an error when they do opkg update. We should decide whether we accept this or whether we want to make the transition easier, for example by leveraging the toltec-bootstrap package.

This PR improves Toltec’s support for reMarkable 2 in a few ways:

* Prevents users from installing packages that are known to be
  incompatible with their device (fixes #129).
* Automatically installs and sets up the rm2fb server when an app that
  uses the framebuffer is installed.
* Stops relying on LD_PRELOAD to load the rm2fb shim into apps that need
  it. Instead, change their dynamic dependencies to include the shim
  directly. Thanks to this change, we no longer need to set LD_PRELOAD
  globally (fixes #248) and users can launch apps that need the shim
  without prefixing them with `rm2fb-client`.

Tooling changes (more details in the updated docs/package.md):

* Added the `archs` field for declaring which device(s) a recipe is
  compatible with. Supported values are `rmall` (for device-independent
  packages), `rm1` (for reMarkable 1) and `rm2` (for reMarkable 2).
  When multiple arch values are set for a recipe, the recipe is
  generated once per entry, which allows specializing builds for
  specific devices.
* Replaced the `depends` field with an `installdepends` field used for
  declaring install-time dependencies. Previously, dependencies declared
  in the `depends` field were fulfilled both at build time and at
  install time, with no way of declaring install-time-only dependencies.
* Added a `replaces` field that sets the `Replaces` value in the
  generated package. Used for migrating users from the `rm2fb` package
  to the `display` and `rm2fb-shim` packages (see below).
* Changed [the web package listing](https://toltec-dev.org/dev/split-architectures)
  to include download links for each available architecture of each
  package.
* Packages are now stored in subdirectories named after their
  architecture. This is because opkg does not like seeing packages for
  architectures other than the ones it is configured to accept in
  opkg.conf. With this layout change, we can make opkg happy by
  only adding subdirectories for enabled architectures.

Install changes:

* The bootstrap script now adds the `rmall` architecture (and folder) on
  all installs, and the `rm1` or `rm2` architecture (and folder)
  depending on the type of the device it’s executed on. It will fail if
  the device is not of a recognized type (based on the value of
  `/sys/devices/soc0/machine`).
* Running the new bootstrap script on an existing install will upgrade
  the configuration to use the new architecture subdirectories layout.

rm2fb changes:

* The package is split into two packages, `display` (which provides the
  rm2fb server on rM2 and is an empty package on rM1) and `rm2fb-shim`
  (which provides the client shim on rM2 and a mock shim on rM1).
* The `rm2fb-server` command is removed (the most common use-case being
  that the display server is started from the systemd unit).
* The `rm2fb-client` command is renamed to `rm2fb-shim`, with the
  expectation that users will not need it anymore, unless for
  development purposes.

Apps marked as reMarkable-1-only:

* appmarkable (LinusCDE/appmarkable#2)
* rmservewacominput (because it depends on appmarkable)
* vnsee (matteodelabre/vnsee#15)
* whiteboard-hypercard (fenollp/reMarkable-tools#12)

Apps marked as reMarkable-2-only:

* rm2-suspend-fix

Apps updated to include the rM2 shim:

* calculator
* chessmarkable
* draft
* fingerterm
* keywriter
* koreader (through setting LD_PRELOAD in the existing wrapper script)
* oxide
* plato
* recrossable
* remarkable-splash
* retris
* rmkit
* xochitl (through a new wrapper script at /opt/bin/xochitl)
* zshelf

Test plan (on rM1 and rM2):

* Starting from a clean stable or testing install of Toltec, download
  the updated bootstrap script and run:

```console
$ toltec_branch=dev/split-architectures bash bootstrap
```

* This should update the `/opt/etc/opkg.conf` configuration file to
  enable the `rmall` architecture and, depending on the device you’re
  running this on, the `rm1` or the `rm2` architecture.
* It should also replace the `src/gz toltec` line with two source lines,
  `src/gz toltec-rmall` and `src/gz toltec-rm1` or `src/gz toltec-rm2`,
  pointing to subfolders of
  `https://toltec-dev.org/dev/split-architectures`.
* Run `opkg upgrade`. Most installed packages should get upgraded, and
  if you had installed any packages that use the screen, the `display`
  and `rm2fb-shim` packages should get installed. If you previously
  installed the `rm2fb` package, it should get removed automatically.
* Try installing `rm2-suspend-fix` on a reMarkable 1, or `vnsee` on a
  reMarkable 2: both operations should fail.
* Test all updated packages to make sure they can properly draw to the
  screen (both on rM1 and rM2) and that you can interact with them
  through the touchscreen and the pen, when applicable.
* In particular, make sure that apps work properly when launched from
  draft, oxide or remux (some issues remain with oxide and remux).
@matteodelabre
Copy link
Member Author

matteodelabre commented Mar 11, 2021

I made a few design choices on this PR that I realize may not suit everyone. Comments on the design or the implementation are very welcome. Some issues remain with the Oxide and remux launchers:

  • On rM2, the display glitches when launching apps from Oxide, even though it’s using the rm2fb shim. I haven’t found the source of this issue yet. Fixed — see below.
  • remux does not use the xochitl wrapper script since it’s using the path from /lib/systemd/system/xochitl.service. A solution to this is being discussed. Fixed in the latest rmkit release.

@Eeems
Copy link
Member

Eeems commented Mar 11, 2021

* On rM2, the display glitches when launching apps from Oxide, even though it’s using the rm2fb shim. I haven’t found the source of this issue yet.

Have you changed anything with the Oxide package?

@matteodelabre
Copy link
Member Author

Yes. I changed the package to build from source and to patch binaries that I believe use the framebuffer to add a dynamic dependency on librm2fb_client.so.

@Eeems
Copy link
Member

Eeems commented Mar 11, 2021

Yes. I changed the package to build from source and to patch binaries that I believe use the framebuffer to add a dynamic dependency on librm2fb_client.so.

Right, we stopped building from source a while back due to issues with the current toltec toolchain. I expect that might be the issue still.

As for which binaries use the framebuffer, all but fret and rot at this point I believe.

@matteodelabre
Copy link
Member Author

Right, we stopped building from source a while back due to issues with the current toltec toolchain. I expect that might be the issue still.

Indeed. I’ll check whether using the pre-built binaries fixes the issues I’m seeing.

As for which binaries use the framebuffer, all but fret and rot at this point I believe.

So tarnish would need to have the shim too, which it doesn’t in the current state of the PR. That might also be the cause of the issue.

@Eeems
Copy link
Member

Eeems commented Mar 11, 2021

So tarnish would need to have the shim too, which it doesn’t in the current state of the PR. That might also be the cause of the issue.

This like is why the screen messes up. When you launch an application, the current screen is saved. Resuming an application will redraw that saved screen.

@matteodelabre
Copy link
Member Author

This like is why the screen messes up. When you launch an application, the current screen is saved. Resuming an application will redraw that saved screen.

That was it! Thanks for the insights.

@danshick
Copy link
Member

Quick question, for packages like reStream that automatically support rM1, rM2 stock, and rM2 with rm2fb, would there be a way to write an rM2 recipe that only optionally depends on rm2fb? Or would we just leave the display dependency off and if it happens to be installed, make use of it anyway?

@Eeems
Copy link
Member

Eeems commented Mar 11, 2021

I think that depends on how they've implemented their screen support.

@danshick
Copy link
Member

One of three ways. rM1 reads directly from /dev/fb0. rM2 without rm2fb finds the xochitl process and directly hooks its memory. rM2 with rm2fb uses /dev/shm/swtfb.01.

@raisjn
Copy link
Contributor

raisjn commented Mar 12, 2021

this looks great! i've read through all the recipes and am making my way through the build scripts. in the package recipes, it strikes me as repetitive that everyone needs to know to use patchelf in their package, is there a way to simplify it / reduce the burden for package writers?

@matteodelabre matteodelabre added packages Add or improve packages of the repository tooling Set of scripts and configuration files for building the packages install Installation scripts labels Mar 12, 2021
@matteodelabre
Copy link
Member Author

One of three ways. rM1 reads directly from /dev/fb0. rM2 without rm2fb finds the xochitl process and directly hooks its memory. rM2 with rm2fb uses /dev/shm/swtfb.01.

Unless there is a specific advantage in using reStream with rm2fb, I’d say that we should leave out the dependency.

@matteodelabre
Copy link
Member Author

in the package recipes, it strikes me as repetitive that everyone needs to know to use patchelf in their package, is there a way to simplify it / reduce the burden for package writers?

Good point. Unfortunately there isn’t a mechanism in place to load a library of pre-defined Bash functions into recipes. I’m a bit reluctant to add such a mechanism in this PR since it’s already quite complex. Do you think it’s worth implementing this right now?

If we had a function that automated this, it would need to do three things:

  1. Add the build:patchelf dependency to the makedepends array
  2. Set the nostrip option
  3. Run patchelf and strip with appropriate arguments

Steps 1. and 2. need to modify variables at the recipe scope and would need to be called outside of the build function. Step 3. needs to access the build artifacts and would need to be called inside the build function. Therefore we would probably need to define two helper functions, one to do 1. and 2., and one to do 3.

Copy link
Contributor

@raisjn raisjn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall the source looks good. the complexity is definitely on my mind

scripts/toltec/builder.py Outdated Show resolved Hide resolved
@raisjn
Copy link
Contributor

raisjn commented Mar 21, 2021

Good point. Unfortunately there isn’t a mechanism in place to load a library of pre-defined Bash functions into recipes. I’m a bit reluctant to add such a mechanism in this PR since it’s already quite complex. Do you think it’s worth implementing this right now?

my thought here is that we know people copy code. there will be people copying this code and not copying it and we will both have to go fix it later and constantly explain this patchelf stuff to people contributing a package (who forgot to copy the code or apply it correctly). past experiences have shown me this is the case and so i prefer not to create a messy situation if we know we will have to clean it up anyways.

my hope is that we figure out if a binary uses the framebuffer (check for fb0? check for some symbols?) and automatically do all the strip+patchelf stuff. every package will implicitly rely on patchelf.

@matteodelabre
Copy link
Member Author

matteodelabre commented Mar 21, 2021

my hope is that we figure out if a binary uses the framebuffer (check for fb0? check for some symbols?) and automatically do all the strip+patchelf stuff. every package will implicitly rely on patchelf.

I’m all for automating this as much as possible. Deciding whether a binary needs the shim is tricky though: it’s not only a matter of checking whether it contains a call to open("/dev/fb0"), because it may open the framebuffer when run on rM1 and talk to rm2fb on rM2 (such apps will not need the shim, see for example reStream and VNSee).

* Remove unused GenericRecipe.variables attribute
* Make Package.variables extend the parent Recipe.variables
@matteodelabre
Copy link
Member Author

Thanks @raisjn & @LinusCDE for your valuable input on the support status. I agree these are all things we should double check before announcing rM2 support. Since this is already a large PR with a fair amount of discussion, I simply reverted the badge change, and we shall discuss this further in another thread.

raisjn
raisjn previously approved these changes Apr 2, 2021
@matteodelabre
Copy link
Member Author

matteodelabre commented Apr 3, 2021

I re-did some tests and there seems to be a bug with remux and KOReader. When I tap on KOReader in the list of apps, two instances are launched (can be seen in [h]top). KOReader in itself works fine but does not appear as launched in the remux menu. Tapping again on KOReader in remux creates even more instances of the app.

This bug does not exist in testing, so this is a regression from this PR.

# SPDX-License-Identifier: MIT

KO_DONT_GRAB_INPUT=1 bash /opt/koreader/koreader.sh
LD_PRELOAD=/opt/lib/librm2fb_client.so.1 KO_DONT_GRAB_INPUT=1 exec bash /opt/koreader/koreader.sh
Copy link
Contributor

@raisjn raisjn Apr 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this exec is confusing remux in regards to koreader. I think it replaces the current process and now remux can't find /opt/bin/koreader - normally remux uses the launching binary to find if the process exists.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for investigating that! Yes, exec replaces the parent process. It makes sense to use it here because we don’t need the parent Bash process to linger after it has launched KOReader. I think we can use the (Bash-specific) -a flag to change the name of the replacing process and thereby avoid confusing remux. I’ll try it out.

@matteodelabre
Copy link
Member Author

This change seems to fix the issue. Thanks for your help @raisjn.

@matteodelabre matteodelabre merged commit 01e213e into testing Apr 3, 2021
@matteodelabre matteodelabre deleted the tooling/split-architectures branch April 3, 2021 16:03
matteodelabre added a commit that referenced this pull request Apr 4, 2021
In PR #310, a regression was introduced that makes `make repo` fail.
This slipped through the testing because we only did full repo builds
(i.e., `make repo-local`). This regression makes CI builds fail, as they
only rebuilt packages that have changed (for example see #328).

The failure comes from `Repo.fetch_package`: when fetching an existing
packages from the remote server, the method should first make sure that
the parent architecture directory (e.g. `build/repo/rmall`) exists.

To test, you only need to run `make clean && make repo` and observe that
the fetching process completes successfully.
matteodelabre added a commit that referenced this pull request Apr 4, 2021
In PR #310, a regression was introduced that makes `make repo` fail.
This slipped through the testing because we only did full repo builds
(i.e., `make repo-local`). This regression makes CI builds fail, as they
only rebuilt packages that have changed (for example see #328).

The failure comes from `Repo.fetch_package`: when fetching an existing
packages from the remote server, the method should first make sure that
the parent architecture directory (e.g. `build/repo/rmall`) exists.

To test, you only need to run `make clean && make repo` and observe that
the fetching process completes successfully.
danshick pushed a commit to danshick/toltec that referenced this pull request May 5, 2021
This PR improves Toltec’s support for reMarkable 2 in a few ways:

* Prevents users from installing packages that are known to be incompatible with their device (fixes toltec-dev#129).
* Automatically installs and sets up the rm2fb server when an app that uses the framebuffer is installed.
* Stops relying on LD_PRELOAD to load the rm2fb shim into apps that need it. Instead, change their dynamic dependencies to include the shim directly. Thanks to this change, we no longer need to set LD_PRELOAD globally (fixes toltec-dev#248) and users can launch apps that need the shim without prefixing them with `rm2fb-client`.

It replaces the previous PR toltec-dev#136.

#### Tooling changes

More details in the updated [docs/package.md](https://github.com/toltec-dev/toltec/blob/tooling/split-architectures/docs/package.md).

* Added the `archs` field for declaring which device(s) a recipe is compatible with. Supported values are `rmall` (for device-independent packages), `rm1` (for reMarkable 1) and `rm2` (for reMarkable 2). When multiple arch values are set for a recipe, the recipe is generated once per entry, which allows specializing builds for specific devices.
* Replaced the `depends` field with an `installdepends` field used for declaring install-time dependencies. Previously, dependencies declared in the `depends` field were fulfilled both at build time and at install time, with no way of declaring install-time-only dependencies.
* Added a `replaces` field that sets the `Replaces` value in the generated package. Used for migrating users from the `rm2fb` package to the `display` and `rm2fb-client` packages (see below).
* Added a `patch_rm2fb` flag for recipes to instruct the build system to patch all the generated binaries that use the rM1 framebuffer with the rm2fb client shim.
* Changed [the web package listing](https://toltec-dev.org/dev/split-architectures) to include download links for each available architecture of each package.
* Packages are now stored in subdirectories named after their architecture. This is because opkg does not like seeing packages for architectures other than the ones it is configured to accept in opkg.conf. With this layout change, we can make opkg happy by only adding subdirectories for enabled architectures.

#### Install changes

* The bootstrap script now adds the `rmall` architecture (and folder) on all installs, and the `rm1` or `rm2` architecture (and folder) depending on the type of the device it’s executed on. It will fail if the device is not of a recognized type (based on the value of `/sys/devices/soc0/machine`).
* Running the new bootstrap script on an existing install will upgrade the configuration to use the new architecture subdirectories layout.

#### rm2fb changes

* The package is split into two packages, `display` (which provides the rm2fb server on rM2 and is an empty package on rM1) and `rm2fb-client` (which provides the client shim on rM2 and a mock shim on rM1).
* The `rm2fb-server` command is removed (the most common use-case being that the display server is started from the systemd unit).

#### Apps changes

Apps marked as reMarkable-1-only:

* vnsee (matteodelabre/vnsee#15)
* whiteboard-hypercard (fenollp/reMarkable-tools#12)

Apps marked as reMarkable-2-only:

* rm2-suspend-fix

Apps updated to include the rM2 shim:

* appmarkable
* calculator
* chessmarkable
* draft
* fingerterm
* keywriter
* koreader (through setting LD_PRELOAD in the existing wrapper script)
* oxide
* plato
* recrossable
* remarkable-splash
* retris
* rmkit
* wikipedia
* xochitl (through a new wrapper script at /opt/bin/xochitl)
* zshelf

#### Test plan

On rM1 and rM2 (system version: 2.5).

* Starting from a clean install of Toltec stable or testing, download the updated bootstrap script and run:

```console
$ toltec_branch=dev/split-architectures bash bootstrap
```

* This should update the `/opt/etc/opkg.conf` configuration file to enable the `rmall` architecture and, depending on the device you’re running this on, the `rm1` or the `rm2` architecture.
* It should also replace the `src/gz toltec` line with two source lines, `src/gz toltec-rmall` and `src/gz toltec-rm1` or `src/gz toltec-rm2`, pointing to subfolders of `https://toltec-dev.org/dev/split-architectures`.
* Run `opkg upgrade`. Most installed packages should get upgraded, and if you had installed any packages that use the screen, the `display` and `rm2fb-client` packages should get installed. If you previously installed the `rm2fb` package, it should get removed automatically.
* Try installing `rm2-suspend-fix` on a reMarkable 1, or `vnsee` on a reMarkable 2: both operations should fail.
* Test all updated packages to make sure they can properly draw to the screen (both on rM1 and rM2) and that you can interact with them through the touchscreen and the pen, when applicable. It is no longer needed to prefix each app invocation with `rm2fb-client` on the command line on reMarkable 2.
* In particular, make sure that apps work properly when launched from draft, oxide or remux (things to check: launching, pausing, killing apps, switching back and forth from xochitl).

#### Rollout plan

When this PR hits testing and stable, existing users will no longer receive updates until they run the new bootstrap script on their existing installs. They will also probably see an error when they do `opkg update`. Figuring out how to automatic transition users is left to a future PR.

Co-authored-by: okay <okay@arkose>
danshick pushed a commit to danshick/toltec that referenced this pull request May 5, 2021
In PR toltec-dev#310, a regression was introduced that makes `make repo` fail.
This slipped through the testing because we only did full repo builds
(i.e., `make repo-local`). This regression makes CI builds fail, as they
only rebuilt packages that have changed (for example see toltec-dev#328).

The failure comes from `Repo.fetch_package`: when fetching an existing
packages from the remote server, the method should first make sure that
the parent architecture directory (e.g. `build/repo/rmall`) exists.

To test, you only need to run `make clean && make repo` and observe that
the fetching process completes successfully.
danshick added a commit to danshick/toltec that referenced this pull request May 5, 2021
LinusCDE pushed a commit that referenced this pull request Jun 3, 2021
This PR improves Toltec’s support for reMarkable 2 in a few ways:

* Prevents users from installing packages that are known to be incompatible with their device (fixes #129).
* Automatically installs and sets up the rm2fb server when an app that uses the framebuffer is installed.
* Stops relying on LD_PRELOAD to load the rm2fb shim into apps that need it. Instead, change their dynamic dependencies to include the shim directly. Thanks to this change, we no longer need to set LD_PRELOAD globally (fixes #248) and users can launch apps that need the shim without prefixing them with `rm2fb-client`.

It replaces the previous PR #136.

#### Tooling changes

More details in the updated [docs/package.md](https://github.com/toltec-dev/toltec/blob/tooling/split-architectures/docs/package.md).

* Added the `archs` field for declaring which device(s) a recipe is compatible with. Supported values are `rmall` (for device-independent packages), `rm1` (for reMarkable 1) and `rm2` (for reMarkable 2). When multiple arch values are set for a recipe, the recipe is generated once per entry, which allows specializing builds for specific devices.
* Replaced the `depends` field with an `installdepends` field used for declaring install-time dependencies. Previously, dependencies declared in the `depends` field were fulfilled both at build time and at install time, with no way of declaring install-time-only dependencies.
* Added a `replaces` field that sets the `Replaces` value in the generated package. Used for migrating users from the `rm2fb` package to the `display` and `rm2fb-client` packages (see below).
* Added a `patch_rm2fb` flag for recipes to instruct the build system to patch all the generated binaries that use the rM1 framebuffer with the rm2fb client shim.
* Changed [the web package listing](https://toltec-dev.org/dev/split-architectures) to include download links for each available architecture of each package.
* Packages are now stored in subdirectories named after their architecture. This is because opkg does not like seeing packages for architectures other than the ones it is configured to accept in opkg.conf. With this layout change, we can make opkg happy by only adding subdirectories for enabled architectures.

#### Install changes

* The bootstrap script now adds the `rmall` architecture (and folder) on all installs, and the `rm1` or `rm2` architecture (and folder) depending on the type of the device it’s executed on. It will fail if the device is not of a recognized type (based on the value of `/sys/devices/soc0/machine`).
* Running the new bootstrap script on an existing install will upgrade the configuration to use the new architecture subdirectories layout.

#### rm2fb changes

* The package is split into two packages, `display` (which provides the rm2fb server on rM2 and is an empty package on rM1) and `rm2fb-client` (which provides the client shim on rM2 and a mock shim on rM1).
* The `rm2fb-server` command is removed (the most common use-case being that the display server is started from the systemd unit).

#### Apps changes

Apps marked as reMarkable-1-only:

* vnsee (matteodelabre/vnsee#15)
* whiteboard-hypercard (fenollp/reMarkable-tools#12)

Apps marked as reMarkable-2-only:

* rm2-suspend-fix

Apps updated to include the rM2 shim:

* appmarkable
* calculator
* chessmarkable
* draft
* fingerterm
* keywriter
* koreader (through setting LD_PRELOAD in the existing wrapper script)
* oxide
* plato
* recrossable
* remarkable-splash
* retris
* rmkit
* wikipedia
* xochitl (through a new wrapper script at /opt/bin/xochitl)
* zshelf

#### Test plan

On rM1 and rM2 (system version: 2.5).

* Starting from a clean install of Toltec stable or testing, download the updated bootstrap script and run:

```console
$ toltec_branch=dev/split-architectures bash bootstrap
```

* This should update the `/opt/etc/opkg.conf` configuration file to enable the `rmall` architecture and, depending on the device you’re running this on, the `rm1` or the `rm2` architecture.
* It should also replace the `src/gz toltec` line with two source lines, `src/gz toltec-rmall` and `src/gz toltec-rm1` or `src/gz toltec-rm2`, pointing to subfolders of `https://toltec-dev.org/dev/split-architectures`.
* Run `opkg upgrade`. Most installed packages should get upgraded, and if you had installed any packages that use the screen, the `display` and `rm2fb-client` packages should get installed. If you previously installed the `rm2fb` package, it should get removed automatically.
* Try installing `rm2-suspend-fix` on a reMarkable 1, or `vnsee` on a reMarkable 2: both operations should fail.
* Test all updated packages to make sure they can properly draw to the screen (both on rM1 and rM2) and that you can interact with them through the touchscreen and the pen, when applicable. It is no longer needed to prefix each app invocation with `rm2fb-client` on the command line on reMarkable 2.
* In particular, make sure that apps work properly when launched from draft, oxide or remux (things to check: launching, pausing, killing apps, switching back and forth from xochitl).

#### Rollout plan

When this PR hits testing and stable, existing users will no longer receive updates until they run the new bootstrap script on their existing installs. They will also probably see an error when they do `opkg update`. Figuring out how to automatic transition users is left to a future PR.

Co-authored-by: okay <okay@arkose>
LinusCDE pushed a commit that referenced this pull request Jun 3, 2021
In PR #310, a regression was introduced that makes `make repo` fail.
This slipped through the testing because we only did full repo builds
(i.e., `make repo-local`). This regression makes CI builds fail, as they
only rebuilt packages that have changed (for example see #328).

The failure comes from `Repo.fetch_package`: when fetching an existing
packages from the remote server, the method should first make sure that
the parent architecture directory (e.g. `build/repo/rmall`) exists.

To test, you only need to run `make clean && make repo` and observe that
the fetching process completes successfully.
matteodelabre added a commit that referenced this pull request Jul 21, 2021
Important notices:

* When upgrading, users should run `opkg update && opkg upgrade` **twice** because of the changes introduced by this merge. (#310)
* Toltec **no longer supports** OS releases **earlier than 2.6**. You can check your current OS version in the settings panel. Please upgrade to 2.6 or newer after upgrading your Toltec install. (#366)
* To re-enable Toltec, **run `toltecctl reenable`** instead of the previous `entware-reenable` script that has been removed with this update.

Major change: The repository is now split into architecture subfolders. The `rmall` subfolder contains packages that can run both on reMarkable 1 and reMarkable 2 without modification. The `rm1` and `rm2` subfolders contain packages that are specific to each of the two devices. This change requires updating the Opkg configuration to point to the appropriate architectures. The migration is automated by placing an upgrade to the `toltec-bootstrap` package in the root folder of the repo that contains a migration script. (Note that this package is not generated automatically by our build system but will be added manually by the server administrator.)

New packages:

* bash-completion - 2.11-3 (#277, #375)
* display - 1.0.1-7 _(replaces the server part of the old rm2fb package)_ (#310)
* fuse - 1.0.0-2 (#331)
* gocryptfs - 2.0-beta2 (#318)
* innernet - 1.3.1-2 (#350, #358, #361)
* netsurf - 0.4.0-1 (#339)
* puzzles - 0.2.2-2 (#326)
* remarkable-stylus - 0.0.3-1 (#383)
* rm2fb-client - 1.0.1-7 _(replaces the client part of the old rm2fb package)_ (#310)
* tilem - 0.0.7-1 (#352)
* toltec-completion - 0.2.0-1 (#277)
* yaft - 0.0.4-3 (#261, #353)
* zoneinfo-utils - 2021a-2 (#182)

Updated packages:

* appmarkable - 0.0.0-11
* bufshot - 0.1.0-5
* calculator - 0.0.0-15
* chessmarkable - 0.7.1-2 (#362, #373)
* ddvk-hacks - 23.02-1 (#385)
* draft - 0.2.0-20
* evtest - 1.34-3
* fbink fbdepth fbink-doom - 1.23.1-4
* fingerterm - 1.3.5-14
* genie - 0.1.5-3
* harmony - 0.1.3-3
* iago - 0.1.0-4
* keywriter - 0.1.0-3
* koreader - 2021.06-1 (#330, #348, #357, #378)
* lamp - 0.1.0-4
* libdlib libdlib-dev - 19.21-2
* libvncserver libvncclient libvncserver-dev - 0.9.13-2
* mines - 0.1.2-4
* nao - 0.1.3-3
* oxide erode fret rot tarnish decay corrupt anxiety - 2.2-1 (#382)
* plato - 0.9.17-2 (#344)
* quickjs - 2020.11.08-2
* recrossable - 0.0.0-7
* remarkable-splash - 1.0-5
* remux - 0.1.9-3
* restream - 1.1-2
* retris - 0.6.3-3
* rmservewacominput - 0.3.0-2
* simple - 0.1.4-3
* templatectl - 0.1.3-3
* toltec-bootstrap - 0.1.0-1 _(includes toltecctl)_ (#379)
* vnsee - 0.4.0-2 (#355)
* wikipedia - 0.1.0-3
* wireguard - 1.0.20210219-2
* xochitl - 0.0.0-7 (#383, #382, #386)
* zshelf - 0.3.1-3

Website: The source files and scripts used for generating the website have been transferred to a separate repository (<https://github.com/toltec-dev/web>). A website rebuild is triggered automatically from this repository’s workflow when the stable branch is updated, so that the displayed checksum of the bootstrap script is always accurate.
matteodelabre added a commit that referenced this pull request Jul 31, 2021
Important notices:

* When upgrading, users should run `opkg update && opkg upgrade` **twice** because of the changes introduced by this merge. (#310)
* Toltec **no longer supports** OS releases **earlier than 2.6**. You can check your current OS version in the settings panel. If you’re still running on 2.5 or earlier, please upgrade to 2.6 or newer after upgrading your Toltec install. (#366)

Less important notices:

* To re-enable Toltec after a system upgrade, **run `toltecctl reenable`** instead of the previous `entware-reenable` script that has been removed with this update.
* You can now easily uninstall Toltec by running `toltec uninstall`.
* This release marks the end of our work towards transitioning to a new repository layout. This means that **stable will be updated more frequently from now on.** If you switched to the testing branch because stable has been outdated for a few months, and if you prefer to run stable releases, you can go back to the stable branch by running `toltecctl switch-branch stable`.

Layout change: The repository is now split into architecture subfolders. The `rmall` subfolder contains packages that can run both on reMarkable 1 and reMarkable 2 without modification. The `rm1` and `rm2` subfolders contain packages that are specific to each of the two devices. This change requires updating the Opkg configuration to point to the appropriate architectures. The migration is automated by placing an upgrade to the `toltec-bootstrap` package in the root folder of the repo that contains a migration script.

New packages:

* bash-completion - 2.11-3 (#277, #375)
* display - 1.0.1-7 _(replaces the server part of the old rm2fb package)_ (#310)
* fuse - 1.0.0-3 (#331)
* gocryptfs - 2.0-beta2 (#318)
* innernet-client - 1.3.1-2 (#350, #358, #361)
* netsurf - 0.4.0-2 (#339)
* puzzles - 0.2.2-3 (#326)
* remarkable-stylus - 0.0.3-1 (#383)
* rm2fb-client - 1.0.1-7 _(replaces the client part of the old rm2fb package)_ (#310)
* tilem - 0.0.7-2 (#352)
* toltec-completion - 0.2.0-1 (#277)
* yaft - 0.0.4-4 (#261, #353)
* zoneinfo-utils - 2021a-2 (#182)

Updated packages:

* appmarkable - 0.0.0-11
* bufshot - 0.1.0-5
* calculator - 0.0.0-15
* chessmarkable - 0.8.0-1 (#362, #373, #392)
* ddvk-hacks - 23.02-1 (#385)
* draft - 0.2.0-20
* evtest - 1.34-3
* fbink fbdepth fbink-doom - 1.23.2-1 (#387)
* fingerterm - 1.3.5-14
* genie - 0.1.5-3
* harmony - 0.1.3-3
* iago - 0.1.0-4
* keywriter - 0.1.0-3
* koreader - 2021.07-1 (#330, #348, #357, #378, #389)
* lamp - 0.1.0-4
* libdlib libdlib-dev - 19.21-2
* libvncserver libvncclient libvncserver-dev - 0.9.13-2
* mines - 0.1.2-4
* nao - 0.1.3-3
* oxide erode fret rot tarnish decay corrupt anxiety - 2.2.1-1 (#382)
* plato - 0.9.17-2 (#344)
* quickjs - 2020.11.08-2
* recrossable - 0.0.0-7
* remarkable-splash - 1.0-5
* remux - 0.1.9-4
* restream - 1.1-2
* retris - 0.6.3-3
* rmservewacominput - 0.3.0-2
* simple - 0.1.4-3
* templatectl - 0.1.3-3
* toltec-bootstrap - 0.1.0-1 _(includes toltecctl)_ (#379)
* vnsee - 0.4.0-2 (#355)
* wikipedia - 0.1.0-3
* wireguard - 1.0.20210219-2
* xochitl - 0.0.0-7 (#383, #382, #386)
* zshelf - 0.3.1-3

Website: The source files and scripts used for generating the website have been transferred to a separate repository (<https://github.com/toltec-dev/web>). A website rebuild is triggered automatically from this repository’s workflow when the stable branch is updated, so that the displayed checksum of the bootstrap script is always accurate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
install Installation scripts packages Add or improve packages of the repository tooling Set of scripts and configuration files for building the packages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

rm2fb: disable LD_PRELOAD for busybox services feature: prevent installation of rM2 incompatible apps on rM2
6 participants