Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve documentation for how to switch to the testing branch #341

Closed
gtsiam opened this issue Apr 14, 2021 · 2 comments · Fixed by #379
Closed

Improve documentation for how to switch to the testing branch #341

gtsiam opened this issue Apr 14, 2021 · 2 comments · Fixed by #379
Labels
documentation Improvements or additions to the documentation and to workflows

Comments

@gtsiam
Copy link

gtsiam commented Apr 14, 2021

I tried to switch to the testing branch for the newer koreader version. The documentation isn't all that clear in what I need to do for an existing toltec installation:

To use the testing branch, update the /opt/etc/opkg.conf file on an existing install

Yes, update how? Well, after some searching, I'm pretty sure what I'm supposed to do is change the package feed, from:

src/gz toltec https://toltec-dev.org/stable

to

src/gz toltec https://toltec-dev.org/testing

However this still gives me the following error when I try opkg update:

Downloading https://toltec-dev.org/testing/Packages.gz
wget: note: TLS certificate validation not implemented
wget: server returned error: HTTP/1.1 404 Not Found
*** Failed to download the package list from https://toltec-dev.org/testing/Packages.gz

Collected errors:
 * opkg_download: Failed to download https://toltec-dev.org/testing/Packages.gz, wget returned 1.

At least at the time of writing this issue, both https://toltec-dev.org/testing/Packages.gz and https://toltec-dev.org/testing/Packages return 404.
@matteodelabre
Copy link
Member

matteodelabre commented Apr 14, 2021 •
edited
Loading

Note: This comment is outdated, there is now a simpler way to switch to the testing branch. Refer to the comment at the end of this thread.

Hi @gtsiam, thanks for your report. You’re right that the documentation isn’t very precise as to how one can switch to the testing branch. This is definitely something that we plan on improving in the future; we’ve been discussing the addition of a script to switch branches automatically for some time now. There are several other issues with a higher priority that we’re working to address first, though.

We’ve recently changed the repository layout of the testing branch to better separate packages that work on rM1 only from those that work on rM2 only (see #310 for details). This layout change hasn’t landed in stable yet, hence the difference you’re seeing between https://toltec-dev.org/stable and https://toltec-dev.org/testing. With this new layout, the packages are now sorted into subfolders depending on which devices they support.

To switch to the testing branch, you’ll therefore need to update your Opkg configuration (located in /opt/etc/opkg.conf) for the new layout, as follows.

-src/gz toltec https://toltec-dev.org/stable
+arch rmall 200
+arch rm1 250
+src/gz toltec-rmall https://toltec-dev.org/testing/rmall
+src/gz toltec-rm1 https://toltec-dev.org/testing/rm1

If you’re using a reMarkable 2, you’ll need to replace all of the three occurrences of rm1 above with rm2. After doing this change, run opkg update and opkg upgrade and you should be all set.

Note that you’re more likely to encounter bugs when using the testing branch. When you find one, it would be much appreciated if you could report it in a new issue in this repository. Thanks!

@matteodelabre matteodelabre added the documentation Improvements or additions to the documentation and to workflows label Apr 14, 2021
@matteodelabre matteodelabre changed the title Missing testing feed Improve documentation for how to switch to the testing branch Apr 14, 2021
matteodelabre added a commit that referenced this issue May 8, 2021
@matteodelabre
Add uninstall script and make the bootstrap script install-only
c619158 
This PR adds a new `toltecctl` script, installed by the `toltec-bootstrap` package, that supersedes the former `entware-reenable` script and provides a way to uninstall Toltec (fixes #295). This script could be extended to add new install management features in the future. The uninstall script does the following things:

* Uninstalls all Toltec and Entware packages, running removal scripts for each package.
* Removes the bind mount of Toltec files on /opt (the mount is removed lazily so that it does not fail even if some processes that have files opened in that folder are still running)
* Removes all Toltec files
* Clears Toltec definitions from .bashrc
* Re-enables xochitl
* Reboots the device

Another major change of this PR is to make the bootstrap script fail if files from previous installs remain in the folders used by Toltec (fixes #332). With this change, the bootstrap script only has a single role, that is to create a new Toltec install in a clean system. Roles previously assumed explicitly or implicitly by the bootstrap script are deprecated:

* To re-enable an existing install after a system update, users should run `toltecctl enable` instead of re-running the bootstrap script.
* To create a new clean install, users should first uninstall the existing one and then run the bootstrap script.
* To switch to another branch (see #341), users should manually edit the `/opt/etc/opkg.conf` file instead of re-running the bootstrap script (a future PR could extend the `toltecctl` script to handle switching branches).

Test plan
=========

(Only tested on rM2 so far.)

Uninstall script
----------------

* Upgrade the toltec-bootstrap package to version 0.0.5-1 (this will remove the `bootstrap` and `entware-reenable` scripts from the home folder and add the `toltecctl` script to the `~/.local/bin` folder)
* Run `~/.local/bin/toltecctl enable` and re-source .bashrc (to add .local/bin to the PATH)
* Run `toltecctl uninstall`
* After the tablet reboots, make sure no traces of the Toltec install remain (in ~/.entware, /opt, /lib/systemd/system/opt.mount)
* If you had disabled xochitl, make sure that it starts up again after uninstalling

Re-enable script
----------------

* Upgrade the toltec-bootstrap package to version 0.0.5-1 (this will remove the `bootstrap` and `entware-reenable` scripts from the home folder and add the `toltecctl` script to the `~/.local/bin` folder)
* Simulate a system update, for example by switching to the other root partition.
* Run `~/.local/bin/toltecctl enable`
* Make sure that Toltec has been re-enabled (files visible in /opt, opkg command working, apps running)

Bootstrap script
----------------

* Copy the new bootstrap script to the tablet.
* Try running the bootstrap script with an existing Toltec install (the script should fail and tell you that Toltec is already installed)
* Try running the bootstrap script with only some parts of a Toltec install remaining (for example, only the .entware folder - the script should fail and tell you which files need to be cleaned up)
* Run the bootstrap script after uninstalling Toltec or from a clean system, and make sure the install works.
matteodelabre added a commit that referenced this issue May 8, 2021
@matteodelabre
Add uninstall script and make the bootstrap script install-only
6d46b45 
This PR adds a new `toltecctl` script, installed by the `toltec-bootstrap` package, that supersedes the former `entware-reenable` script and provides a way to uninstall Toltec (fixes #295). This script could be extended to add new install management features in the future. The uninstall script does the following things:

* Uninstalls all Toltec and Entware packages, running removal scripts for each package.
* Removes the bind mount of Toltec files on /opt (the mount is removed lazily so that it does not fail even if some processes that have files opened in that folder are still running)
* Removes all Toltec files
* Clears Toltec definitions from .bashrc
* Re-enables xochitl
* Reboots the device

Another major change of this PR is to make the bootstrap script fail if files from previous installs remain in the folders used by Toltec (fixes #332). With this change, the bootstrap script only has a single role, that is to create a new Toltec install in a clean system. Roles previously assumed explicitly or implicitly by the bootstrap script are deprecated:

* To re-enable an existing install after a system update, users should run `toltecctl enable` instead of re-running the bootstrap script.
* To create a new clean install, users should first uninstall the existing one and then run the bootstrap script.
* To switch to another branch (see #341), users should manually edit the `/opt/etc/opkg.conf` file instead of re-running the bootstrap script (a future PR could extend the `toltecctl` script to handle switching branches).

Test plan
=========

(Only tested on rM2 so far.)

Uninstall script
----------------

* Upgrade the toltec-bootstrap package to version 0.0.5-1 (this will remove the `bootstrap` and `entware-reenable` scripts from the home folder and add the `toltecctl` script to the `~/.local/bin` folder)
* Run `~/.local/bin/toltecctl enable` and re-source .bashrc (to add .local/bin to the PATH)
* Run `toltecctl uninstall`
* After the tablet reboots, make sure no traces of the Toltec install remain (in ~/.entware, /opt, /lib/systemd/system/opt.mount)
* If you had disabled xochitl, make sure that it starts up again after uninstalling

Re-enable script
----------------

* Upgrade the toltec-bootstrap package to version 0.0.5-1 (this will remove the `bootstrap` and `entware-reenable` scripts from the home folder and add the `toltecctl` script to the `~/.local/bin` folder)
* Simulate a system update, for example by switching to the other root partition.
* Run `~/.local/bin/toltecctl enable`
* Make sure that Toltec has been re-enabled (files visible in /opt, opkg command working, apps running)

Bootstrap script
----------------

* Copy the new bootstrap script to the tablet.
* Try running the bootstrap script with an existing Toltec install (the script should fail and tell you that Toltec is already installed)
* Try running the bootstrap script with only some parts of a Toltec install remaining (for example, only the .entware folder - the script should fail and tell you which files need to be cleaned up)
* Run the bootstrap script after uninstalling Toltec or from a clean system, and make sure the install works.
matteodelabre added a commit that referenced this issue May 8, 2021
@matteodelabre
Add uninstall script and make the bootstrap script install-only
16327d1 
This PR adds a new `toltecctl` script, installed by the `toltec-bootstrap` package, that supersedes the former `entware-reenable` script and provides a way to uninstall Toltec (fixes #295). This script could be extended to add new install management features in the future. The uninstall script does the following things:

* Uninstalls all Toltec and Entware packages, running removal scripts for each package.
* Removes the bind mount of Toltec files on /opt (the mount is removed lazily so that it does not fail even if some processes that have files opened in that folder are still running)
* Removes all Toltec files
* Clears Toltec definitions from .bashrc
* Re-enables xochitl
* Reboots the device

Another major change of this PR is to make the bootstrap script fail if files from previous installs remain in the folders used by Toltec (fixes #332). With this change, the bootstrap script only has a single role, that is to create a new Toltec install in a clean system. Roles previously assumed explicitly or implicitly by the bootstrap script are deprecated:

* To re-enable an existing install after a system update, users should run `toltecctl enable` instead of re-running the bootstrap script.
* To create a new clean install, users should first uninstall the existing one and then run the bootstrap script.
* To switch to another branch (see #341), users should manually edit the `/opt/etc/opkg.conf` file instead of re-running the bootstrap script (a future PR could extend the `toltecctl` script to handle switching branches).

Test plan
=========

(Only tested on rM2 so far.)

Uninstall script
----------------

* Upgrade the toltec-bootstrap package to version 0.0.5-1 (this will remove the `bootstrap` and `entware-reenable` scripts from the home folder and add the `toltecctl` script to the `~/.local/bin` folder)
* Run `~/.local/bin/toltecctl enable` and re-source .bashrc (to add .local/bin to the PATH)
* Run `toltecctl uninstall`
* After the tablet reboots, make sure no traces of the Toltec install remain (in ~/.entware, /opt, /lib/systemd/system/opt.mount)
* If you had disabled xochitl, make sure that it starts up again after uninstalling

Re-enable script
----------------

* Upgrade the toltec-bootstrap package to version 0.0.5-1 (this will remove the `bootstrap` and `entware-reenable` scripts from the home folder and add the `toltecctl` script to the `~/.local/bin` folder)
* Simulate a system update, for example by switching to the other root partition or using [remarkable-update](https://github.com/ddvk/remarkable-update)
* Run `~/.local/bin/toltecctl enable`
* Make sure that Toltec has been re-enabled (files visible in /opt, opkg command working, apps running)

Bootstrap script
----------------

* Copy the new bootstrap script to the tablet.
* Try running the bootstrap script with an existing Toltec install (the script should fail and tell you that Toltec is already installed)
* Try running the bootstrap script with only some parts of a Toltec install remaining (for example, only the .entware folder - the script should fail and tell you which files need to be cleaned up)
* Run the bootstrap script after uninstalling Toltec or from a clean system, and make sure the install works.
@matteodelabre matteodelabre mentioned this issue May 8, 2021
Merged
matteodelabre added a commit that referenced this issue Jun 4, 2021
@matteodelabre @Eeems @LinusCDE
Add uninstall script and make the bootstrap script install-only (#356)
b305be7 
This PR adds a new `toltecctl` script, installed by the `toltec-bootstrap` package, that supersedes the former `entware-reenable` script and provides a way to uninstall Toltec (fixes #295). This script could be extended to add new install management features in the future. The uninstall script does the following things:

* Uninstalls all Toltec and Entware packages, running removal scripts for each package.
* Removes the bind mount of Toltec files on /opt (the mount is removed lazily so that it does not fail even if some processes that have files opened in that folder are still running)
* Removes all Toltec files
* Clears Toltec definitions from .bashrc
* Re-enables xochitl
* Reboots the device

Another major change of this PR is to make the bootstrap script fail if files from previous installs remain in the folders used by Toltec (fixes #332). With this change, the bootstrap script only has a single role, that is to create a new Toltec install in a clean system. Roles previously assumed explicitly or implicitly by the bootstrap script are deprecated:

* To re-enable an existing install after a system update, users should run `toltecctl enable` instead of re-running the bootstrap script.
* To create a new clean install, users should first uninstall the existing one and then run the bootstrap script.
* To switch to another branch (see #341), users should manually edit the `/opt/etc/opkg.conf` file instead of re-running the bootstrap script (a future PR could extend the `toltecctl` script to handle switching branches).

Co-authored-by: Nathaniel van Diepen <Eeems@users.noreply.github.com>
Co-authored-by: Linus K <linus@cosmos-ink.net>
@matteodelabre matteodelabre mentioned this issue Jun 8, 2021
Closed
@matteodelabre matteodelabre mentioned this issue Jun 24, 2021
Closed
matteodelabre added a commit that referenced this issue Jun 25, 2021
@matteodelabre
Manage Opkg configuration with toltecctl
88e3766 
This is a three-headed monster of a PR, addressing three distinct yet linked issues.

New toltecctl features
----------------------

At its core, this PR extends the toltecctl script (provided by toltec-bootstrap, introduced in #356) and enables it to manage the Opkg configuration, i.e., switch branches and migrate the configuration when needed. To achieve this without risking to overwrite custom config directives added by users, a split configuration approach is taken (as suggested by @Eeems):

* The main `/opt/etc/opkg.conf` file is automatically generated by concatenating the files in the `/opt/etc/opkg.conf.d/` folder (in lexicographic order).
* The file `/opt/etc/opkg.conf.d/10-entware.conf` configures the Entware repository and architecture names.
* The file `/opt/etc/opkg.conf.d/15-toltec.conf` configures the Toltec repository and architecture names.
* Users may create new files in `/opt/etc/opkg.conf.d` to add their custom directives.

New toltecctl commands are made available to users for managing the Opkg configuration:

* `toltecctl generate-opkg-conf` - Regenerates the main config file and automatically runs `opkg update` if the config changed
* `toltecctl switch-branch [BRANCH]` - Switches the current Toltec branch, rebuilds the config file and runs `opkg update`

This new `toltecctl switch-branch` fixes #341. A confirmation prompt is shown when someone tries to switch to the `testing` branch to warn them of the potential instability.

Bootstrap script revamp
-----------------------

This fixes #141.

With the previously mentioned additions, toltecctl now almost fully duplicates the features of the bootstrap script: it can create bind mounts, initialize the Opkg configuration, and define the user’s PATH. This PR therefore also contains a new bootstrap script that fetches the `toltec-bootstrap` package (containing toltecctl) first, and then uses its functions to perform the install, instead of reimplementing the same logic. By consolidating the features and variables of the two scripts, I hope to reduce the surface for potential bugs. This also makes it easier to add migrations to the `toltec-bootstrap` package, since it shares the code of the bootstrap script that’s used for new installs.

Migration to the new Opkg configuration
---------------------------------------

Building on the toltecctl improvements, a config migration logic was added to the `toltec-bootstrap` package’s configure function. This will automatically migrate single-opkg.conf-file installs to the new split config layout, while keeping the currently configured branch.

This migration is also able to transition configs using the old repository layout (with a single folder containing all packages) to the new arch-subfolder layout. By publishing the new toltec-bootstrap package to the root folder of the stable branch, this will solve #370.
@matteodelabre matteodelabre mentioned this issue Jun 25, 2021
Merged
@matteodelabre matteodelabre linked a pull request Jun 25, 2021 that will close this issue
Manage Opkg configuration with toltecctl #379
Merged
@matteodelabre matteodelabre mentioned this issue Jul 20, 2021
Open
matteodelabre added a commit that referenced this issue Jul 20, 2021
@matteodelabre @Eeems @LinusCDE
Manage Opkg configuration with toltecctl (#379)
85ca5ee 
This PR addresses three issues:

1. It enables toltecctl to manage the Opkg configuration.
2. It updates the bootstrap script to delegate most of its features to toltecctl.
3. It adds a migration script in toltec-bootstrap to make existing installs on stable or testing automatically migrate to the new repository layout.

New toltecctl features
----------------------

At its core, this PR extends the toltecctl script (provided by toltec-bootstrap, introduced in #356) and enables it to manage the Opkg configuration, i.e., switch branches and migrate the configuration when needed. To achieve this without risking to overwrite custom config directives added by users, a split configuration approach is taken (as suggested by @Eeems):

* The main `/opt/etc/opkg.conf` file is automatically generated by concatenating the files in the `/opt/etc/opkg.conf.d/` folder (in lexicographic order).
* The file `/opt/etc/opkg.conf.d/10-entware.conf` configures the Entware repository and architecture names.
* The file `/opt/etc/opkg.conf.d/15-toltec.conf` configures the Toltec repository and architecture names.
* Users may create new files in `/opt/etc/opkg.conf.d` to add their custom directives.

New toltecctl commands are made available to users for managing the Opkg configuration:

* `toltecctl generate-opkg-conf` - Regenerates the main config file and automatically runs `opkg update` if the config changed
* `toltecctl switch-branch [BRANCH]` - Switches the current Toltec branch, rebuilds the config file and runs `opkg update`

This new `toltecctl switch-branch` fixes #341. A confirmation prompt is shown when someone tries to switch to the `testing` branch to warn them of the potential instability.

Bootstrap script revamp
-----------------------

This fixes #141.

With the previously mentioned additions, toltecctl now almost fully duplicates the features of the bootstrap script: it can create bind mounts, initialize the Opkg configuration, and define the user’s PATH. This PR therefore also contains a new bootstrap script that fetches the `toltec-bootstrap` package (containing toltecctl) first, and then uses its functions to perform the install, instead of reimplementing the same logic. By consolidating the features and variables of the two scripts, I hope to reduce the surface for potential bugs. This also makes it easier to add migrations to the `toltec-bootstrap` package, since it shares the code of the bootstrap script that’s used for new installs.

Migration to the new Opkg configuration
---------------------------------------

Building on the toltecctl improvements, a config migration logic was added to the `toltec-bootstrap` package’s configure function. This will automatically migrate single-opkg.conf-file installs to the new split config layout, while keeping the currently configured branch.

This migration is also able to transition configs using the old repository layout (with a single folder containing all packages) to the new arch-subfolder layout. By publishing the new toltec-bootstrap package to the root folder of the stable branch, this will solve #370.

Co-authored-by: Nathaniel van Diepen <Eeems@users.noreply.github.com>
Co-authored-by: Linus K. <linus@cosmos-ink.net>
@matteodelabre
Copy link
Member

matteodelabre commented Jul 20, 2021 •
edited
Loading

You can now switch between testing and stable by running toltecctl switch-branch testing or toltecctl switch-branch stable. This is documented here: https://github.com/toltec-dev/toltec/blob/stable/docs/branches.md#release-branches.

@matteodelabre matteodelabre mentioned this issue Jul 21, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
documentation Improvements or additions to the documentation and to workflows
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Manage Opkg configuration with toltecctl toltec-dev/toltec
2 participants
@matteodelabre @gtsiam