Initial support for >=3.6 firmwares, toolchain v4.x #27

ilyaluk · 2023-09-06T16:32:27Z

This updates all libraries in toolchain/base images according to 3.6.0.1865 firmware.

Includes patches for glibc 2.35 (to be upstreamed) and for breakpad.

Updated crosstool-ng to latest master and regerated crosstool.config using it.

Does not include QT updates yet, waiting on reMarkable toolchain with up-to-date libqsgepaper

base/Dockerfile

ilyaluk · 2023-09-06T16:35:04Z

base/Dockerfile

@@ -103,16 +95,18 @@ RUN export DEBIAN_FRONTEND=noninteractive \
    && cd .. \
    # Clean up
    && rm -rf systemd \
+    # TODO: remove pip packages as well


Not sure if it's required, they are pretty small (jinja2+yaml, I think)

I missed replying to this the first time. Yes, removing the packages is required as it's not part of what we intend to deliver by default, especially if we aren't delivering pip.

I'm thinking that we can use multi-stage builds: one that cross-compiles all the stuff, then copy the headers/libs to the final one with pre-installed compilers without any extra deps. This should require less friction and layers.

That would be appreciated, especially if there is caching we can do to avoid rebuilding if we already have a built version later.

Yep, docker allows to cache multi-stage builds with buildx, that should be supported by GHA, will look into that

I've had issues with it not properly caching before, and there is also a limit to how much can be cached, which this would fill it up really quick, so you'd probably still want to clean up as much as possible to keep the cache size needed small.

A better way to do this would be to move the build of these artifacts to a different process that stores them elsewhere with a version number, and then the docker image pulls the versioned artifact.

toolchain/Dockerfile

ilyaluk · 2023-09-06T16:37:15Z

toolchain/Dockerfile

@@ -90,9 +92,8 @@ RUN export DEBIAN_FRONTEND=noninteractive \
        libssl-dev \
        libtool-bin \
    && cd /root \
-    && git clone https://git.yoctoproject.org/git/opkg \
+    && git clone --depth 1 --branch v0.4.5 https://git.yoctoproject.org/git/opkg \


Updated opkg a bit as well, 0.4.4->0.4.5.

We can bump it all the way to 0.6.2, if this won't break anything.

My reply to this was missing for some reason. The version of opkg in entware is from 2022-02-24, so we should try to match its version here.

Entware uses opkg-lede, which is a (lesser maintained?) fork of opkg. For compatibility we can it as well, not sure what issues that might bring now.

It's what we use on device, so it would be best to use the same thing. It's probably a miss that we are using version.

ilyaluk · 2023-09-06T16:38:21Z

toolchain/crosstool-ng/crosstool.config

@@ -326,16 +335,20 @@ CT_LINUX_V_5_4=y
 # CT_LINUX_V_3_10 is not set
 # CT_LINUX_V_3_4 is not set
 # CT_LINUX_V_3_2 is not set
-CT_LINUX_VERSION="5.4.70"
+CT_LINUX_VERSION="5.4.248"


This is change made by ./ct-ng arm-remarkable-linux-gnueabihf. I guess it doesn't matter, as we only need kernel headers, which should not change in minor releases (not sure about that though)

Even previous version of ct-ng bumped version when copying from crosstool.config to .config, to something around 5.4.140

We would prefer to target the exact kernel version in use on the device.

Yeah, I tried that for the same reasons, but even if exact older version is specified here, after calling ./ct-ng arm-remarkable-linux-gnueabihf the latest kernel patch version supported by current ct-ng (5.4.248 as of now) is written to .config, and ./ct-ng build uses newer one.

I think the same happens in 3.x toolchain, but the version is 5.4.179: https://github.com/crosstool-ng/crosstool-ng/tree/a21748bd5342b72ddf5bfef782e058c3245161f7/packages/linux/5.4.179

We'd need to modify .config after calling ./ct-ng arm-remarkable-linux-gnueabihf if we want exact version, which is discouraged by crosstool-ng.

Have you found in the documentation why this is the case? Or opened an issue about it?

I'm not exactly happy that it's ignoring our configuration.

ilyaluk · 2023-09-06T16:40:39Z

toolchain/crosstool-ng/crosstool.config

@@ -399,12 +414,14 @@ CT_BINUTILS_V_2_36=y
 # CT_BINUTILS_V_2_28 is not set
 # CT_BINUTILS_V_2_27 is not set
 # CT_BINUTILS_V_2_26 is not set
-CT_BINUTILS_VERSION="2.36.1"
+CT_BINUTILS_VERSION="2.35.1"


Downgraded binutils just to be safe when debugged the issues with compilation and forgot to upgrade back. But I guess it's better to have binutils matching glibc?

It should match what is in the official toolchain ideally. Which would match what is on device if it ends up on device. I don't really think we want to downgrade them unless they actually need to be downgraded.

Still haven't received official toolchain for 3.6, sadly

Nothing from binutils end up on device, as far as I can tell. Also, 3.1.15 toolchain included 2.34.0 binutils, not 2.36.1:

# /opt/codex/rm11x/3.1.15/sysroots/x86_64-codexsdk-linux/usr/libexec/arm-remarkable-linux-gnueabi/gcc/arm-remarkable-linux-gnueabi/9.3.0/ld --version GNU ld (GNU Binutils) 2.34.0.20200220

I don't think that binutils versions are that important for linking binaries, and probably we can use either one until we get official toolchain, your call.

It looks like you have the latest toolchain now. Could you also open a PR against https://github.com/Eeems-Org/remarkable.guide/ to add the toolchain to https://github.com/Eeems-Org/remarkable.guide/blob/main/src/devel/toolchains.rst with ipfs links?

Official toolchain includes 2.38, apparently

$ /opt/codex/rm2/4.0.117-1/sysroots/x86_64-codexsdk-linux/usr/libexec/arm-remarkable-linux-gnueabi/gcc/arm-remarkable-linux-gnueabi/11.3.0/ld --version GNU ld (GNU Binutils) 2.38.20220708

Probably also should bump gcc to 11.3.0

toolchain/crosstool-ng/crosstool.config

base/Dockerfile

Eeems · 2023-09-07T20:35:14Z

toolchain/crosstool-ng/crosstool.config

@@ -326,16 +335,20 @@ CT_LINUX_V_5_4=y
 # CT_LINUX_V_3_10 is not set
 # CT_LINUX_V_3_4 is not set
 # CT_LINUX_V_3_2 is not set
-CT_LINUX_VERSION="5.4.70"
+CT_LINUX_VERSION="5.4.248"


We would prefer to target the exact kernel version in use on the device.

Eeems · 2023-09-07T20:37:41Z

toolchain/crosstool-ng/crosstool.config

@@ -399,12 +414,14 @@ CT_BINUTILS_V_2_36=y
 # CT_BINUTILS_V_2_28 is not set
 # CT_BINUTILS_V_2_27 is not set
 # CT_BINUTILS_V_2_26 is not set
-CT_BINUTILS_VERSION="2.36.1"
+CT_BINUTILS_VERSION="2.35.1"


It should match what is in the official toolchain ideally. Which would match what is on device if it ends up on device. I don't really think we want to downgrade them unless they actually need to be downgraded.

toolchain/crosstool-ng/crosstool.config

This updates all libraries in toolchain/base images according to 3.6.0.1865 firmware. Includes patches for glibc 2.35 (to be upstreamed) and for breakpad. Updated crosstool-ng to latest master and regerated crosstool.config using it.

ilyaluk · 2023-09-11T17:35:41Z

Btw, @Eeems, ct-ng offers way smaller configs by exporting them:

root@64aab2c30c79:~/crosstool-ng# ./ct-ng saveconfig
Reporter name [Mattéo Delabre]:
Reporter URL [https://matteo.delab.re]:
Old comment:
  > Toolchain for the reMarkable - Adapted from arm-cortexa9_neon-linux-gnueabihf
Reporter comment (Ctrl-D to finish, '.' to use previous):
.
root@64aab2c30c79:~/crosstool-ng# cat samples/arm-remarkable-linux-gnueabihf/crosstool.config
CT_CONFIG_VERSION="4"
CT_EXPERIMENTAL=y
CT_ALLOW_BUILD_AS_ROOT=y
CT_ALLOW_BUILD_AS_ROOT_SURE=y
CT_ARCH_ARM=y
CT_ARCH_CPU="cortex-a9"
CT_ARCH_FPU="neon"
CT_ARCH_FLOAT_HW=y
CT_TARGET_VENDOR="remarkable"
CT_TARGET_ALIAS="arm-linux-gnueabihf"
CT_TOOLCHAIN_ENABLE_NLS=y
CT_KERNEL_LINUX=y
CT_LINUX_V_5_4=y
CT_BINUTILS_V_2_35=y
CT_BINUTILS_LINKER_LD_GOLD=y
CT_BINUTILS_GOLD_THREADS=y
CT_BINUTILS_LD_WRAPPER=y
CT_BINUTILS_PLUGINS=y
CT_GLIBC_V_2_35=y
CT_GLIBC_USE_LIBIDN_ADDON=y
# CT_GLIBC_ENABLE_DEBUG is not set
CT_GLIBC_LOCALES=y
CT_GLIBC_KERNEL_VERSION_NONE=y
CT_GCC_V_10=y
# CT_CC_GCC_ENABLE_TARGET_OPTSPACE is not set
CT_CC_LANG_CXX=y
CT_DEBUG_GDB=y
CT_GDB_V_11=y
# CT_GDB_CROSS_PYTHON is not set
CT_GDB_NATIVE=y
CT_GDB_NATIVE_STATIC=y
CT_GETTEXT_V_0_20_1=y
CT_ISL_V_0_22=y
CT_NCURSES_V_6_2=y
# CT_NCURSES_NEW_ABI is not set

Which yields the same .config after ./ct-ng arm-remarkable-linux-gnueabihf.

I can update the PR to include this config format.

Eeems · 2023-09-17T17:12:47Z

Btw, @Eeems, ct-ng offers way smaller configs by exporting them:

...

Which yields the same .config after ./ct-ng arm-remarkable-linux-gnueabihf.

I can update the PR to include this config format.

Sure, a simpler config will be easier to maintain.

Eeems · 2023-09-17T17:14:26Z

base/Dockerfile

@@ -103,16 +95,18 @@ RUN export DEBIAN_FRONTEND=noninteractive \
    && cd .. \
    # Clean up
    && rm -rf systemd \
+    # TODO: remove pip packages as well


I missed replying to this the first time. Yes, removing the packages is required as it's not part of what we intend to deliver by default, especially if we aren't delivering pip.

Eeems · 2023-09-17T17:16:05Z

toolchain/Dockerfile

@@ -90,9 +92,8 @@ RUN export DEBIAN_FRONTEND=noninteractive \
        libssl-dev \
        libtool-bin \
    && cd /root \
-    && git clone https://git.yoctoproject.org/git/opkg \
+    && git clone --depth 1 --branch v0.4.5 https://git.yoctoproject.org/git/opkg \


My reply to this was missing for some reason. The version of opkg in entware is from 2022-02-24, so we should try to match its version here.

Eeems · 2023-09-17T17:17:27Z

toolchain/crosstool-ng/crosstool.config

@@ -326,16 +335,20 @@ CT_LINUX_V_5_4=y
 # CT_LINUX_V_3_10 is not set
 # CT_LINUX_V_3_4 is not set
 # CT_LINUX_V_3_2 is not set
-CT_LINUX_VERSION="5.4.70"
+CT_LINUX_VERSION="5.4.248"


Have you found in the documentation why this is the case? Or opened an issue about it?

I'm not exactly happy that it's ignoring our configuration.

Eeems · 2023-09-17T17:18:57Z

toolchain/crosstool-ng/crosstool.config

@@ -399,12 +414,14 @@ CT_BINUTILS_V_2_36=y
 # CT_BINUTILS_V_2_28 is not set
 # CT_BINUTILS_V_2_27 is not set
 # CT_BINUTILS_V_2_26 is not set
-CT_BINUTILS_VERSION="2.36.1"
+CT_BINUTILS_VERSION="2.35.1"


It looks like you have the latest toolchain now. Could you also open a PR against https://github.com/Eeems-Org/remarkable.guide/ to add the toolchain to https://github.com/Eeems-Org/remarkable.guide/blob/main/src/devel/toolchains.rst with ipfs links?

Eeems · 2023-12-17T18:44:29Z

@jilyaluk poke, any chance you've had time to continue working on this?

Eeems · 2024-02-08T23:52:36Z

@jilyaluk poke again?

ilyaluk · 2024-09-15T18:58:51Z

@Eeems sorry, I did not have access to my reMarkable for quite some time, so could not test anything. Now I got it back, will look at this soon.

ilyaluk · 2024-09-27T00:25:40Z

Hey @Eeems, could you bump the v4.x branch in the repo to match v3.x? I'll rebase my branch

Eeems · 2024-09-27T00:27:57Z

Done.

ilyaluk commented Sep 6, 2023

View reviewed changes

Eeems requested changes Sep 7, 2023

View reviewed changes

Eeems changed the base branch from v3.x to v4.x September 8, 2023 20:27

Initial support for >=3.6 firmwares, toolchain v4.x

b413fc9

This updates all libraries in toolchain/base images according to 3.6.0.1865 firmware. Includes patches for glibc 2.35 (to be upstreamed) and for breakpad. Updated crosstool-ng to latest master and regerated crosstool.config using it.

ilyaluk force-pushed the v4.x branch from 61c7359 to b413fc9 Compare September 11, 2023 17:32

ilyaluk requested a review from Eeems September 11, 2023 17:35

Eeems mentioned this pull request Sep 13, 2023

Bootstrap does not work on 3.6.0.1865 due to segmentation fault toltec-dev/toltec#731

Closed

Eeems requested changes Sep 17, 2023

View reviewed changes

timower mentioned this pull request Oct 9, 2023

rm2fb for version 3.6 timower/rM2-stuff#21

Closed

Initial support for >=3.6 firmwares, toolchain v4.x #27

Are you sure you want to change the base?

Initial support for >=3.6 firmwares, toolchain v4.x #27

Conversation

ilyaluk commented Sep 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ilyaluk Sep 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ilyaluk commented Sep 11, 2023 • edited Loading

Eeems commented Sep 17, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Eeems commented Dec 17, 2023

Eeems commented Feb 8, 2024

ilyaluk commented Sep 15, 2024

ilyaluk commented Sep 27, 2024

Eeems commented Sep 27, 2024

ilyaluk commented Sep 6, 2023 •

edited

Loading

ilyaluk Sep 6, 2023 •

edited

Loading

ilyaluk commented Sep 11, 2023 •

edited

Loading