Support for Remarkable 2

[faroit]

• KOReader version: v2020.10
• Device: Remarkable 2

Issue

Koreader does not start on Remarkable 2.

• screen freezes
• systemctl start xochitl brings it back to normal

Steps to reproduce

Followed installation instruction on https://github.com/koreader/koreader/wiki/Installation-on-ReMarkable

crash.log (if applicable)

Original fb settings: bitdepth = 32, rotation = 0
Switching fb bitdepth to 8bpp & rotation to Portrait
[FBInk] Detected a reMarkable (Zero Gravitas)
Variable fb info: 260x1408 (260x23936), 32bpp @ rotation: 1 (Clockwise, 90°)
Fixed fb info: ID is "mxs-lcdif", length of fb mem: 33554432 bytes & line length: 1040 bytes
---------------------------------------------
                launching...
  _  _____  ____                _
 | |/ / _ \|  _ \ ___  __ _  __| | ___ _ __
 | ' / | | | |_) / _ \/ _` |/ _` |/ _ \ '__|
 | . \ |_| |  _ <  __/ (_| | (_| |  __/ |
 |_|\_\___/|_| \_\___|\__,_|\__,_|\___|_|

 It's a scroll... It's a codex... It's KOReader!

 [*] Current time: 10/14/20-14:29:45
 [*] Version: v2020.10-1-gac6b121_2020-10-10

ffi.load: blitbuffer
ffi.load: SDL2
ffi.load (warning): libSDL2.so: cannot open shared object file: No such file or directory
ffi.load: libSDL2-2.0.so
ffi.load (warning): libSDL2-2.0.so: cannot open shared object file: No such file or directory
ffi.load: libSDL2-2.0.so.0
ffi.load (warning): libSDL2-2.0.so.0: cannot open shared object file: No such file or directory
SDL2 not loaded:        ./setupkoenv.lua:27: Not able to load dynamic library: libSDL2-2.0.so.0
[ko-input] Forked off fake event generator(pid:325).
10/14/20-14:29:45 INFO  initializing for device reMarkable
10/14/20-14:29:45 INFO  framebuffer resolution: {
    ["w"] = 260,
    ["h"] = 1408
}
ffi.load: libs/libmupdf.so
ffi.load: libs/libwrap-mupdf.so
ffi.load: libs/libfreetype.so.6
10/14/20-14:29:46 INFO  Loading plugins from directory: plugins
ffi.load: sqlite3
ffi.load: z
ffi.load (assisted searchpath): ./libs/libz.so.1
ffi.load: sqlite3
10/14/20-14:29:46 INFO  Plugin  plugins/autofrontlight.koplugin/main.lua  has been disabled.
10/14/20-14:29:46 INFO  Plugin  plugins/autostandby.koplugin/main.lua  has been disabled.
10/14/20-14:29:46 INFO  Plugin  plugins/timesync.koplugin/main.lua  has been disabled.
10/14/20-14:29:46 INFO  Plugin  plugins/SSH.koplugin/main.lua  has been disabled.
10/14/20-14:29:46 INFO  Plugin  plugins/keepalive.koplugin/main.lua  has been disabled.
10/14/20-14:29:46 INFO  Plugin  plugins/hello.koplugin/main.lua  has been disabled.
10/14/20-14:29:46 INFO  FM loaded plugin autosuspend at plugins/autosuspend.koplugin
10/14/20-14:29:46 INFO  FM loaded plugin backgroundrunner at plugins/backgroundrunner.koplugin
10/14/20-14:29:46 INFO  FM loaded plugin batterystat at plugins/batterystat.koplugin
10/14/20-14:29:46 INFO  FM loaded plugin calibre at plugins/calibre.koplugin
10/14/20-14:29:46 INFO  FM loaded plugin coverbrowser at plugins/coverbrowser.koplugin
10/14/20-14:29:46 INFO  FM loaded plugin docsettingtweak at plugins/docsettingtweak.koplugin
10/14/20-14:29:46 INFO  FM loaded plugin evernote at plugins/evernote.koplugin
10/14/20-14:29:46 INFO  FM loaded plugin gestures at plugins/gestures.koplugin
ffi.load: libs/liblept.so.5
ffi.load: libs/libk2pdfopt.so.2

[NiLuJe]

The rM 2 isn't actually supported, as you've just found out ;o).

The fb info looks... wonky as hell. There wouldn't happen to be another fb device would it? Because fb0 doesn't look to be the one we want here...

[faroit]

@NiLuJe thanks for the reply. If there is anything I could do to help, just let me know.

[NiLuJe]

I'd start with what tinkerers have managed to do on the rM 2 in general. I don't have (either) device, so I don't even really know where those kind of discussions usually happen, but I don't think it's on MobileRead like our other platforms. Reddit? Discord?

[NiLuJe]

Apparently, surprise, surprise, the kernel sources aren't out. So, I assume people are waiting on that, first.

[faroit]

@NiLuJe @pazos @raisjnn @tcrs I am glad to help (e.g. giving you more details or giving you access to my device through a video call).

[gbyl]

@faroit Even for RM1 there are simply not enough contributors who own the device to have feature-parity with say the Kobo port of KOReader. It might take a while until RM2 is supported. Here is a bit more info about what you can look forward to: Eeems-Org/oxide#48 (comment)

[raisjn]

thanks @faroit, i'm still waiting to receive my rM2 but have been very interested in trying to draw anything to the framebuffer.

you can ping me on discord (username: rsjn) if you have time to do a call and poke around. there's a lot of friendly homebrew devs on discord server, so you can usually get help with any homebrew apps there.

it originally took me about a week (maybe less, maybe more) to figure out how to draw to framebuffer for rM1 and that was with other people's work available. my main goal is figuring out how to draw anything into the FB, from there, i think everything else will follow.

currently, i think the display is elcdif with mxsfb driver, but not sure if we are using kms/drm driver or fbdev. if its using kms, then drawing may be pretty different. i'm hoping that we still have access via /dev/fb0 and that there aren't undocumented ioctls we need to use.

[raisjn]

also, cc @ddvk

[NiLuJe]

Assuming this behaves like a sane Linux system, KMS shouldn't prevent using the raw fb device.

e.g., that jokey comment at the end is accurate: over the years, I've tested it against both amdgpudrmfb fbs & whatever the binary nvidia blob actually does ^^.

[FBInk] Variable fb info: 2560x1440, 32bpp @ rotation: 0 (Upright, 0°)
[FBInk] Fixed fb info: ID is "amdgpudrmfb", length of fb mem: 14745600 bytes & line length: 10240 bytes

[raisjn]

Assuming this behaves like a sane Linux system, KMS shouldn't prevent using the raw fb device.

good point! bokluk also checked and it looks like /dev/fb0 is being used (and not /dev/dri/render*), so xochitl is using fbdev (at least for some things)

[NiLuJe]

The fb setup on fb0 looks extremely out of whack for an EPDC, though, which is why I'm wondering if there isn't an fb1 or something...

rM1:

[FBInk] Variable fb info: 1404x1872, 16bpp @ rotation: 1 (Clockwise, 90°)
[FBInk] Fixed fb info: ID is "mxc_epdc_fb", length of fb mem: 10813440 bytes & line length: 2816 bytes

rM2:

Variable fb info: 260x1408 (260x23936), 32bpp @ rotation: 1 (Clockwise, 90°)
Fixed fb info: ID is "mxs-lcdif", length of fb mem: 33554432 bytes & line length: 1040 bytes

[NiLuJe]

I mean, if I look at a Kindle Oasis 3 kernel (which is using an i.MX7D, which I'm assuming is what's in the rM2), the mxsfb driver really doesn't look like what we're looking for:

  #define DRIVER_NAME "mxsfb"
  
 /**
  * @file
  * @brief LCDIF driver for i.MX23 and i.MX28
  *
  * The LCDIF support four modes of operation
  * - MPU interface (to drive smart displays) -> not supported yet
  * - VSYNC interface (like MPU interface plus Vsync) -> not supported yet
  * - Dotclock interface (to drive LC displays with RGB data and sync signals)
  * - DVI (to drive ITU-R BT656)  -> not supported yet
  *
  * This driver depends on a correct setup of the pins used for this purpose
  * (platform specific).
  *
  * For the developer: Don't forget to set the data bus width to the display
  * in the imx_fb_videomode structure. You will else end up with ugly colours.
  * If you fight against jitter you can vary the clock delay. This is a feature
  * of the i.MX28 and you can vary it between 2 ns ... 8 ns in 2 ns steps. Give
  * the required value in the imx_fb_videomode structure.
  */

Instead, it's probably still the mxc_epdc_fb we want, (and it ought to be a real v2 this time, so, more fun stuff in the ioctls, like on Kindle Zelda/Rex & Kobo Mk. 7).

[raisjn]

There are two mxsfb drivers: one is fbdev, one is drm/kms. The drm one has patches for imx7: https://lwn.net/Articles/823498/. Without my hands on rm2, I can't really investigate too well, though, I can only speculate and you have more knowledge / experience in this area.

I have looked at both xochitl binaries (for rm1 and rm2) and mxc does show up in the rm1 xochitl file as an error string next to opening fb0, while there is no mention of mxc (or mxs) in rm2 binary. This doesn't mean rm2 isn't using mxc though.

[Eeems]

@ddvk do you have any insight?

[ddvk]

well, it is totally different, tables from a .wbf file get loaded, then written somewhere. only pan and blank ioctls, the fb resolution doesnt make sense, etc i am trying to understand, how exactly it all fits together but i need time

…

On 15 Oct 2020, at 18:27, Nathaniel van Diepen ***@***.***> wrote:  @ddvk do you have any insight? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[NiLuJe]

Huh. Well, that looks fun (not). :s

[bttnns]

@NiLuJe @pazos @raisjnn @tcrs I am glad to help (e.g. giving you more details or giving you access to my device through a video call).

I also have a rm2 and can offer this.

[harrylepotter]

solution for getting framebuffer data from process memory outlined here:
#rien/reStream#28

[tadfisher]

Hopefully the device tree and kernel config clears up a couple of things: https://gist.github.com/tadfisher/0d4ac03e03b8099cd1075e00c95b8f12

Observations:

• The EPDC is not even enabled in the device tree. Only the eLCDIF interface is being used, which matches up with the mxc_lcdif driver providing /dev/fb0.

Speculation:

• A possibility is that the reMarkable 2 uses an external TCON to drive the panel, which would make sense as there is probably a custom protocol to upload waveform data to the TCON via writes on the LCD data bus, which explains why xochitl writes waveform data directly to the framebuffer device.

[raisjn]

we've (bokluk and I) had a small amount of success in drawing to screen:

as mentioned elsewhere, we think it's happening in software (and now SWTCON makes sense: it is SW TCON).

https://github.com/ddvk/remarkable2-framebuffer

[raisjn]

a video of koreader on rM2

also, i still dont know wtf a TCON is

[pazos]

This diff is the same, but should work on both.

Doesn't incude the ev hook replacements. I would suggest to split these in different hooks. AFAICT the diffs are:

• RM2 requires updated timestamp (should work on RM1 too?)
• RM1/2 have swapped X axis between them. In that case it would be a matter of adding touch_mirrored_x on RM1? and use something like https://github.com/koreader/koreader/blob/master/frontend/device/kobo/device.lua#L461-L467

diff --git a/frontend/device/remarkable/device.lua b/frontend/device/remarkable/device.lua
index a562916b..019a0aab 100644
--- a/frontend/device/remarkable/device.lua
+++ b/frontend/device/remarkable/device.lua
@@ -4,6 +4,32 @@ local logger = require("logger")
 local function yes() return true end
 local function no() return false end
 
+local function isRM2()
+    local f = io.open("/sys/class/power_supply/max77818_battery/capacity", "r")
+    if f then
+        f:close()
+        return true
+    end
+end
+
+local EV_ABS = 3
+local ABS_X = 00
+local ABS_Y = 01
+local ABS_MT_POSITION_X = 53
+local ABS_MT_POSITION_Y = 54
+
+-- Resolutions from libremarkable src/framebuffer/common.rs
+local screen_width = 1404 -- unscaled_size_check: ignore
+local screen_height = 1872 -- unscaled_size_check: ignore
+local wacom_width = 15725 -- unscaled_size_check: ignore
+local wacom_height = 20967 -- unscaled_size_check: ignore
+local wacom_scale_x = screen_width / wacom_width
+local wacom_scale_y = screen_height / wacom_height
+local mt_width = isRM2() and 1403 or 767 -- unscaled_size_check: ignore
+local mt_height = isRM2() and 1871 or 1023 -- unscaled_size_check: ignore
+local mt_scale_x = screen_width / mt_width
+local mt_scale_y = screen_height / mt_height
+
 local Remarkable = Generic:new{
     model = "reMarkable",
     isRemarkable = yes,
@@ -20,22 +46,22 @@ local Remarkable = Generic:new{
     home_dir = "/mnt/root",
 }
 
-local EV_ABS = 3
-local ABS_X = 00
-local ABS_Y = 01
-local ABS_MT_POSITION_X = 53
-local ABS_MT_POSITION_Y = 54
--- Resolutions from libremarkable src/framebuffer/common.rs
-local screen_width = 1404 -- unscaled_size_check: ignore
-local screen_height = 1872 -- unscaled_size_check: ignore
-local wacom_width = 15725 -- unscaled_size_check: ignore
-local wacom_height = 20967 -- unscaled_size_check: ignore
-local wacom_scale_x = screen_width / wacom_width
-local wacom_scale_y = screen_height / wacom_height
-local mt_width = 767 -- unscaled_size_check: ignore
-local mt_height = 1023 -- unscaled_size_check: ignore
-local mt_scale_x = screen_width / mt_width
-local mt_scale_y = screen_height / mt_height
+local Remarkable1 = Remarkable:new{
+    model = "reMarkable1",
+    home_dir = "/mnt/root",
+    input_wacom = "/dev/input/event0",
+    input_ts = "/dev/input/event1",
+    input_buttons = "/dev/input/event2",
+}
+
+local Remarkable2 = Remarkable:new{
+    model = "reMarkable2",
+    home_dir = "/home/root",
+    input_wacom = "/dev/input/event1",
+    input_ts = "/dev/input/event2",
+    input_buttons = "/dev/input/event0",
+}
+
 local adjustTouchEvt = function(self, ev)
     if ev.type == EV_ABS then
         -- Mirror X and scale up both X & Y as touch input is different res from
@@ -68,9 +94,9 @@ function Remarkable:init()
         event_map = require("device/remarkable/event_map"),
     }
 
-    self.input.open("/dev/input/event0") -- Wacom
-    self.input.open("/dev/input/event1") -- Touchscreen
-    self.input.open("/dev/input/event2") -- Buttons
+    self.input.open(self.input_wacom) -- Wacom
+    self.input.open(self.input_ts) -- Touchscreen
+    self.input.open(self.input_buttons) -- Buttons
     self.input:registerEventAdjustHook(adjustTouchEvt)
     -- USB plug/unplug, battery charge/not charging are generated as fake events
     self.input.open("fake_events")
@@ -110,5 +136,8 @@ function Remarkable:reboot()
     os.execute("systemctl reboot")
 end
 
-return Remarkable
-
+if isRM2() then
+    return Remarkable2
+else
+    return Remarkable1
+end

[raisjn]

@pazos nice!

@NiLuJe shared a slightly different way of determining rM2 (instead of using battery path): ddvk/remarkable2-framebuffer#5 (comment)

also, our env variable (to know if running inside rM2FB) is RM2FB_SHIM and if koreader is in rM2 mode with that env variable unset, an error should be emitted / displayed.

[pazos]

Indeed @NiLuJe's way makes sense. Anyways, anything that can diff between models should work.

also, our env variable (to know if running inside rM2FB) is RM2FB_SHIM and if koreader is in rM2 mode with that env variable unset, an error should be emitted / displayed.

Agree. Something like

if isRM2() then
    if not os.getenv("RM2FB_SHIM") then
        error("reMarkable 2 requires RM2FB to work")
    end
    return Remarkable2
else
    return Remarkable1
end

[raisjn]

just to check: is the plan to bundle rm2fb's server.so and client.so with koreader's rm2 package and supply a shell script that starts server, then launches KOReader?

[pazos]

@raisjn: I don't think we want to ship blobs with the program and building something Qt from source would be a PITA. So probably we can consider the server a prerequisite and just update platform instructions.

The same if the client requires Qt on our toolchain. If not it would be ok to integrate it in https://github.com/koreader/koreader-base.

[raisjn]

I don't think we want to ship blobs with the program and building something Qt from source would be a PITA. So probably we can consider the server a prerequisite and just update platform instructions.

i need to think more here. my main worry is that its possible the rm2fb server and client APIs can drift in the future and KOReader should just ship with the server API that its compatible with or we should finalize the API (as v1) before shipping a client.so inside KOReader.

build wise: server.so does require QT (sadly). client.so does not require QT and is easy enough to build.

i've tried out the build of KOReader from ddvk and touches / swipes work, so... quite exciting progress, imo :-D

[pazos]

build wise: server.so does require QT (sadly). client.so does not require QT and is easy enough to build.

if both the server and the client share a revision and that's available somehow (with a call in the client or with a env var) it can be easily integrated (server as prerequisite, client in base and a runtime check in the frontend that checks that the server is compatible with the client)

i've tried out the build of KOReader from ddvk and touches / swipes work, so... quite exciting progress, imo :-D

The videos I saw look good. My suggestion would be pushing a branch with the changes needed and make a WIP PR here and in koreader-base. Certainly it's the only way we can help you with the integration.

Making that PR won't force you to submit code quickly or merging platform support before it is ready, but makes it easier to see what happens as we can speak "glue code" language.

AFAICT required code changes in the program are minimum and most of the burden is in the RM2FB project. In that case the PR makes even more sense as can be reviewed by maintainers, tested by device owners and merged when you consider the main program is ready.

I need to think more here. my main worry is that its possible the rm2fb server and client APIs can drift in the future and KOReader should just ship with the server API that its compatible with or we should finalize the API (as v1) before shipping a client.so inside KOReader.

Take all the time you want, of course. None of the maintainers has a ReMarkable so it is up to you to provide code/support if you like, when you like and until you like :)

[dmhowcroft]

I don't necessarily have time for development at the moment, but I'm also happy to help with testing on rM2, as mine has just arrived. Let me know where to find the current testing branch/fork/whatever for Koreader on rM2 and I'll give it a shot :)

[Eeems]

@ddvk Could you open a WIP PR with your PoC that we can start suggesting changes to?

[raisjn]

#6992 is the pr

[msantanap]

I have an issue probably related with this tread.

I tried to install Koreader on my Remarkable2 (I know, my bad), and after trying to launch it, the screen froze. I have tried a hard-reset (by pressing the power button for 10 secs and then for 2 secs again) and nothing happens. I still see the "reMarkable is starting". I can't connect with the device via the OpenSSH client either (I guess because it is 'starting')

Please help!!!!

[Eeems]

@ManuelSantana USB ssh doesn't work?

[msantanap]

@Eeems, I just managed to access it via USB ssh. I am looking at what to do next.

[msantanap]

Solved! I had to reboot from the terminal and then type: systemctl start xochitl

[jannes]

Hi, it's great that Koreader is gonna support Remarkable 2!
Will the installation steps be exactly the same as for Remarkable 1,
or are they different? The wiki is not updated yet.

[Eeems]

Hi, it's great that Koreader is gonna support Remarkable 2!
Will the installation steps be exactly the same as for Remarkable 1,
or are they different? The wiki is not updated yet.

I believe they might have to be different since you need rm2fb

[pazos]

@jannes: proper support isn't ready. Currently (and probably forever) support is tied to https://github.com/ddvk/remarkable2-framebuffer, and its current status is pre-alpha.

What was updated were device quirks that make possible to identify the RM2 and work on top of remarkable2-framebuffer, so rm devs can play with it in their tests. Is not intended for final users until these devs decide it is ready :)

As for the wiki, feel free to update it adding the pre-release statement or just wait until remarkable2 is ready for end users to use.

[jannes]

Thanks for the clarification:)

[Miladiir]

@ddvk https://www.bountysource.com/issues/93429111-support-for-remarkable-2

[Eeems]

I mean, it's almost ready already >.>
toltec-dev/toltec#159

[ddvk]

@ddvk https://www.bountysource.com/issues/93429111-support-for-remarkable-2

i guess i can whip a script that installs koreader with the shim and something to switch between xochitl, or use tolltec

[ddvk]

@ddvk https://www.bountysource.com/issues/93429111-support-for-remarkable-2

ok, here is one way: https://github.com/ddvk/remarkable-autoinstall/tree/master/rm2
auto installer that enables toggling between xochitl and koreader

[Miladiir]

I tried it out and it's working with one of the nightlies. Thanks to everyone who was involved. I just wanted to remind you guys to go and claim the bounty.

[Frenzie]

@ddvk Btw, is there anything I/we need to do to allow you to claim the bounty?

[ddvk]

no idea how that works, i would claim it just out of curiosity

…

On Fri, 8 Jan 2021 at 11:08 Frans de Jonge ***@***.***> wrote: @ddvk <https://github.com/ddvk> Btw, is there anything I/we need to do to allow you to claim the bounty? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6792 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIYZFLV6VZCKB7T2TNSHGNLSY3KS5ANCNFSM4SQWQBVA> .

[Frenzie]

I don't really know either, but in any case it's your prerogative. I assume the fact that your #6992 is linked as closing this issue should give you the right to claim it.