JDK-8217605: Add support for e-paper displays #339
Issue Type: Request for Enhancement
Add support for e-paper displays.
This is a request for JavaFX to support devices with electrophoretic displays, commonly known as electronic paper displays, with a new Monocle platform called “EPD.”
The ultimate goal of this request is to be able to run a JavaFX application on a device with an e-paper display by simply installing the Ubuntu OpenJFX package for the armhf architecture. That goal includes:
The following are specifically out of scope for this request:
One measure of success would be running a JavaFX animation program on an e-paper display at eight frames per second and delivering the correct touch screen input events to the program.
The motivation for this request comes from a work of art by Joseph Farbrook exhibited at the SIGGRAPH 2014 Art Gallery and part of a series called Lineographs. See the artist’s website for a brief description of the work and a one-minute video demonstration. The video is also available directly on YouTube as Dancer.
The artwork was remarkable in at least two ways. First, it was stunningly simple, using a device that emits no light, makes no sound, and employs the most minimal user interface. Among the bright screens and loud noises of the SIGGRAPH Conference, it was a passive and silent wonder—an ink drawing that came to life with a touch. Secondly, few people knew at the time that e-paper could display animations. Electronic ink was invented in 1997, and the first commercial e-reader introduced in 2004, but it wasn’t until 2012 that a patent application described the various waveform modes, including one for fast black-and-white animation.
When I looked into writing my own software for e-paper, I found the information buried deep in archives of documents for embedded processors. The documents are helpful, and improve with each revision, but they describe only a C-language interface to the Linux frame buffer device and some IOCTL codes. I was looking for a way to write the software in a higher-level language with libraries for graphics, threads, and networking. I was hoping to test the programs on my workstation and run them unmodified on the device without having to use cross-compilers or emulators. In short, I wanted an interface in Java.
The device driver for an Electrophoretic Display Controller supports the programming interface of the Linux frame buffer device with one key caveat: the contents of the frame buffer are not automatically updated to the display. The Monocle Linux platform in JavaFX already provides software rendering to the Linux frame buffer and can be extended to provide support for e-paper displays.
I propose a new Monocle platform, named EPD, that allows a JavaFX application to run on a device with an Electrophoretic Display Controller (EPDC) frame buffer driver. To select the platform, the application is started with the system properties:
$ java -Dglass.platform=Monocle -Dmonocle.platform=EPD -Dprism.order=sw ...
Additional system properties for the EPD platform are described below in three groups: settings for the Linux frame buffer device, general display update settings, and update flags.
Frame Buffer Device
The EPDC drivers on the NXP i.MX6SL and i.MX6SLL processors support software dithering, while the i.MX7D supports dithering in hardware. Dithering is not available on the i.MX507 and i.MX508 processors.
Messages are logged using the “javafx” system logger with the following levels:
There are three main alternatives to providing the support in JavaFX:
Among the popular cross-platform toolkits of Qt, Swing, JavaFX, and Flutter, only JavaFX supports both desktop computers and embedded devices with a single executable file.
The closest I have found to an inexpensive general-purpose computer with an e-paper display is one of the Rakuten Kobo e-readers, shown on that page with pricing in U.S. Dollars. (See the Canada website for pricing in Canadian Dollars.) Older used Kobo e-readers, such as the Kobo Touch, can be found for less than 40 USD, and newer refurbished models sell for less than 70 USD from Rakuten on eBay.
On the other hand, the E Ink ICE evaluation kit with a 6-inch display costs 449 USD and is not intended for personal or household use. “Orders placed by consumers will not be accepted,” says the notice at the E Ink Shop.
The advantage of a Rakuten Kobo over an Amazon Kindle is that you don’t need to jailbreak the device for access. A single setting added to a configuration file (“Kobo eReader.conf”) enables root access with the telnet command:
Once logged in, it’s relatively easy to create a chroot environment with a small Ubuntu Base image and to set up Ethernet over USB. You then have a stable high-speed connection with over 45,000 open-source packages ready to install—including OpenJDK and OpenJFX—already compiled for the armhf architecture. If necessary, you can emulate everything but the display on a regular Linux workstation by running Ubuntu in a QEMU virtual machine with the armv7l architecture.
Furthermore, all of the source code for the Linux kernel, the EPDC frame buffer drivers, and related software is posted in the Kobo-Reader repository on GitHub. Of course, you can also use the Kobo e-reader to read books.
Risks and Assumptions
There is the risk of Kobo devices being locked down or discontinued, making the development of this enhancement more difficult and expensive. In addition, this request is tied to the current technology for electronic paper, the microencapsulated electrophoretic display, creating the risk of a new technology taking over the market and making the implementation obsolete.
In the long run, both of these events are likely to occur as companies, markets, and technologies change. Meanwhile, the Kobo e-reader is distributed globally, has the leading market share in Canada, and is second to the Kindle in market share worldwide. The current E Ink technology was invented in 1997 and has yet to be replaced, despite many well-funded efforts to do so.
The implementation of this request should avoid making changes to the current JavaFX source code so that it may be added without any risk of causing errors in the existing features.
System / OS / Java Runtime Information
I propose starting with the Kobo Touch Models N905B and N905C, which have either an NXP i.MX507 or i.MX508 processor based on the ARM Cortex-A8 core. These are the oldest and least powerful devices still supported by Kobo with system updates. Once the implementation is stable, other more powerful models can be added, such as the Kobo Glo HD with an i.MX6SL processor and the Kobo Clara HD with an i.MX6SLL processor, each based on the ARM Cortex-A9 core.
Given that plan, the first test system is a Kobo Touch Model N905C with an 800-MHz i.MX507 Multimedia Application Processor Model MCIMX507CVM8B and 256 MB of RAM. This processor has an ARM Cortex-A8 core implementing the ARMv7-A architecture. The system runs Ubuntu Base 14.04.5 LTS (Trusty Tahr) in a chroot environment.
$ uname -a Linux koboa 126.96.36.199-850-gbc67621+ #619 PREEMPT Thu Dec 22 15:29:00 CST 2016 armv7l armv7l armv7l GNU/Linux $ ldd --version ldd (Ubuntu EGLIBC 2.19-0ubuntu6.14) 2.19 $ getconf GNU_LIBPTHREAD_VERSION NPTL 2.19 $ $HOME/opt/jdk-12-client/bin/java -version openjdk version "12-internal" 2019-03-19 OpenJDK Runtime Environment (build 12-internal+0-adhoc.ubuntu.jdk) OpenJDK Client VM (build 12-internal+0-adhoc.ubuntu.jdk, mixed mode)
The screen is a 6-inch E Ink Pearl display panel with a resolution of 800 × 600 pixels and a density of 167 pixels per inch, supporting up to 16 levels of gray (4-bit grayscale).
The text was updated successfully, but these errors were encountered:
As far as I can tell, there's no 3D graphics support in any of the devices I plan to test. The processors for the devices have the hardware accelerators listed below, where EPDC is the Electrophoretic Display Controller, and PXP is the pixel processing pipeline.
My current test device has the i.MX507 processor.