8227425: Add support for e-paper displays on i.MX6 devices

[jgneff]

This pull request adds support for e-paper displays with the i.MX 6 Series of Applications Processors and implements Issue #521 in the obsolete javafxports/openjdk-jfx repository. Some of the changes were made to support the new device models, while others are minor changes to the existing support in JavaFX 13.

The following changes were made to support the new device models.

• Work around problems on the Kobo Glo HD Model N437.

  Ignore the error ENOTTY (25), "Inappropriate ioctl for device," when setting the waveform modes. The IOCTL call is ignored by the driver on the Kobo Glo HD where the error occurs, anyway.

  Use the visible y-resolution (yres), not the virtual y-resolution (yres_virtual), when calculating the capacity of the off-screen byte buffer and the length of the frame buffer mapping. The virtual y-resolution reported by the frame buffer device driver can be an incorrect value.

• Work around problems on the Kobo Clara HD Model N249.

  The Kobo Clara HD requires the native screen width to be set to the visible x-resolution (xres), instead than the normal virtual x-resolution (xres_virtual), when using an unrotated and uninverted 8-bit grayscale frame buffer. The work-around is provided through a new Boolean system property called monocle.epd.fixWidthY8UR.

The following changes were made to the existing code that supports e-paper displays in JavaFX 13.

• Use the correct constant for the number of bytes per pixel.

  Use the number of bytes per pixel (Integer.BYTES), not the number of bits per pixel (Integer.SIZE), when calculating the capacity of the 32-bit off-screen byte buffer.

• Do not round the luma value to the nearest integer.

  Use the value of luma rounded toward zero automatically by Java when converting a float to an int instead of rounding to the nearest integer. The additional rounding operation can decrease performance anywhere from zero to 10 percent and doesn't seem worth it for a display with just 16 levels of gray.

• Change camel case of system property y8inverted.

  Change the camel case of the system property monocle.epd.y8inverted to the form monocle.epd.Y8Inverted so that it is consistent with the other system properties containing Y1, Y4, and Y8 in their names.

• Improve error handling when mapping the frame buffer.

  Log the mapping and unmapping of the frame buffer. Log any errors that occur on either of the system calls. Handle a null return value from EPDFrameBuffer.getMappedBuffer.

• Replace non-ASCII characters with their ASCII equivalent.

  Replace non-ASCII characters in comments and log messages as follows:

  • "×" with "x" for display resolution and color depth,
  • "×" with "*" for multiplication,
  • "→" with "to" for transition, and
  • "°C" with "degrees Celsius" for temperature.

• Add descriptions to Monocle EPD system properties.

  Add Javadoc comments to each constant that defines a Monocle EPD system property.

• Rename IntStructure.getInteger to IntStructure.get.

  Rename the methods getInteger and setInteger in IntStructure to avoid confusion with the Java method Integer.getInteger, which gets the integer value of a system property.

Below are some of the more interesting test results.

• The Kobo Glo HD and Kobo Clara HD implement a true GC4 waveform mode that displays only pixels with the four gray values 0x00, 0x55, 0xAA, and 0xFF. On the older Kobo Touch models, the GC4 waveform mode is the same as GC16.

• When the forceMonochrome update flag is enabled, the Kobo Clara HD uses a zero-percent threshold that displays black for pixels with the value 0x00 and white for all other values. The other models use a 50-percent threshold that displays black for values 0x7F or less and white for values 0x80 or greater.

• The OpenJDK 11 package in Ubuntu 18.04 LTS runs twice as fast as the AdoptOpenJDK build of OpenJDK 13 for some of the tests. The speed difference is greatest for the tests that require pixel conversion into an 8-bit or 16-bit frame buffer. I plan to investigate the cause of the performance difference.

• In general, the faster processor and memory bus of the newer models does not fully compensate for the threefold increase in the number of pixels in their displays. The table below shows the animation speed of each model in milliseconds per frame and frames per second.

  Product Name	Speed (ms)	Rate (fps)
  Kobo Touch B	125	8.0
  Kobo Touch C	130	7.7
  Kobo Glo HD	140	7.1
  Kobo Clara HD	145	6.9

---

Progress

• Change must not contain extraneous whitespace
• Commit message must refer to an issue
• Change must be properly reviewed

Issue

• JDK-8227425: Add support for e-paper displays on i.MX6 devices

Reviewers

• Johan Vos (jvos - Reviewer)
• Kevin Rushforth (kcr - Reviewer)

Download

$ git fetch https://git.openjdk.java.net/jfx pull/60/head:pull/60
$ git checkout pull/60

[bridgekeeper]

Hi jgneff, welcome to this OpenJDK project and thanks for contributing!

We do not recognize you as Contributor and need to ensure you have signed the Oracle Contributor Agreement (OCA). If you have not signed the OCA, please follow the instructions. Please fill in your GitHub username in the "Username" field of the application. Once you have signed the OCA, please let us know by writing /signed in a comment in this pull request.

If you already are an OpenJDK Author, Committer or Reviewer, please click here to open a new issue so that we can record that fact. Please use "Add GitHub user jgneff" as summary for the issue.

If you are contributing this work on behalf of your employer and your employer has signed the OCA, please let us know by writing /covered in a comment in this pull request.

[jgneff]

/signed

[bridgekeeper]

Thank you! Please allow for up to two weeks to process your OCA, although it is usually done within one to two business days. Also, please note that pull requests that are pending an OCA check will not usually be evaluated, so your patience is appreciated!

[mlbridge]

Webrevs

• 02: Full - Incremental (ba668b5)
• 01: Full - Incremental (f7028e8)
• 00: Full (aa33524)

[kevinrushforth]

/reviewers 2

[openjdk]

@kevinrushforth
The number of required reviews for this PR is now set to 2 (with at least 1 of role reviewers).

[jgneff]

@kevinrushforth Did I merge the upstream master branch correctly? The last time I did this on the old openjdk-jfx repository, the pull request showed only the single merge commit. This time all 87 commits from the upstream master branch are shown as part of this pull request, even though the code I propose has not changed since the pull request was created on December 4, 2019. Let me know if you would like me to rebase my changes and force push a cleaner update.

While I have your attention ... would you be able to be my second reviewer? This pull request is a small update that completes the large project begun by JDK-8217605 over a year ago. Your comments on that first part were very helpful, so I'm hoping you can bring that experience to this second and final part of the project.

[kevinrushforth]

/reviewers 2

[openjdk]

@kevinrushforth
The number of required reviews for this PR is now set to 2 (with at least 1 of role reviewers).

[kevinrushforth]

Let me know if you would like me to rebase my changes and force push a cleaner update.

Yes, this is what you will need to do.

More than just being needed for a cleaner update, the PR lists 6,151 changed files! Either the merge went badly or it has somehow managed to confuse GitHub (my money is on the latter based on what I see in the diffs).

Either way, rebasing and force-pushing your branch is the way to go. You might need to export your patch and reset your branch to master if git has trouble rebasing.

[kevinrushforth]

I just checked and there is nothing wrong with your merge. But somehow it managed to confuse GitHub.

[jgneff]

I just checked and there is nothing wrong with your merge. But somehow it managed to confuse GitHub.

Thanks, Kevin. A simple git rebase master and git push --force did the trick!

I hope it's okay for me to add some advocacy to the conversation here. My initial comment makes it easy to miss the general context of this update.

This pull request completes the project of adding support for e-paper displays that we started back in JavaFX 13. I would love to see these last few changes merged in time for JavaFX 15.

Below are five reasons to merge this pull request into the next release:

1. It's small. It's only 50 lines of code — the last 3 percent of the entire project.

2. It's done. It completes the work already included in JavaFX 13, which was 1,812 lines of code and 97 percent of the project.

3. It's safe. The code runs only when the system properties include glass.platform=Monocle and monocle.platform=EPD, and even then, only when the system has an e-paper display with the EPD Controller frame buffer driver. There are no changes to any JavaFX code outside of the Monocle EPD platform.

4. It's tested. I tested the code in November with JDK 13 and JavaFX 14. I tested the code again this week with JDK 14 and JavaFX 15. I ran tests with Ubuntu OpenJDK 11 and 13 as well as AdoptOpenJDK 11 and 14. I tested older devices under Ubuntu 14.04 and newer devices under Ubuntu 20.04 Beta. All tests passed.

5. It's unique. This pull request makes JavaFX the only cross-platform application framework with built-in support for e-paper displays. It demonstrates that the Java slogan "write once, run anywhere" is as true today as it was in 1995, not just for different operating systems and processor architectures, but even for radically different display technologies like electronic ink.

To my potential reviewers, @johanvos and @kevinrushforth, is there anything I can do to make your code review easier?

[kevinrushforth]

Looks good to me.

[openjdk]

@jgneff This change now passes all automated pre-integration checks. When the change also fulfills all project specific requirements, type /integrate in a new comment to proceed. After integration, the commit message will be:

8227425: Add support for e-paper displays on i.MX6 devices

Reviewed-by: jvos, kcr

• If you would like to add a summary, use the /summary command.
• To credit additional contributors, use the /contributor command.
• To add additional solved issues, use the /solves command.

Since the source branch of this PR was last updated there have been 26 commits pushed to the master branch:

• e30049f: 8242077: Add information about HTTP/2 and HttpClient usage in WebEngine
• e0ffca3: 8242505: Some WebKit tests might fail because Microsoft libraries are not loaded
• ceb3fce: 8087555: [ChoiceBox] uncontained value not shown
• 818ac00: 8175358: Memory leak when moving MenuButton into another Scene
• 91d4c8b: 8241737: TabPaneSkin memory leak on replacing selectionModel
• 48476eb: 8241582: Infinite animation does not start from the end when started with a negative rate
• dedf7cb: 8242490: Upgrade to gcc 9.2 on Linux
• 5e9fb82: 8242577: Cell selection fails on iOS most of the times
• 69e4266: 8242489: ChoiceBox: initially toggle not sync'ed to selection
• 1d88180: 8243112: Skip failing test SVGTest.testSVGRenderingWithPattern
• ... and 16 more: https://git.openjdk.java.net/jfx/compare/4d69a0d2a5c61b67dd671b582eee9e06dc072a0b...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid automatic rebasing, please merge master into your branch, and then specify the current head hash when integrating, like this: /integrate e30049f0443f12547993ad7285faae9868821b3a.

As you are not a known OpenJDK Author, an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@johanvos, @kevinrushforth) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

[jgneff]

/integrate

[openjdk]

@jgneff
Your change (at version ba668b5) is now ready to be sponsored by a Committer.

[kevinrushforth]

/sponsor

[openjdk]

@kevinrushforth @jgneff The following commits have been pushed to master since your change was applied:

• e30049f: 8242077: Add information about HTTP/2 and HttpClient usage in WebEngine
• e0ffca3: 8242505: Some WebKit tests might fail because Microsoft libraries are not loaded
• ceb3fce: 8087555: [ChoiceBox] uncontained value not shown
• 818ac00: 8175358: Memory leak when moving MenuButton into another Scene
• 91d4c8b: 8241737: TabPaneSkin memory leak on replacing selectionModel
• 48476eb: 8241582: Infinite animation does not start from the end when started with a negative rate
• dedf7cb: 8242490: Upgrade to gcc 9.2 on Linux
• 5e9fb82: 8242577: Cell selection fails on iOS most of the times
• 69e4266: 8242489: ChoiceBox: initially toggle not sync'ed to selection
• 1d88180: 8243112: Skip failing test SVGTest.testSVGRenderingWithPattern
• ec8608f: 8223298: SVG patterns are drawn wrong
• e82046e: 8242530: [macos] Some audio files miss spectrum data when another audio file plays first
• 7044cef: 8241476: Linux build warnings issued on gcc 9
• 9d50c4c: Merge
• ef37669: Merge
• f2bca9f: Merge
• 6900d29: Merge
• e91bec4: Merge
• 66a8f49: Merge
• fde42da: Merge
• e21fd1f: Merge
• 443c845: Merge
• 31e63de: Merge
• 14c6938: 8236798: Enhance FX scripting support
• bfb2d0e: Merge
• 39f6127: Merge

Your commit was automatically rebased without conflicts.

Pushed as commit 66c3b38.

[jgneff]

Below are some of the more interesting test results.

• The OpenJDK 11 package ... runs twice as fast as the AdoptOpenJDK build of OpenJDK 13 for some of the tests. ... I plan to investigate the cause of the performance difference.

This is just a note to follow up on that performance problem I saw in December. I think I found the cause of the problem, and it appears to be fixed in JDK 14 and 15.

The chart below shows the drastic drop in performance before the fix.

The following flame graph, created by async-profiler, led quickly to an existing Java bug report.

See the description at JDK 13 Performance for details. Thank you for the quick fix, @voitylov!