What is the pagination strategy in Readium-2? #10

rkwright · 2016-09-21T18:49:51Z

This is a Discussion

What is the pagination strategy?
CSS columns have several problems
Per platform or cross-platform?

rkwright · 2016-09-21T18:52:00Z

From Jean-Marie:
Probably per-platform.
On iOS it would be great to get some insurance from Apple that the paginationMode that has become private in WkWebview (but can be used) will not disappear… Is this realistic?

On Android the situation is more complex ;-)

rkwright · 2016-09-21T18:53:49Z

From Jean-Marie:

Here is the code snippet that does the trick:

<body style="overflow:-webkit-paged-x; overflow:paged-x;">

It flies!!!
Tested on huge pages on Android devices. Instant display, blazing-fast horizontal scroll.

It was implemented in 2012, I didn't know that. https://trac.webkit.org/changeset/126343

rkwright · 2016-09-21T18:54:18Z

From Juan Corona:
I'm for the the per-platform approach. But one of my concerns is with the increased overhead supporting multiple strategies for serializing/deserializing page locations, figuring out visible page ranges, and all the related code we wrote in the cfi_navigation_logic monolith. Although if we can keep things modular and with well designed interfaces I can see this burden being eased.

rkwright · 2016-09-21T18:56:38Z

From Jean-Marie:

seems to work like a charm (very fast) on Android Chromium (available since 4.4 and even before for those who embed a Chromium-based WebView) and on iOS too. Any known drawback? My first tests cumulating this pagination and multi columns (are not very positive (it works but thare are small scrolling issues on iOS and viewport issues on Android), but we can dig [into them] if nobody already has negative conclusions.

Edit: I see that this option was discussed back in 2014...

rkwright · 2016-09-21T18:57:55Z

From Juan Corona:
I wasn't part of the discussion back in 2014, but if this is a viable strategy for Android then it sounds great.

rkwright · 2016-09-21T18:59:04Z

From Juan Corona:
CSS scroll snap points provide a nice scrolling experience, as if flipping through pages. https://www.chromestatus.com/feature/5721832506261504
It’s supported everywhere but Chrome and IE (but is shipped in Edge!)
Demo (vertical, but can be horizontal): https://blog.hospodarets.com/demos/scroll-snap-full-screen

rkwright · 2016-09-28T18:05:33Z

Using overflow-paged sounds very promising. However, to be viable it needs to work on all platforms we support (if at all possible). It is much more complex to have one solution for MOST platforms and one or two others on other platforms.
We need some research into how works across different platforms and versions. Also, is this a feature that the browser vendors will continue to support or will they drop it just as we finish R2?

dauwhe · 2016-09-28T18:12:42Z

I've made little demos using overflow: -webkit-paged-x with scroll snapping. It's a lovely experience, requiring three lines of CSS:

overflow: -webkit-paged-x;
-webkit-scroll-snap-type: mandatory;
-webkit-scroll-snap-points-x: repeat(100%);

But AFAIK it's only working publicly in Desktop Safari. Sadly, I haven't heard of interest from other browsers for this.

rkwright · 2016-09-28T18:15:16Z

@dauwhe Can you share your demos?

dauwhe · 2016-09-28T18:17:12Z

http://dauwhe.github.io/epub-zero/acme-publishing/MobyDickPaged/

jmgeffroy · 2016-09-28T18:24:37Z

Right. Theoretically it should work on all webkit-based webviews... It seems to be present at least since 2012 in WebKit (See this description).
We should probably define a fallback strategy however, for browsers that don't support it or don't support it well...
PS: - Just for fun - While doing some random "googling" on this topic I found a funny paper to read later... They transform the page with XSLT to break the page into pages, so preserving the CFIs with this technique is probably a little bit tricky ;-) But this is probably an entry point to the world of of pagination, and the References section contains some interesting stuff (...for insomniacs ;-))

rkwright · 2016-09-28T18:26:26Z

@dauwhe Thanks. I see the pagination layout but how is the snap-navigation supposed to work?Seems to simply lay out the "pages" then you have to manually move the viewport so a given page is in view. Tested in Safari 10.0 on OSX.

jmgeffroy · 2016-09-28T18:27:40Z

@dauwhe According to my tests, the paged-x works fine on my Android Webkit-based webviews and on iOS. Snap points don't.

dauwhe · 2016-09-28T18:30:50Z

@rkwright if I do the horizontal two-finger gesture on my trackpad, the "page" slides and then sticks to the snap point. It's very much like page-swipe gestures in many EPUB reading systems.

One bad thing is that, at least in my file, when I go to a page via a TOC link, it doesn't automatically snap to the page boundary. I have to nudge it.

HadrienGardeur · 2016-09-28T18:37:21Z

Works fine for me on macOS with Safari, no snap points with Chrome though.

rkwright · 2016-09-28T18:37:37Z

@dauwhe Ah so. Yes, that works. But clearly there are a bunch of wrinkles to this approach. But if it can be made to work it would be a great boon. And after all, WebKit is open source.

rkwright · 2016-09-28T18:48:41Z

Shock! It doesn't work in Edge. And last time I looked, Edge wasn't open source...

jmgeffroy · 2016-09-28T18:52:34Z

I thought it was, look :-D

jmgeffroy · 2016-09-28T19:00:50Z

Interesting reference about Edge

rkwright · 2016-09-28T19:28:45Z

@jmgeffroy Huh. I hadn't stumbled on that. But I don't see the source code. Still, interesting.

jmgeffroy · 2016-09-28T19:34:10Z

@rkwright The source code isn't there but it contains interesting stuff and pointers. It's on this Github repo that I found the link to the Edge Platform Status page, for example. It's also probably interesting to look at the tests. It's reading the Webkit tests that I discovered the overflow: paged-x, for example. At least, it's easier to read than the complete source code in a first approach...

llemeurfr · 2016-09-29T12:46:30Z

At the W3C TPAC, CSS Houdini was cited as a possible great solution for our pagination problem. It would be a DIY solution, where we create a pagination CSS polyfill in JS (more that a polyfill, a real new thing).

More specifically, I was advised by A.Stearns (Adobe), co-chair of the CSSWG, that:

readium architecture folks should consult https://drafts.css-houdini.org/css-typed-om/ and http://wicg.github.io/CSS-Parser-API/ and give feedback on whether they would be useful to you.

The new liaison between the publishing industry and the W3C could help us getting more from the browser vendors.

rkwright · 2016-09-29T15:53:41Z

Of course, there is also:
http://sorotokin.com/adaptive-layout/
The CSS working group reviewed it several years ago. Some liked it, some didn't. But it is very nice, very full-featured and works.

dauwhe · 2016-09-29T15:57:28Z

@rkwright Vivliostyle has used Peter's work as the basis for their stuff. https://github.com/vivliostyle

llemeurfr · 2016-09-29T17:05:39Z

Note that I met Florian Rivoal from Vivliostyle at TPAC, we talked about their work, achieved from sorotokin's work. They have some open-source software they could share.

olivierkorner · 2016-10-11T12:59:46Z

FYI the CSS paged-x property doesn't seem to work in Safari on iOS (version 10).

HadrienGardeur · 2016-10-11T13:01:48Z

You should always include both paged-x and -webkit-paged-x in your test.

olivierkorner · 2016-10-11T13:26:23Z

Well support looks buggy: I was indeed able on iOS to "page" some content by adding the CSS to a simple HTML page, but @dauwhe's example displays a very long iframe on my iPad:

HadrienGardeur · 2016-10-19T00:14:28Z

I've also created a demo for paged-x: https://hadriengardeur.github.io/webpub-manifest/examples/pagination/overflow-paged-x/

Just a few notes about the demo:

it detects clicks on left/right on desktop and Chrome Android in order to navigate to the next page
this is still buggy on mobile Safari (page turn), will need some extra work from me
since taps do not work yet in mobile Safari, scrolling is still allowed in this demo

My main concern regarding -webkit-paged-x is that I can't find a single way to set a top and bottom margin on overflowed content (page 2+).

jmgeffroy · 2016-10-19T01:15:58Z

Top and bottom margin can easily be outside the overflowed area.
3 obvious "pros" of this solution wrt multi-column are simplicity, speed and the fact that it doesn't break multi-columned layouts. Note that I'm still studying other solutions, so I'm not pushing it as the only way to go.
Peter Sorotokin's adaptive layout is impressive (as always) but I don't know how fast it can be.
BTW more generally I think that pagination strategy is probably one of the areas that should be normalized as much as possible across all implementations and integrations if we still want to fulfill the promise of "providing a unified Epub 3 rendering"...

HadrienGardeur · 2016-10-19T10:00:50Z

@jmgeffroy that means introducing a synthetic margin (top/bottom) that won't respect the publication's CSS. Sure that works, but it's slightly inconsistent with the author's intent.

BTW more generally I think that pagination strategy is probably one of the areas that should be normalized as much as possible across all implementations and integrations if we still want to fulfill the promise of "providing a unified Epub 3 rendering"...

Do you mean normalized across apps on the same platform or normalized across different platforms?

rkwright · 2016-10-19T14:37:44Z

Peter's adaptive layout was (is) very performant. The more serious problem was that it had to essentially take over CSS layout from the browser. Even so it was performant (Peter at his best) but IIRC it did interfere with accessibility (or at least was never optimized to support accessibility properly).

My larger concern in this area (and with paged-x in particular) is having a consistent solution across the supported platforms.

HadrienGardeur · 2016-10-19T15:57:39Z

I've created a more advanced demo (limited to desktop Chrome/Safari/Opera and Chrome on Android for now) that does the following:

pagination both within an HTML and between documents
offline through a Service Worker
three tap areas (previous page, menu and next page)

It's entirely based on "overflow: paged-x" with very little JS/CSS injected in the iframe.

It's available at https://hadriengardeur.github.io/webpub-manifest/examples/paged-viewer

HadrienGardeur · 2016-10-20T05:08:42Z

Just a quick update based on something that @rkwright said yesterday and that @olivierkorner mentioned earlier in this thread:

"overflow: paged-x" works fine if applied to a document directly on iPad, you can try it at https://hadriengardeur.github.io/webpub-manifest/examples/pagination/overflow-paged-x/
but when applied on an iframe, the content is not properly paginated: the width is stretched to the proper size, but the text is not divided into pages/columns as you can see on @dauwhe example (https://dauwhe.github.io/epub-zero/acme/main.html?manifest=MobyDick/manifest.json) or my recent experiment (https://hadriengardeur.github.io/webpub-manifest/examples/paged-viewer)

Overall, this means that while "overflow: paged-x" works really quite well on Webkit-based browsers on either desktop (Chrome, Opera, Safari) or on Android (Chrome), it's not supported at all in Edge/Firefox and doesn't behave properly in an iframe on iOS.

In order to continue these tests with various techniques for pagination, I'd like to set up a test environment that would have the following components:

a basic viewer written in JS (I've been using https://github.com/HadrienGardeur/webpub-viewer) with the ability to quickly switch between pagination modules
a set of pagination modules, each module has one JS and one CSS that is injected in the content document
a collection of EPUB files that we can use to test these modules (should we use the standard EPUB3 test documents?)
a server component that will serve the resources and the manifest of these EPUB documents in HTTP/HTTPS

I've also listed the following things that IMO have an impact on pagination and aren't mentioned as often in our discussions:

Do we use an iframe? Sandboxed?
Can we inject the JS+CSS directly in the content document without using an iframe?
What's the impact on author and user stylesheets when we inject such JS/CSS?
How does the accessibility options of each platform behave?
How do we capture the current position in a publication? How do we navigate back to this location? (This should also cover the use cases for annotations and search)

There are also a number of pagination solutions for which we can't do the implementation strictly in JS: using multiple webviews, private APIs on WKWebview etc.
Should we also test some of these solutions? Do we have a good idea how various RS actually implement pagination (iBooks, Kindle, Kobo, Google, Scribd...)?

winniequinn · 2016-10-25T23:46:32Z

I'm just starting to play with the idea of not using iframes at all. They cause issues with paged-x (at least on mobile), they can get in the way when using VoiceOver or TalkBack (as the browser sees the iframe as another container to be navigated into), et cetera.

The whole point of using an iframe, at least as far as I know, is to be able to maintain an outer container for the reading system with the iframe being responsible for holding the contents of the book. This certainly makes sense if the goal is to be able to move between different chapters of a book all within the browser as is presently the case with Readium.

At least on mobile though, I think another solution is possible. We could get rid of the iframe, make the browser responsible only for showing a single chapter, and then load up other chapters in different web views that are then swapped into view as appropriate by the application-layer code written in Java, Swift, et cetera. This approach would seem to eliminate the need for an iframe, would make precaching adjacent spine items very simple, and would allow more work to be lifted out of the land of JavaScript and brought into the application layer.

It may well be possible to use the same approach on the desktop as well. Electron has a web view component that presumably could be used to the same effect as WKWebKit and android.webkit.WebView. I need to look into this a bit further.

rkwright · 2016-10-26T15:09:33Z

@winniequinn This is not a bad idea (at all). Iframes are certainly not ideal. However, trying to put separate spine items in separate containers has been tried more than once. See the prefetch branch in shared-js. One of the major problems encountered is NOT playing audio and video (as well as scripting) before the web-view is "brought into view". Both EvidentPoint and Bluefire experimented with it, but were unable to resolve it satisfactorily. But YMMV... It WOULD be a good approach if the problems can be resolved.

HadrienGardeur · 2016-10-26T15:31:38Z

This feels like two separate discussions:

what the reader/container is and how we inject CSS/JS
how we handle navigation between spine items

I'm also in favour of either dropping the iframe or offering an alternative, as I've said before in #12 (comment)

How we actually handle different spine items is another question, probably worth having its own issue here.

danielweck · 2016-10-26T15:34:07Z

@winniequinn this is how other "i"-named reading systems do it.

In Readium-1 we have reader.html that effectively bootstraps the entire readium-shared-js subsystem (rendering engine), which lays out the EPUB content iframe hosts, and is responsible for injecting behaviours into spine items (Media Overlays, link hijacking, keyboard / input device intercept and forwarding, CSS styling, epubReadingSystem, MathJax, etc.).

Using native webviews instead of iframes would mean writing a large portion of the rendering code in native per-platform code, which I think is acceptable in the context of Readium-2.

Note that we did look into using Electron's special web frame, but this is basically an extended iframe with special features (still in the renderer process though).

danielweck · 2016-10-26T15:44:09Z

Continued iframe discussion:
#15

HadrienGardeur · 2016-11-03T15:50:15Z

During our weekly-call, @jccr volunteered to create a prototype based on CSS-columns in order to continue to explore and compare various options for pagination.

jccr · 2016-11-14T23:45:33Z

Added the example for CSS-columns on develop:
https://github.com/readium/readium-2/tree/develop/examples/pagination/columns

I see that @HadrienGardeur did the same for overflow: paged-x:
https://github.com/readium/readium-2/tree/develop/examples/pagination/overflow-paged-x

HadrienGardeur · 2016-11-15T16:12:40Z

Thanks @jccr that's very helpful.

I'll try to adapt my WebPub Viewer code as soon as possible to provide the ability to quickly switch between pagination modes.
I'd like to roll out a full test suite where we can test a number of EPUB live and switch between different pagination modes.

dauwhe · 2016-12-06T03:17:04Z

Note I filed a bug on the webkit iframe issue: https://bugs.webkit.org/show_bug.cgi?id=165304

HadrienGardeur · 2016-12-06T11:17:46Z

@dauwhe thanks Dave !

In other news, I'd like to discuss the APIs available for the pagination module in the next call.
No matter what we decide to use in order to paginate, we could have a number of APIs that work across all implementations (even for native code).

Let's start with some basic stuff that we can all agree on and expand from there.

danielweck · 2016-12-14T16:52:32Z

@HadrienGardeur started that page: https://github.com/readium/readium-2/tree/master/doc/pagination

bluefirepatrick · 2017-03-14T15:53:01Z

The location of the document seems to have changed. This works for me:
https://github.com/readium/readium-2/tree/master/pagination

JayPanoz · 2017-08-06T12:55:50Z

For the (track) record, explored overflow: paged-x as soon as we started working on pagination for Readium CSS and it was quite unreliable. For some reason, blink’s and webkit’s implementations differ a lot, with Blink being more complete.

There are simply too many issues at the moment:

background-color doesn’t work in Safari if the element has overflow: paged-x declared, which is problematic for reading modes, while it works in Chrome;
Webkit doesn’t normalize scrollWidth while Chromes does. So only chrome sets the scrollWidth of the element as a multiple of the viewport width;
margins and paddings are completely screwed up in both implementations (i.e. they’re not based on page containers);
you get text cut-off in some cases;
you still need columns to emulate two-page spread;
I had to resolve to left: value to make it scroll everywhere, scrollLeft didn’t work with columns for some reason.

It is also my understanding that Opera’s intent was to use it to solve a lot of the column-box model issues and limitations (padding, margin, overflow, position: fixed|absolute etc.). I’m taking the Presto’s implementation (Opera Reader) as a reference there since it was feature complete.

So they used it with columns anyway, it was not designed as a replacement but more like a sanitizer for paged media.

HadrienGardeur · 2017-08-06T16:27:09Z

By the way @JayPanoz which page should we reference officially for the pagination strategy in Readium CSS?

JayPanoz · 2017-08-06T16:47:21Z

@HadrienGardeur Well, stylesheets are currently in branches (mapped to the project’s phases) as I can’t necessarily PR parts of the CSS as they come along e.g. pagination impacts user settings and vice versa. Besides, we still have to deal with scroll at the moment, which once again is impacting/impacted by user settings.

In other words, I’ll be able to PR when we have a fundamental system designed and then improve details.

But I can edit the Readium CSS wiki for reference in the meantime, especially as we now know how we’ll inject contents. That was on my TODO list and I’ll have to wait for basic user settings to be implemented in the iOS proto anyway so I can prioritize that tomorrow.

BTW, which would be the best repo to discuss the iOS proto issue about odd number of columns when two-column view? I may have an idea, which could also save us a lot of layout trashing, but Daniel could probably give a lot of insights about it as it would impact CFI at first sight.

JayPanoz · 2017-08-07T09:37:22Z

@HadrienGardeur So I guess you can reference this Wiki page as it sums up injection and pagination approaches.

I’ll add references to technical docs as soon as we push the stylesheets on master.

zhouhao27 · 2023-04-06T08:17:40Z

From Jean-Marie:

Here is the code snippet that does the trick:
<body style="overflow:-webkit-paged-x; overflow:paged-x;"> 
It flies!!! Tested on huge pages on Android devices. Instant display, blazing-fast horizontal scroll.

It was implemented in 2012, I didn't know that. https://trac.webkit.org/changeset/126343

It works. But I have another question. How to get total number of pages?

HadrienGardeur · 2023-04-06T08:33:10Z

We don't use "pages" in Readium because this concept only works for pre-paginated media such as EPUB FXL or PDF.

We use positions instead, which are computed when you first load an EPUB. You can request the total number of positions as a replacement for total number of pages.

rkwright added priority:Blocking Discussion labels Sep 21, 2016

danielweck mentioned this issue Oct 26, 2016

iframe vs. native webview to host EPUB content documents (spine items) #15

Closed

HadrienGardeur closed this as completed Mar 13, 2017

ahmedk92 mentioned this issue May 29, 2020

WKWebView Horizontal Paging ahmedk92/Blog#1

Open

What is the pagination strategy in Readium-2? #10

What is the pagination strategy in Readium-2? #10

Comments

rkwright commented Sep 21, 2016

This is a Discussion

rkwright commented Sep 21, 2016 • edited

rkwright commented Sep 21, 2016 • edited

rkwright commented Sep 21, 2016 • edited

rkwright commented Sep 21, 2016 • edited

rkwright commented Sep 21, 2016

rkwright commented Sep 21, 2016

rkwright commented Sep 28, 2016

dauwhe commented Sep 28, 2016

rkwright commented Sep 28, 2016

dauwhe commented Sep 28, 2016

jmgeffroy commented Sep 28, 2016 • edited

rkwright commented Sep 28, 2016

jmgeffroy commented Sep 28, 2016

dauwhe commented Sep 28, 2016

HadrienGardeur commented Sep 28, 2016

rkwright commented Sep 28, 2016

rkwright commented Sep 28, 2016

jmgeffroy commented Sep 28, 2016

jmgeffroy commented Sep 28, 2016

rkwright commented Sep 28, 2016

jmgeffroy commented Sep 28, 2016

llemeurfr commented Sep 29, 2016 • edited

rkwright commented Sep 29, 2016

dauwhe commented Sep 29, 2016

llemeurfr commented Sep 29, 2016

olivierkorner commented Oct 11, 2016

HadrienGardeur commented Oct 11, 2016

olivierkorner commented Oct 11, 2016

HadrienGardeur commented Oct 19, 2016

jmgeffroy commented Oct 19, 2016

HadrienGardeur commented Oct 19, 2016

rkwright commented Oct 19, 2016 • edited

HadrienGardeur commented Oct 19, 2016

HadrienGardeur commented Oct 20, 2016

winniequinn commented Oct 25, 2016

rkwright commented Oct 26, 2016

HadrienGardeur commented Oct 26, 2016

danielweck commented Oct 26, 2016 • edited

danielweck commented Oct 26, 2016

HadrienGardeur commented Nov 3, 2016

jccr commented Nov 14, 2016

HadrienGardeur commented Nov 15, 2016

dauwhe commented Dec 6, 2016

HadrienGardeur commented Dec 6, 2016

danielweck commented Dec 14, 2016

bluefirepatrick commented Mar 14, 2017 • edited

JayPanoz commented Aug 6, 2017

HadrienGardeur commented Aug 6, 2017

JayPanoz commented Aug 6, 2017

JayPanoz commented Aug 7, 2017

zhouhao27 commented Apr 6, 2023

HadrienGardeur commented Apr 6, 2023

rkwright commented Sep 21, 2016 •

edited

rkwright commented Sep 21, 2016 •

edited

rkwright commented Sep 21, 2016 •

edited

rkwright commented Sep 21, 2016 •

edited

jmgeffroy commented Sep 28, 2016 •

edited

llemeurfr commented Sep 29, 2016 •

edited

rkwright commented Oct 19, 2016 •

edited

danielweck commented Oct 26, 2016 •

edited

bluefirepatrick commented Mar 14, 2017 •

edited