Can't open djvu files on Android

[Germanc]

Hi ! I managed to get koreader working through patch.lua on my Boris reader (Argentinian Onyx c67ml rebranded) but I can't open any djvu file with error 'no reader engine for this file'. There are no not ascii character in the path (I checked). The reader runs Android 4.2.2 with Play Services. I'm running latest nightly version but I tested with stable and a previous nightly with no success.
How can I get a log ? If you give me the adb command I will manage to run it (never coded on android). Would love to use it as my main reader after years of using kindlepdfviewer.

Update: I got this
I/luajit-launcher( 1241): # opening file /mnt/sdcard/Books/Physics/FT2/Libros/Sakurai J.J.-Modern Quantum Mechanics (1993).djvu
I/luajit-launcher( 1241): # cannot open document /mnt/sdcard/Books/Physics/FT2/Libros/Sakurai J.J.-Modern Quantum Mechanics (1993).djvu frontend/document/djvudocument.lua:30: module 'libs/libkoreader-djvu' not found:
I/luajit-launcher( 1241): no field package.preload['libs/libkoreader-djvu']
I/luajit-launcher( 1241): no file 'libs/libkoreader-djvu.lua' (checked with asset loader)
I/luajit-launcher( 1241): no file 'common/libs/libkoreader-djvu.lua'
I/luajit-launcher( 1241): no file 'rocks/share/lua/5.1/libs/libkoreader-djvu.lua'
I/luajit-launcher( 1241): no file 'frontend/libs/libkoreader-djvu.lua'
I/luajit-launcher( 1241): no file 'libs/libkoreader-djvu.lua'
I/luajit-launcher( 1241): no file '/data/data/org.koreader.launcher/files/libs/libkoreader-djvu.lua'
I/luajit-launcher( 1241): no file 'common/libs/libkoreader-djvu.so'
I/luajit-launcher( 1241): no file 'common/libs/libkoreader-djvu.dll'
I/luajit-launcher( 1241): no file '/usr/lib/lua/libs/libkoreader-djvu.so'
I/luajit-launcher( 1241): no file 'rocks/lib/lua/5.1/libs/libkoreader-djvu.so'
I/luajit-launcher( 1241): no file 'libs/libkoreader-djvu.so'
I/luajit-launcher( 1241): no file '/data/data/org.koreader.launcher/files/libs/libkoreader-djvu.so'
I/luajit-launcher( 1241): # show widget

How can I provide more info ?

[hwhw]

Hm, I think currently the djvu module isn't build on Android as djvulibre needed some parts that were not provided by Android's userland API (Bionic libc). I guess somewhere on the net patches are floating around for this issue?

[Germanc]

This patch is floating around some chinese forums so koreader doesn't crash all the time and to remove the black bar at the bottom. With this it works quite well (it crashes sometimes but it's mostly usable):

require("jit.opt").start("sizemcode=64","maxmcode=512", "recunroll=8", "callunroll=12", "loopunroll=60", "instunroll=16", "tryside=16", "hotexit=160", "hotloop=1792", "maxsnap=160000", "maxside=32000", "maxirconst=160000", "maxrecord=1280000", "maxtrace=320000")
local screen = require("android").screen
screen.height = screen.height + 80
for i=1,1000 do end -- Force allocation of one large segment

If there is no djvu support on android yet I guess I will have to wait. I will convert my most used books to pdf then since the reflow works really well compared to the stock reader (which seems to also implement k2pdfopt but with no pre render of the next page).
I will be waiting for djvu support then !

[chrox]

Yes, djvulibre needs some patches to compile with android NDK. But it's possible since some open source projects like ebookdroid and orion viewer have already done that.

[gerroon]

Hi

Any progress on this issue?

thanks

[Frenzie]

I'll quote this new development from the chat for simple future reference. Fixing this seems more straightforward than I anticipated (because over at Orion Viewer they've already done all the hard work). Here's the blueprint I put together that lays out everything I can anticipate in advance.

https://gitter.im/koreader/koreader?at=598841a745fc67074600725d

@marmistrz
10:55
Hi! I'd like to help with issue #1534
Can you help me where to start?
Frans de Jonge
@Frenzie
12:32

@marmistrz The simple part would be to enable the build, see https://github.com/koreader/koreader-base/blob/d197047f88c37c1b10116730377b0251e6f91b83/Makefile#L74

The hard part starts there: figuring out why it was disabled in the first place and how to fix it. 😉

However, the first thing I'd check if things go wrong is simply if a bump automagically fixes it: https://github.com/koreader/koreader-base/blob/c6c1bd512dfaf22bdf10c9f8a1a74665b27b78e0/thirdparty/djvulibre/CMakeLists.txt#L28

There hasn't been a new stable since 2015, but the djvulibre Git repo has received a few commits since then.

I hope that helps! 😀

For simple patches you can use patch files like this: https://github.com/koreader/koreader-base/blob/c6c1bd512dfaf22bdf10c9f8a1a74665b27b78e0/thirdparty/mupdf/CMakeLists.txt#L56-L59

(but don't forget to contribute 'em upstream 😛)
Frans de Jonge
@Frenzie
12:43
I was just looking at Orion Viewer and it looks like they simply overwrite some of the libdjvu files with their own patched versions: https://github.com/max-kammerer/orion-viewer/tree/10eb69cdef36c00d64672bc63237f7ee8698b756/orion-viewer/jni/djvu/libdjvu
https://github.com/max-kammerer/orion-viewer/blob/1c62a273499525528a40e3df21e51d6b017157d9/thirdparty_build.gradle#L22-L31
So yeah @marmistrz if you could put the puzzle pieces together that'd be very helpful. 😀

[Frenzie]

I was taking a look at this and there's an annoying combination of circumstances making this harder than I thought it would be.

Android doesn't support full pthreads: http://mobilepearls.com/labs/native-android-api/#pthreads

libdjvu stripped out the ability to use NOTHREADS: https://sourceforge.net/p/djvu/djvulibre-git/ci/7d68e39b307e94ad58a4a53688eeac4eebcde8ce/tree/libdjvu/GThreads.h?diff=cc34dbefd8499ba14b3391a1f618b5f31a8c0bc3&diformat=sidebyside

https://sourceforge.net/p/djvu/bugs/249/

If you really want to try without threads, take the 3.5.25.3 code base
and remove the three lines involving MACTHREADS in GThreads.h. Then it will
compile. But it will not run correctly in all circumstances. Only the simplest
djvu files can be decoded without threads...

PS For the most part it might not be too hard to slap together something that works, but I'd like to do it as properly as possible including upstream patch submission. For example, that means using something like this at the top of DjVuToPS.cpp

#ifdef __ANDROID__
# undef UNIX
#endif

Or maybe something like #if defined(UNIX) && ! defined(__ANDROID__). But in any case, not commenting it out. :-)

In our CMakeLists.txt we can add set(CFLAGS "${CFLAGS} -Wunused-variable") for Android.

Some other outdated notes can be found here:

• https://github.com/smyts/tengwa-djvu/blob/master/tengwa-djvu/jni/djvulibre/INSTALL-ANDROID
• and also https://github.com/max-kammerer/orion-viewer/blob/10eb69cdef36c00d64672bc63237f7ee8698b756/orion-viewer/jni/djvu/android/jni/Core.mk (but it does some things like adding HAVE_WCHAR_H and such that are done automatically; maybe an older NDK issue?)

PS I forgot to mention, this is the error in question:

GThreads.cpp: In member function 'void DJVU::GThread::terminate()':
GThreads.cpp:453:24: error: 'pthread_cancel' was not declared in this scope
     pthread_cancel(hthr);
                        ^
make[7]: *** [libdjvulibre_la-GThreads.lo] Error 1

Maybe it could be changed to pthread_exit?

[Frenzie]

I was browsing around a little and noticed that in EBookDroid they don't cancel the threads at all: https://github.com/mortenpi/ebookdroid/blob/e9a18b0842088d956c297aeb6697441a6b92c5fd/jni/djvu/GThreads.cpp#L474-L477

GThread::terminate()
{
#ifndef __ANDROID__
  if (xentry || xarg)
    pthread_cancel(hthr);
#endif
}

It'd create a working build, though I fear about memory leaks. :-)