Crash when trying to select / highlight several words in pdf files

[khariche]

I am sorry to say I still experience this issue on a BQ Cervantes 4. If I just long-press a single word, no popup window comes up, and if i directly try to select several words (I have direct highlight option enabled), the pdf file just closes and I find myself in the file manager.
Settings: pdf reflow, render quality default, without cropping and tried all zoom options, auto-save highlights on every page turn.
The only workaround I have found is to lower the "render quality" to low, but it is obviously not optimal. Also, when I change the render quality, old highlights that I deleted reappear with a weird contrast (less readable)...
Thanks for your help!

Here is the crashlog
03/14/20-00:28:21 INFO opening file /mnt/sd/agregation/sync/civilisation/Franquismo - memoria/CIVI-ESPAGNE-memoire-du-franquisme-cours-I-CNED.pdf
03/14/20-00:28:21 INFO RD loaded plugin SSH at plugins/SSH.koplugin
03/14/20-00:28:21 INFO RD loaded plugin autosuspend at plugins/autosuspend.koplugin
03/14/20-00:28:21 INFO RD loaded plugin autoturn at plugins/autoturn.koplugin
03/14/20-00:28:21 INFO RD loaded plugin backgroundrunner at plugins/backgroundrunner.koplugin
03/14/20-00:28:21 INFO RD loaded plugin batterystat at plugins/batterystat.koplugin
03/14/20-00:28:21 INFO RD loaded plugin calibrecompanion at plugins/calibrecompanion.koplugin
03/14/20-00:28:21 INFO RD loaded plugin coverbrowser at plugins/coverbrowser.koplugin
03/14/20-00:28:21 INFO RD loaded plugin docsettingtweak at plugins/docsettingtweak.koplugin
03/14/20-00:28:21 INFO RD loaded plugin evernote at plugins/evernote.koplugin
03/14/20-00:28:21 INFO RD loaded plugin goodreads at plugins/goodreads.koplugin
03/14/20-00:28:21 INFO RD loaded plugin keepalive at plugins/keepalive.koplugin
03/14/20-00:28:21 INFO RD loaded plugin kosync at plugins/kosync.koplugin
03/14/20-00:28:21 INFO RD loaded plugin newsdownloader at plugins/newsdownloader.koplugin
03/14/20-00:28:21 INFO RD loaded plugin perceptionexpander at plugins/perceptionexpander.koplugin
03/14/20-00:28:21 INFO RD loaded plugin readtimer at plugins/readtimer.koplugin
03/14/20-00:28:21 INFO RD loaded plugin send2ebook at plugins/send2ebook.koplugin
03/14/20-00:28:21 INFO RD loaded plugin statistics at plugins/statistics.koplugin
03/14/20-00:28:21 INFO RD loaded plugin systemstat at plugins/systemstat.koplugin
03/14/20-00:28:21 INFO RD loaded plugin terminal at plugins/terminal.koplugin
03/14/20-00:28:21 INFO RD loaded plugin texteditor at plugins/texteditor.koplugin
03/14/20-00:28:21 INFO RD loaded plugin timesync at plugins/timesync.koplugin
03/14/20-00:28:21 INFO RD loaded plugin wallabag at plugins/wallabag.koplugin
03/14/20-00:28:21 INFO RD loaded plugin zsync at plugins/zsync.koplugin
03/14/20-00:28:21 INFO setting zoom mode to page
ffi.load: pthread
GET https://sync.koreader.rocks:443/syncs/progress/8094b5516c593858808d365c27aa5a74
host or service not provided, or not known
PUT https://sync.koreader.rocks:443/syncs/progress
host or service not provided, or not known
PUT https://sync.koreader.rocks:443/syncs/progress
host or service not provided, or not known
Error in pixWordMaskByDilation: imin = 1 is too small
Error in pixConnComp: pixs not defined
Error in boxaSelectBySize: boxas not defined
Error in boxaSelectBySize: boxas not defined
Error in pixaCreateFromBoxa: boxa not defined
Error in boxaSort2d: boxas not defined
Error in pixaSort2dByIndex: pixas not defined
Error in pixaaFlattenToPixa: paa not defined
Error in pixaGetBoxa: pixa not defined
./luajit: frontend/document/koptinterface.lua:785: attempt to get length of local 'boxes' (a nil value)
stack traceback:
frontend/document/koptinterface.lua:785: in function 'getWordFromBoxes'
frontend/document/koptinterface.lua:1016: in function 'getTextFromPositions'
frontend/apps/reader/modules/readerhighlight.lua:705: in function 'handler'
frontend/ui/widget/container/inputcontainer.lua:255: in function 'handleEvent'
frontend/ui/uimanager.lua:736: in function 'sendEvent'
frontend/ui/uimanager.lua:324: in function 'default'
frontend/ui/uimanager.lua:1076: in function 'handleInputEvent'
frontend/ui/uimanager.lua:1137: in function 'handleInput'
frontend/ui/uimanager.lua:1181: in function 'run'
./reader.lua:298: in main chunk
[C]: at 0x00013f05

Originally posted by @khariche in #2954 (comment)

[pazos]

@khariche: test-case, please. (ie: pdf that you can share here, legally)

Also, Does the same crash happens without reflow?

[khariche]

Hi, thank you so much for your quick answer. The crash does not happen without reflow, in fact, like I said, it does not happen either with reflow on as long as I lower the" render quality" to low. Thus, it only happens when "render quality" is set to "default" or "high". I also noticed that changing the orientation to landscape or lowering the font size fixes the problem, but forces me to read at an uncomfortable size. Unfortunately, the documents I have had the problem with are not supposed to be shared or published, but here is just one of the pages, hoping it will provide enough data for a diagnosis.
On this particular page, one can see the second problem I mentioned in my original message : there is one part of the highlighted text which corresponds to a fragment for which I deleted the highlight, but it reappeared later on when changing the "render quality", with a more intense highlighting that "veils" the text and makes it less readable. The fragment I am referring to is the one starting with the word "perspectiva" and ending with "defensores".
test-case.pdf

[mergen3107]

What is your setting at Gear - Document - “Save document (write highlights into PDF)” ?
It looks like you have “Always” there, so it writes any highlights right away.

[khariche]

Hi, the setting is "Prompt", I changed it purposefully, because several files kept crashing, and I wanted to make sure that I had a confirmation that the file was being saved every time I closed it. However, I have also changed DAUTO_SAVE_PAGING_COUNT to 0 in defaults.persistent.lua file, as suggested in this thread #2152, so the highlights should indeed be saved on every page turn. However, the main problem is not with highlights not being saved (although they are indeed lost when the file crashes suddenly), but with being unable to do the highlighting in the first place (1- no popup window when long pressing a word, and 2- fil crash when sliding over several word to highlight them at once). As stated previously, lowering the the "render quality" fixes the issue. Thanks for your help.

…

On Sun, Mar 15, 2020 at 4:15 AM mergen3107 ***@***.***> wrote: What is your setting at Gear - Document - “Save document (write highlights into PDF)” ? It looks like you have “Always” there, so it writes any highlights right away. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5952 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALNZ4UOA6N2AUORATKZYIQLRHRB6VANCNFSM4LH3AHHQ> .

[pazos]

@khariche: the error thrown by k2opt is:

Error in pixWordMaskByDilation: imin = 1 is too small
Error in pixConnComp: pixs not defined
Error in boxaSelectBySize: boxas not defined
Error in boxaSelectBySize: boxas not defined
Error in pixaCreateFromBoxa: boxa not defined
Error in boxaSort2d: boxas not defined
Error in pixaSort2dByIndex: pixas not defined
Error in pixaaFlattenToPixa: paa not defined
Error in pixaGetBoxa: pixa not defined

I have no idea about the interface and couldn't reproduce the crash. Tried quickly on the emulator an on the C4. These days I don't have much free time 😭.

OTOH, the specific crash that you reported can be fixed with:

** updated with #5952 (comment) **

--- a/frontend/document/koptinterface.lua
+++ b/frontend/document/koptinterface.lua
@@ -782,7 +782,7 @@ end
 Get word and word box around `pos`.
 --]]
 function KoptInterface:getWordFromBoxes(boxes, pos)
-    if not pos or #boxes == 0 then return {} end
+    if not pos or not boxes or #boxes == 0 then return {} end
     local i, j = getWordBoxIndices(boxes, pos)
     local lb = boxes[i]
     local wb = boxes[i][j]

The whole koptinterface isn't nil-proof, so you might face other crashes after applying the patch.

[khariche]

Thank you for your time. How would I go about applying the patch? I have almost no knowledge of coding... Am I supposed to change the red text with the green one? In which file?
Another thing: Willus is really active on the k2pdfopt MR forum, do you think he would be able to help with this issue if I show him the error message?
Thanks again

[Frenzie]

The file is listed at the top. ;-)

The crash here probably isn't related to k2pdfopt, just to how we're interfacing with it.

@pazos That should probably be something more like if not pos or not boxes or #boxes == 0 then return {} end?

[khariche]

Hi, sorry for such a newbie question :)
I have tried with both suggestions:
if not pos or not boxes then return {} end
and
if not pos or not boxes or #boxes == 0 then return {} end

but I still get the same behavior : no popup on long press, crash on slide to highlight several words.
I have noticed that before closing the file, a few animated dots appear at the bottom of the screen, similar to when the device boots up.
Here it the crash log:
crash.log

[pazos]

but I still get the same behavior : no popup on long press, crash on slide to highlight several words.

the crash happens but the message is different. This time:

./luajit: frontend/document/koptinterface.lua:1021: attempt to index field 'box' (a nil value). As I said earlier the whole interface lacks safe-guards against nil boxes.

@khariche: you might want to reflow the pdfs on your computer using k2pdfopt and turn off reflow in KOReader until this gets fixed (if this gets fixed).

@Frenzie: Yeah!, boxes shouldn't be an empty table nor nil.

[khariche]

@khariche: you might want to reflow the pdfs on your computer using k2pdfopt and turn off reflow in KOReader until this gets fixed (if this gets fixed).

OK, thanks, I understand. The inconvenience with reflowing on pc with k2pdfopt as opposed to on-the-fly reflow with koreader is that the file cannot be returned to the "normal" layout for viewing on a pc...
Since it seems unlikely that this will get solved in the near future, is there a way to use highlighting on epubs that allows to view them in the files (so kohighlights is not an option) on pc? In other words, is there a pc app that can read koreader's .sdr folders (other than Koreader through bash, which is not really convenient)?
Many thanks

[poire-z]

Possibly solved by #6827. Re-open if not.