Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash opening KOReader on Kindle legacy (K3) #6024

Closed
gargomoma opened this issue Apr 4, 2020 · 44 comments · Fixed by #6025 or koreader/koreader-base#1074
Closed

Crash opening KOReader on Kindle legacy (K3) #6024

gargomoma opened this issue Apr 4, 2020 · 44 comments · Fixed by #6025 or koreader/koreader-base#1074

Comments

@gargomoma
Copy link

gargomoma commented Apr 4, 2020

  • KOReader version: v2020.03.2-23-g2945a2f_2020-03-29
  • Device: legacy (Kindle 3)

Issue

After using previous version for a while (2019.2) I tried to update to the latest version with no luck.
I tested latest stable and nightly (also other 2019 releases), but none works. (Unless I rollback to 2019.02)
Tried clean and dirty install, but it keeps crashing while jumping on the File Browser.

At last but not least, many thanks for your work =)

Steps to reproduce

  • Clean Install latest KOReader on Kindle 3 (nightly or stable)

  • Run KOReader from KUAL:

  • Clean install:
    It will display the quickstart, but after closing it and trying to display the file browser it will crash.

  • Dirty install (or after first try with clean install):
    File browser will also crash.

crash.log

crash.log

@Frenzie
Copy link
Member

Frenzie commented Apr 4, 2020

Could this be related to Kindle dithering or something @NiLuJe ?

ffi.load: libs/liblept.so.5
ffi.load: libs/libk2pdfopt.so.2
Segmentation fault

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Hmm, nope, I'm fairly sure I have a 2020.something running on my end...

There was a drive-by build fix for... something during the PB TC switch, though, I'll double-check....

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Yeah, currently have 2020.01-28 running on the K3, will double-check newer builds ;).

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Huh.

Can't reproduce with 2020.03-23 (i.e., latest nightly) on a K3 on FW 3.4.3 setup in en_US.

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Okay, scratch that. It does crash as soon as I hit a folder with an ePub in it ;).

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Strangely enough, only happens in view modes with metadata parsing.

Classic mode is fine, and opening the book works just fine...

o_O

Any ideas, @poire-z? (I'll be trying a debug build to poke at the backtrace).

@poire-z
Copy link
Contributor

poire-z commented Apr 4, 2020

No idea. Try a CoverBrowser mode without cover image (Mosaic with text covers, Detailed...no images), to see if its thumbnail related.

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Nope, both of thoses crash, too. So, really metadata parsing :s.

@poire-z
Copy link
Contributor

poire-z commented Apr 4, 2020

So, Classic mode > Hold > Book information (where we get the metadata too) ?

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Maddeningly enough, that works ;).

(There's no "view cover fullscreen" entry, though).

@poire-z
Copy link
Contributor

poire-z commented Apr 4, 2020

Open that EPUB > Book status (which fetches the cover)?

@poire-z
Copy link
Contributor

poire-z commented Apr 4, 2020

(I have "Cover image" at the bottom of "Book information" , even in Classic mode.)

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

That works, too.


I don't, it's basically

copy | cut | paste | purge
cut | delete | rename
--
Open with | Book info
Add to fav

:?

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Bad news: it works with xtext disabled :s.

@gargomoma
Copy link
Author

Hi guys,
Thanks for your quick response.
I tried to see if it only crashes with ePub files, but for what it seems it also happens with Pdfs.
And the few that works do not have metadata at all. =/

Let me know if there's anything I can test on my Kindle that might help you.

BTW, I forgot to say on the first post but my Kindle has the same config as @NiLuJe "K3 on FW 3.4.3 setup in en_US".

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

(Yeah, FWIW, my test folder has one ePub and one PDF :D).

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

@poire-z: Weirder, still: it crashes with xtext enabled if there's metadata to populate, but if I do that without xtext, then re-enable xtext, it'll display everything just fine.

(With the requisite restarts in between, ofc).

o_O.

@poire-z
Copy link
Contributor

poire-z commented Apr 4, 2020

Well, backtrace plaese, so we can stop guessing :)

@poire-z
Copy link
Contributor

poire-z commented Apr 4, 2020

Didn't we have something similar, suspecting xtext/harfbuzz, and you solve it with switching something related to "atomic" somewhere ?

found it: koreader/koreader-base#1046

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Yep, that was koreader/koreader-base#1046

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

It doesn't crash in a debug build. -_-".

(Currently checking what happens with an unstripped release build).

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Oh, fun fact about my earlier "it works on 2020.01-28" comment: that was also a debug build :D (from #1046).

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Hmm, still appears to be HB shenanigans...

It's currently failing to parse encrypted AZW, and face-planting with something along the lines of

0x40cf0714 in hb_shape_plan_reference () from /mnt/us/koreader/libs/libharfbuzz.so.0
(gdb) bt full
#0  0x40cf0714 in hb_shape_plan_reference () from /mnt/us/koreader/libs/libharfbuzz.so.0
No symbol table info available.
#1  0x40cf9c40 in hb_shape_plan_create_cached2 () from /mnt/us/koreader/libs/libharfbuzz.so.0
No symbol table info available.
#2  0x40cf9d0c in hb_shape_full () from /mnt/us/koreader/libs/libharfbuzz.so.0
No symbol table info available.
#3  0x40cf9d5c in hb_shape () from /mnt/us/koreader/libs/libharfbuzz.so.0
No symbol table info available.
#4  0x40d72aac in ?? ()
No symbol table info available.
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

(That's an LTO build, so, slightly more aggressive optimizations than the build bots. Will check with a tamer build...).

@Frenzie
Copy link
Member

Frenzie commented Apr 4, 2020

Gah, debug builds not failing is the worst.

We don't use potentially unsafe optimizations, do we?

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Not really, I'm trying a few things, but even just swapping out HB/XText is not enough to make it behave.

The only sure-fire way to have it behave I have found so far is to disable xtext :/.

@poire-z
Copy link
Contributor

poire-z commented Apr 4, 2020

swapping out HB/XText is not enough != found so far is to disable xtext ?

still always crashing (segfault?) at hb_shape_plan_reference ? (an inline function that involves hb_atomic_int_t, that should have been wrapped by simple macros after koreader/koreader-base#1046...)

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

(I meant swapping out the release libs for debug libs. Still crashes. Haven't yet tried adding CRe (shim and/or actual lib) to the list).

Yep. I also got a slightly different bt once:

Program received signal SIGSEGV, Segmentation fault.
hb_shape_plan_execute (shape_plan=shape_plan@entry=0xc1ed7fac, font=font@entry=0x232ad8, buffer=buffer@entry=0x2340c0, features=features@entry=0x234170, num_features=num_features@entry=2) at hb-shape-plan.cc:391
391     hb-shape-plan.cc: No such file or directory.
(gdb) bt full
#0  hb_shape_plan_execute (shape_plan=shape_plan@entry=0xc1ed7fac, font=font@entry=0x232ad8, buffer=buffer@entry=0x2340c0, features=features@entry=0x234170, num_features=num_features@entry=2) at hb-shape-plan.cc:391
        __PRETTY_FUNCTION__ = "hb_bool_t hb_shape_plan_execute(hb_shape_plan_t*, hb_font_t*, hb_buffer_t*, const hb_feature_t*, unsigned int)"
#1  0x40951bb4 in hb_shape_full (font=0x232ad8, buffer=0x2340c0, features=0x234170, num_features=2, shaper_list=shaper_list@entry=0x0) at hb-shape.cc:139
        shape_plan = 0xc1ed7fac
        res = <optimized out>
#2  0x40951bf0 in hb_shape (font=<optimized out>, buffer=<optimized out>, features=<optimized out>, num_features=<optimized out>) at hb-shape.cc:169
No locals.
#3  0x409a9c7c in ?? ()
No symbol table info available.
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

But that one was pooossibly because I fudged my unzip command and I used the old 2020.01 debug zip I still had on hand :D.

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Okay, yeah, when I don't use a completely random zip I had laying around, just swapping out libharbuzz for a debug build "works".

>_<".

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

A plain old no fuss no muss -O2 -fomit-frame-pointer build also crashes.

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Note that -fomit-frame-pointer should be redundant (it's enabled as early as -O1 on a number of platforms, x86 for sure, ARM, too, I think).

Going to try -O1, which is closer to what -Og does in debug builds...

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

-O1 also crashes. -_-".

Swapping only HB for an -Og build still does the trick.

Will try reverting #1046, for science.

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Same bt with #1046 reverted.

Well, I guess it'll have to come to this: @poire-z how would you go about blacklisting xtext on a specific platform/device?

@poire-z
Copy link
Contributor

poire-z commented Apr 4, 2020

Might be as easy as forcing G_reader_settings:saveSetting("use_xtext", false) in the device detection code.
It is first used in reader.lua at Bidi.setup(lang_locale), after Device is available.
(and I wouldn't bother about updating the Dev menu...)

(Fine with me, dunno how popular Kindle legacy are among arabic readers that might miss it.)

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Well, given that it currently crashes instantly, nobody's going to be missing anything, really ^^.

It'll be 10 years old soon, it's basically a terrible device by today's standards.

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Fun fact: I tried a debug build with koreader/koreader-base#1046 reverted, and I realized that I don't even see the atomics stuff in the backtrace if I simply attach gdb to the process.

Letting the kernel create a coredump instead yields a slightly more verbose backtrace, where I do see the atomics stuff...

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Which means that, magic, with 1046 reverted, this also crashes in a fashion eerily similar to the one 1046 worked around...

Program terminated with signal SIGSEGV, Segmentation fault.
#0  hb_atomic_int_t::get_relaxed (this=0xc86b3fac) at hb-atomic.hh:267
267     hb-atomic.hh: No such file or directory.
(gdb) bt full
#0  hb_atomic_int_t::get_relaxed (this=0xc86b3fac) at hb-atomic.hh:267
No locals.
#1  hb_reference_count_t::is_inert (this=0xc86b3fac) at hb-object.hh:157
No locals.
#2  hb_object_is_inert<hb_shape_plan_t> (obj=0xc86b3fac) at hb-object.hh:246
No locals.
#3  hb_shape_plan_execute (shape_plan=shape_plan@entry=0xc86b3fac, font=font@entry=0x20b908, buffer=buffer@entry=0x20b970, features=features@entry=0x20ba20, num_features=num_features@entry=2) at hb-shape-plan.cc:391
        __PRETTY_FUNCTION__ = "hb_bool_t hb_shape_plan_execute(hb_shape_plan_t*, hb_font_t*, hb_buffer_t*, const hb_feature_t*, unsigned int)"
#4  0x40924bb4 in hb_shape_full (font=font@entry=0x20b908, buffer=buffer@entry=0x20b970, features=features@entry=0x20ba20, num_features=num_features@entry=2, shaper_list=shaper_list@entry=0x0) at hb-shape.cc:139
        shape_plan = 0xc86b3fac
        res = <optimized out>
#5  0x40924bf0 in hb_shape (font=font@entry=0x20b908, buffer=buffer@entry=0x20b970, features=features@entry=0x20ba20, num_features=num_features@entry=2) at hb-shape.cc:169
No locals.
#6  0x4097b598 in XText::measureSegment (hints=6, end=2, start=0, font_num=0, this=0x535060) at xtext.cpp:838
        len = <optimized out>
        glyph_pos = <optimized out>
        final_width = <optimized out>
        hcl = <optimized out>
        t_notdef_start = <optimized out>
        _hb_font = <optimized out>
        is_rtl = <optimized out>
        glyph_info = <optimized out>
        t_notdef_end = <optimized out>
        notdef_width = <optimized out>
        hb_data = <optimized out>
        _hb_buffer = <optimized out>
        hb_flags = <optimized out>
        cur_cluster = <optimized out>
        _hb_features = <optimized out>
        _hb_features_nb = <optimized out>
        glyph_count = <optimized out>
        hg = <optimized out>
        len = <optimized out>
        hb_data = <optimized out>
        _hb_font = <optimized out>
        _hb_buffer = <optimized out>
        _hb_features = <optimized out>
        _hb_features_nb = <optimized out>
        hb_flags = <optimized out>
        is_rtl = <optimized out>
        glyph_count = <optimized out>
        glyph_info = <optimized out>
        glyph_pos = <optimized out>
        final_width = <optimized out>
        cur_cluster = <optimized out>
        hg = <optimized out>
        hcl = <optimized out>
        t_notdef_start = <optimized out>
        t_notdef_end = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
        notdef_width = <optimized out>
        t = <optimized out>
        cur_width = <optimized out>
        advance = <optimized out>
        fb_hints = <optimized out>
        fallback_width = <optimized out>
        fb_hints = <optimized out>
        fallback_width = <optimized out>
#7  XText::measure (this=0x535060) at xtext.cpp:761
        end = <optimized out>
        w = <optimized out>
        end = <optimized out>
        w = <optimized out>
        hints = <optimized out>
        hints = <optimized out>
        end_of_text = <optimized out>
        line_break = <optimized out>
        end_of_text = <optimized out>
        line_break = <optimized out>
        bidi_level_changed = <optimized out>
        last_direction = <optimized out>
        script_changed = <optimized out>
        bidi_level_changed = <optimized out>
        last_direction = <optimized out>
        script_changed = <optimized out>
        i = <optimized out>
        i = <optimized out>
        lbCtx = {lang = 0x0, lbpLang = 0x0, lbcCur = LBP_AL, lbcNew = LBP_AL, lbcLast = LBP_SP, fLb8aZwj = false, fLb10LeadSpace = false, fLb21aHebrew = false, cLb30aRI = 0}
        final_width = <optimized out>
        prev_para_start = <optimized out>
        new_bidi_level = <optimized out>
        unicode_funcs = <optimized out>
        prev_script = <optimized out>
        start = <optimized out>
        last_bidi_level = <optimized out>
        lbCtx = <optimized out>
        final_width = <optimized out>
        prev_para_start = <optimized out>
        start = <optimized out>
        last_bidi_level = <optimized out>
        new_bidi_level = <optimized out>
        unicode_funcs = <optimized out>
        prev_script = <optimized out>
        i = <optimized out>
        end_of_text = <optimized out>
        bidi_level_changed = <optimized out>
        last_direction = <optimized out>
        script_changed = <optimized out>
        line_break = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
        script = <optimized out>
        ch = <optimized out>
        brk = <optimized out>
        pch = <optimized out>
        hints = <optimized out>
        end = <optimized out>
        w = <optimized out>
#8  XText_measure (L=<optimized out>) at xtext.cpp:2154
        xt = 0x535060
#9  0x0006e3a0 in lj_BC_FUNCC () at buildvm_arm.dasc:919
No locals.
#10 0x00064990 in lua_pcall (L=L@entry=0x401cd1c0, nargs=nargs@entry=1, nresults=-1, errfunc=errfunc@entry=2) at lj_api.c:1129
        g = 0x401cd1f0
        oldh = 0 '\000'
        ef = <optimized out>
        status = -932495444
#11 0x000145c0 in docall (L=L@entry=0x401cd1c0, narg=narg@entry=1, clear=clear@entry=0) at luajit.c:121
        status = <optimized out>
        base = 2
#12 0x00014f50 in handle_script (L=L@entry=0x401cd1c0, argx=argx@entry=0xbed39148) at luajit.c:292
        narg = 1
        status = <optimized out>
        fname = <optimized out>
#13 0x00015788 in pmain (L=0x401cd1c0) at luajit.c:553
        s = 0x871b8 <smain>
        argv = 0xbed39144
        argn = 1
        flags = 0
#14 0x0006e3a0 in lj_BC_FUNCC () at buildvm_arm.dasc:919
No locals.
#15 0x00064af8 in lua_cpcall (L=L@entry=0x401cd1c0, func=<optimized out>, ud=ud@entry=0x0) at lj_api.c:1153
        g = 0x401cd1f0
        oldh = 0 '\000'
        status = -932495444
#16 0x00015838 in main (argc=3, argv=0xbed39144) at luajit.c:582
        status = <optimized out>
        L = 0x401cd1c0

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

TL;DR: There's something rotten in the state of Denmark.

I'll stop banging my head on the wall and just disable xtext there.

Might be worth taking a look with the next HB version, but I wouldn't hold my breath.
I also wouldn't hold my breath for an updated GCC version, as I can't get anything newer to build against that shitty glibc version ;).

NiLuJe added a commit to NiLuJe/koreader-base that referenced this issue Apr 4, 2020
@poire-z
Copy link
Contributor

poire-z commented Apr 4, 2020

Another option to try would be to compile HB on K5 without all the atomic stuff (HB_NO_MT, or HB_TINY).
https://github.com/harfbuzz/harfbuzz/blob/master/CONFIG.md#thread-safety

NiLuJe added a commit to NiLuJe/koreader that referenced this issue Apr 4, 2020
Because apparently stuff blows up in weird and interesting ways for...
reasons.

Might be relevant on older PB devices, too?

Re koreader#5780
Closes koreader#6024
@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

IIRC, HB_NO_MT basically disables atomics and... possibly one other thing. (EDIT: mutexes. Duh.)

Granted, it's probably cleaner than my dirty, dirty patch.

I'll give it a go.

(FYI: K5 is unaffected, they're all running on armv7, like Kobos ;)).

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Hmm, HB_TINY appears to help...

Double-checking, and then checking w/ only HB_NO_MT :).

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Hmm, HB_TINY may be sneakily asking GCC/Clang to use -Os behind our backs, which is interestingly the only thing I hadn't explicitly tried...

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Yep, confirmed success w/ HB_TINY.

Now checking w/ HB_NO_MT and/or HB_NO_MT + -Os

@NiLuJe
Copy link
Member

NiLuJe commented Apr 4, 2020

Success w/ HB_NO_MT :).

NiLuJe added a commit to koreader/koreader-base that referenced this issue Apr 5, 2020
* Ditch thread-safety in HB on LEGACY & PB to avoid weird atomics issues on armv6

koreader/koreader#6024
koreader/koreader#5780

* Don't ship glib on Kindle

High kablooey potential, because we pull liblipc, which depends on it and
libgthread, so we'd be mixing glib versions between the two...

Regression since #1048
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants