Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build fail on mac mojave #4375

Closed
Dollyn opened this issue Dec 3, 2018 · 19 comments · Fixed by #4400
Closed

build fail on mac mojave #4375

Dollyn opened this issue Dec 3, 2018 · 19 comments · Fixed by #4400

Comments

@Dollyn
Copy link

Dollyn commented Dec 3, 2018

  • KOReader version: head
  • Device:

Issue

[ 37%] Performing patch step for 'lua-Spore'
[ 50%] No update step for 'lua-Spore'
[ 62%] No configure step for 'lua-Spore'
[ 75%] Performing build step for 'lua-Spore'
sh: luarocks-5.1: command not found
make[4]: *** [lua-Spore-prefix/src/lua-Spore-stamp/lua-Spore-build] Error 127
make[3]: *** [CMakeFiles/lua-Spore.dir/all] Error 2
make[2]: *** [all] Error 2
make[1]: *** [build/x86_64-apple-darwin18.2.0-debug/rocks/lib/luarocks/rocks/lua-spore/0.3.1-1/lua-spore-0.3.1-1.rockspec] Error 2
make: *** [base/build/x86_64-apple-darwin18.2.0-debug/luajit] Error 2

Steps to reproduce

git clone https://github.com/koreader/koreader.git
cd koreader && ./kodev fetch-thirdparty
./kodev build

@Frenzie
Copy link
Member

Frenzie commented Dec 3, 2018

Did you brew install?

@pazos
Copy link
Member

pazos commented Dec 3, 2018

Package name changed from lua51 to lua@5.1. We need to update the README.

I'm going to try to build on my macbook later this night to see if it builds.

@Dollyn
Copy link
Author

Dollyn commented Dec 4, 2018

Did you brew install?

yes, i have brew installed

@pazos
Copy link
Member

pazos commented Dec 4, 2018

@Dollyn: You need to install luarocks too. But if you install it from brew you'll find that is configured to work w/ lua 5.3 instead of 5.1. You can check that with luarocks --help.

You can change that with luarocks --lua-dir=/usr/local/opt/lua@5.1.and make an alias called luarocks-5.1, The build should work.

Sadly I'm unable to reproduce since my brew tools (and my osx installation) weren't upgraded in a while. I will check that after a clean install of mojave.

@Frenzie
Copy link
Member

Frenzie commented Dec 4, 2018

yes, i have brew installed

I mean did you run the relevant brew install command or commands. If the package name changed I imagine it would give you an error message.

@Dollyn
Copy link
Author

Dollyn commented Dec 5, 2018

yes, i have brew installed

I mean did you run the relevant brew install command or commands. If the package name changed I imagine it would give you an error message.

I did not do that directly. I only use the comand ./kodev build

@Dollyn
Copy link
Author

Dollyn commented Dec 5, 2018

@pazos
I installed luarocks using brew install, and yes it is configured to lua5.3. Then I added alias by adding the following line to .bash_profile:

alias luarocks-5.1='luarocks --lua-dir=/usr/local/opt/lua@5.1'

I can use luarocs-5.1 from my terminal. But ./kodev build still can not find command 'luarocs-5.1'.
Then I find out that the build is using sh instead of bash, then I add the above alias line to ~/.profile, /etc/profile, none of them works.

Then I modify base/thirdparty/lua-Spore/CMakeLists.txt, change the build command directly:

set(BUILD_CMD bash -c "luarocks --lua-dir=/usr/local/opt/lua@5.1 make --tree=${OUTPUT_DIR}/rocks ${LUA_SPORE_ROCKSPEC}")

Now a new error appears:

[ 12%] Performing build step for 'lua-Spore'
cd /Users/dollyn/workspace/koreader/koreader/base/thirdparty/lua-Spore/build/x86_64-apple-darwin18.2.0-debug/lua-Spore-prefix/src/lua-Spore && bash -c "source /etc/profile && luarocks --lua-dir=/usr/local/opt/lua@5.1 make --tree=/Users/dollyn/workspace/koreader/koreader/base/build/x86_64-apple-darwin18.2.0-debug/rocks rockspec/lua-spore-0.3.1-1.rockspec CC="gcc" CFLAGS="-Og -g -pipe -march=native -fPIC -I/Users/dollyn/workspace/koreader/koreader/base/thirdparty/luajit/build/x86_64-apple-darwin18.2.0-debug/luajit-prefix/src/luajit/src" LD="gcc" LUA_INCDIR="/Users/dollyn/workspace/koreader/koreader/base/thirdparty/luajit/build/x86_64-apple-darwin18.2.0-debug/luajit-prefix/src/luajit/src" LUA_LIBDIR="/Users/dollyn/workspace/koreader/koreader/base/build/x86_64-apple-darwin18.2.0-debug/libs/""

Error: Failed finding Lua library. You may need to configure LUA_LIBDIR.
make[4]: *** [lua-Spore-prefix/src/lua-Spore-stamp/lua-Spore-build] Error 1
make[3]: *** [CMakeFiles/lua-Spore.dir/all] Error 2
make[2]: *** [all] Error 2
make[1]: *** [build/x86_64-apple-darwin18.2.0-debug/rocks/lib/luarocks/rocks/lua-spore/0.3.1-1/lua-spore-0.3.1-1.rockspec] Error 2
make: *** [all] Error 2
Failed to build emulator! Try run with -v for more information.

failed find lua lib, but it did't tell which lib it can't find

@Frenzie
Copy link
Member

Frenzie commented Dec 5, 2018

You didn't say if you installed all the requirements as per the readme or just Lua?

@Dollyn
Copy link
Author

Dollyn commented Dec 5, 2018

Yes, I did execute the commands in the readme:

brew install nasm binutils libtool autoconf automake cmake makedepend sdl2 lua51 gettext pkg-config wget md5sha1sum
echo 'export PATH="/usr/local/opt/gettext/bin:$PATH"' >> "$HOME"/.bash_profile

@NiLuJe
Copy link
Member

NiLuJe commented Dec 5, 2018

I don't have my laptop near me right now, which version of luarocks is brew installing?

Luarocks 3 introduced many changes to help handle // lua installations, but we're probably not handling it correctly (or at all on macOS), as the bots are still using luarocks 2

@pazos
Copy link
Member

pazos commented Dec 5, 2018

@NiLuJe: 3.0.1 as https://formulae.brew.sh/formula/luarocks.

I managed to build luarocks 3.0.4 from source and link against lua@5.1 and it works as long as I modify the base repo:

diff --git a/Makefile.third b/Makefile.third
index 3516aa9..407b2c9 100644
--- a/Makefile.third
+++ b/Makefile.third
@@ -593,8 +593,8 @@ $(LUA_SPORE_ROCK): $(THIRDPARTY_DIR)/lua-Spore/CMakeLists.txt
                $(CMAKE) -DOUTPUT_DIR="$(CURDIR)/$(OUTPUT_DIR)" \
                -DLUA_SPORE_VER=$(LUA_SPORE_VER) -DLD="$(CC)" \
                -DCC="$(CC)" -DCFLAGS="$(CFLAGS) -I$(LUAJIT_DIR)/src" \
-               -DLUAROCKS=$(if $(DARWINHOST),luarocks-5.1,luarocks) \
-               -DLUA_INCDIR="$(LUAJIT_DIR)/src" -DLUA_LIBDIR="$(CURDIR)/$(dir $(LUAJIT_LIB))" \
+               -DLUAROCKS=luarocks \
+               -DLUA_INCDIR=/usr/local/opt/lua@5.1/include/lua5.1 -DLUA_LIBDIR=/usr/local/opt/lua@5.1/lib \
                $(if $(ANDROID),-DLIBFLAG="$(LDFLAGS) $(CURDIR)/$(LUAJIT_LIB) -nostartfiles",) \
                $(CURDIR)/$(THIRDPARTY_DIR)/lua-Spore && \
                $(MAKE)

The rest of the build script worked but, sadly, I needed to use install_name_tool to modify the path of libjpeg in mupdf dylib because otool -L was showing incorrect information.

BTW, movaje breaks compatibility with 10.4, so some packages (like luajit) require a newer deployment target (I'm building with export MACOSX_DEPLOYMENT_TARGET=10.10 but any target from 10.06 should work)

@pazos
Copy link
Member

pazos commented Dec 5, 2018

After building I noticed that running ./kodev run will perform the build step for openssl again and again.

If I run ./reader.lua directly then fails with a problem loading shared libraries

---------------------------------------------
                launching...
  _  _____  ____                _
 | |/ / _ \|  _ \ ___  __ _  __| | ___ _ __
 | ' / | | | |_) / _ \/ _` |/ _` |/ _ \ '__|
 | . \ |_| |  _ <  __/ (_| | (_| |  __/ |
 |_|\_\___/|_| \_\___|\__,_|\__,_|\___|_|

 It's a scroll... It's a codex... It's KOReader!

 [*] Current time: 12/06/18-00:04:31
 [*] Version: v2018.12

12/06/18-00:04:31 WARN  cannot open translation file: l10n/es_ES/koreader.po
ffi.load: blitbuffer
ffi.load (assisted searchpath): ./libs/libblitbuffer.so
ffi.load: SDL2
ffi.load: SDL2
2018-12-06 00:04:31.546 luajit[97312:517702] isPrefsCreateCacheFromEnabledAndDefaultInputSources - can't find anything from GetInputSourceEnabledPrefs, use defaultASCIIKeyLayoutDict = <CFBasicHash 0x7fefe7c3acd0 [0x7fff85bf38f0]>{type = mutable dict, count = 3,
entries =>
        0 : <CFString 0x7fff85c61eb8 [0x7fff85bf38f0]>{contents = "InputSourceKind"} = <CFString 0x7fff85ca73f8 [0x7fff85bf38f0]>{contents = "Keyboard Layout"}
        1 : <CFString 0x7fff85c93538 [0x7fff85bf38f0]>{contents = "KeyboardLayout ID"} = <CFNumber 0x32de2e24e3ad56ab [0x7fff85bf38f0]>{value = +87, type = kCFNumberSInt64Type}
        9 : <CFString 0x7fff85c5cbb8 [0x7fff85bf38f0]>{contents = "KeyboardLayout Name"} = <CFString 0x7fff85ce5418 [0x7fff85bf38f0]>{contents = "Spanish - ISO"}
}
SDL: no gamecontrollers connected
ffi.load: libs/liblodepng.dylib
ffi.load: gtk-3
ffi.load (warning): dlopen(libgtk-3.dylib, 5): image not found
ffi.load: gtk-3.so.0
ffi.load (warning): dlopen(libgtk-3.so.0, 5): image not found
06/12/2018-00:04:31 INFO  initializing for device SDL
06/12/2018-00:04:31 INFO  framebuffer resolution: {
    ["h"] = 800,
    ["w"] = 600
}
ffi.load: libs/libfreetype.6.dylib
ffi.load: libs/libmupdf.dylib
ffi.load (warning): dlopen(libs/libmupdf.dylib, 5): Library not loaded: @rpath/libjpeg.8.dylib
  Referenced from: /Users/pazos/Desktop/koreader/base/build/x86_64-apple-darwin18.2.0-debug/libs/libmupdf.dylib
  Reason: image not found
./luajit: ./setupkoenv.lua:30: Not able to load dynamic library: libs/libmupdf.dylib
stack traceback:
        [C]: in function 'error'
        ./setupkoenv.lua:30: in function 'load'
        ./ffi/mupdf.lua:23: in main chunk
        [C]: in function 'require'
        frontend/ui/widget/htmlboxwidget.lua:10: in main chunk
        [C]: in function 'require'
        frontend/ui/widget/scrollhtmlwidget.lua:6: in main chunk
        [C]: in function 'require'
        frontend/ui/widget/dictquicklookup.lua:18: in main chunk
        [C]: in function 'require'
        frontend/apps/reader/modules/readerdictionary.lua:4: in main chunk
        [C]: in function 'require'
        frontend/apps/reader/readerui.lua:26: in main chunk
        [C]: in function 'require'
        ./reader.lua:212: in main chunk
        [C]: at 0x010bf8a4c0

otool output for libjpeg:

macbook:koreader pazos$ otool -L libs/libjpeg.8.dylib 
libs/libjpeg.8.dylib:
        @rpath/libjpeg.8.dylib (compatibility version 8.0.0, current version 8.2.2)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.200.5)

otool output for libmupdf

macbook:koreader pazos$ otool -L libs/libmupdf.dylib 
libs/libmupdf.dylib:
        /Users/pazos/Desktop/koreader/base/build/x86_64-apple-darwin18.2.0-debug/libs/libmupdf.dylib (compatibility version 0.0.0, current version 0.0.0)
        /Users/pazos/Desktop/koreader/base/thirdparty/zlib/build/x86_64-apple-darwin18.2.0-debug/zlib-prefix/src/zlib/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
        @rpath/libjpeg.8.dylib (compatibility version 8.0.0, current version 8.2.2)
        /Users/pazos/Desktop/koreader/base/thirdparty/freetype2/build/x86_64-apple-darwin18.2.0-debug/freetype2-prefix/src/freetype2-build/lib/libfreetype.6.dylib (compatibility version 23.0.0, current version 23.1.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.200.5)

@pazos
Copy link
Member

pazos commented Dec 5, 2018

To make the program work we need to type:

cd libs
install_name_tool -change @rpath/libjpeg.8.dylib /Users/pazos/Desktop/koreader/base/thirdparty/libjpeg-turbo/build/x86_64-apple-darwin18.2.0-debug/libjpeg-turbo-prefix/src/libjpeg-turbo-build/lib/libjpeg.8.dylib libmupdf.dylib

This will change otool output to:

macbook:libs pazos$ otool -L libmupdf.dylib 
libmupdf.dylib:
        /Users/pazos/Desktop/koreader/base/build/x86_64-apple-darwin18.2.0-debug/libs/libmupdf.dylib (compatibility version 0.0.0, current version 0.0.0)
        /Users/pazos/Desktop/koreader/base/thirdparty/zlib/build/x86_64-apple-darwin18.2.0-debug/zlib-prefix/src/zlib/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
        /Users/pazos/Desktop/koreader/base/thirdparty/libjpeg-turbo/build/x86_64-apple-darwin18.2.0-debug/libjpeg-turbo-prefix/src/libjpeg-turbo-build/lib/libjpeg.8.dylib (compatibility version 8.0.0, current version 8.2.2)
        /Users/pazos/Desktop/koreader/base/thirdparty/freetype2/build/x86_64-apple-darwin18.2.0-debug/freetype2-prefix/src/freetype2-build/lib/libfreetype.6.dylib (compatibility version 23.0.0, current version 23.1.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.200.5)

The program starts fine and logs:

macbook:koreader pazos$ ./reader.lua 
---------------------------------------------
                launching...
  _  _____  ____                _
 | |/ / _ \|  _ \ ___  __ _  __| | ___ _ __
 | ' / | | | |_) / _ \/ _` |/ _` |/ _ \ '__|
 | . \ |_| |  _ <  __/ (_| | (_| |  __/ |
 |_|\_\___/|_| \_\___|\__,_|\__,_|\___|_|

 It's a scroll... It's a codex... It's KOReader!

 [*] Current time: 12/06/18-00:14:35
 [*] Version: v2018.12

12/06/18-00:14:35 WARN  cannot open translation file: l10n/es_ES/koreader.po
ffi.load: blitbuffer
ffi.load (assisted searchpath): ./libs/libblitbuffer.so
ffi.load: SDL2
ffi.load: SDL2
2018-12-06 00:14:35.857 luajit[97336:523296] isPrefsCreateCacheFromEnabledAndDefaultInputSources - can't find anything from GetInputSourceEnabledPrefs, use defaultASCIIKeyLayoutDict = <CFBasicHash 0x7fe88f429560 [0x7fff85bf38f0]>{type = mutable dict, count = 3,
entries =>
        0 : <CFString 0x7fff85c61eb8 [0x7fff85bf38f0]>{contents = "InputSourceKind"} = <CFString 0x7fff85ca73f8 [0x7fff85bf38f0]>{contents = "Keyboard Layout"}
        1 : <CFString 0x7fff85c93538 [0x7fff85bf38f0]>{contents = "KeyboardLayout ID"} = <CFNumber 0xaeb61d9c5d907d37 [0x7fff85bf38f0]>{value = +87, type = kCFNumberSInt64Type}
        9 : <CFString 0x7fff85c5cbb8 [0x7fff85bf38f0]>{contents = "KeyboardLayout Name"} = <CFString 0x7fff85ce5418 [0x7fff85bf38f0]>{contents = "Spanish - ISO"}
}
SDL: no gamecontrollers connected
ffi.load: libs/liblodepng.dylib
ffi.load: gtk-3
ffi.load (warning): dlopen(libgtk-3.dylib, 5): image not found
ffi.load: gtk-3.so.0
ffi.load (warning): dlopen(libgtk-3.so.0, 5): image not found
06/12/2018-00:14:35 INFO  initializing for device SDL
06/12/2018-00:14:35 INFO  framebuffer resolution: {
    ["h"] = 800,
    ["w"] = 600
}
ffi.load: libs/libfreetype.6.dylib
ffi.load: libs/libmupdf.dylib
ffi.load: libs/libwrap-mupdf.so
06/12/2018-00:14:36 INFO  opening file ./help/quickstart-en-v2018.12.html
06/12/2018-00:14:36 INFO  Loading plugins from directory: plugins
06/12/2018-00:14:36 INFO  Plugin  plugins/SSH.koplugin/main.lua  has been disabled.
06/12/2018-00:14:36 INFO  Plugin  plugins/hello.koplugin/main.lua  has been disabled.
ffi.load: sqlite3
ffi.load: z
ffi.load: sqlite3
06/12/2018-00:14:36 INFO  Plugin  plugins/timesync.koplugin/main.lua  has been disabled.
06/12/2018-00:14:36 INFO  Plugin  plugins/autofrontlight.koplugin/main.lua  has been disabled.
06/12/2018-00:14:36 INFO  RD loaded plugin autosuspend at plugins/autosuspend.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin backgroundrunner at plugins/backgroundrunner.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin batterystat at plugins/batterystat.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin calibrecompanion at plugins/calibrecompanion.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin coverbrowser at plugins/coverbrowser.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin evernote at plugins/evernote.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin goodreads at plugins/goodreads.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin keepalive at plugins/keepalive.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin kobolight at plugins/kobolight.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin kosync at plugins/kosync.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin newsdownloader at plugins/newsdownloader.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin perceptionexpander at plugins/perceptionexpander.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin readtimer at plugins/readtimer.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin send2ebook at plugins/send2ebook.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin statistics at plugins/statistics.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin systemstat at plugins/systemstat.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin terminal at plugins/terminal.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin texteditor at plugins/texteditor.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin wallabag at plugins/wallabag.koplugin
06/12/2018-00:14:36 INFO  RD loaded plugin zsync at plugins/zsync.koplugin

@pazos
Copy link
Member

pazos commented Dec 5, 2018

captura de pantalla 2018-12-06 a las 0 18 01

I opened without problems epubs, pdf and djvu from tests folder.

Pinging @chrox because he will know better how to fix the rpath stuff on mac.

@pazos
Copy link
Member

pazos commented Dec 5, 2018

@Dollyn: This issue isn't related to mojave (except the mandatory environment MACOSX_DEPLOYMENT_TARGET > 10.4), and should affect any other mac running el capitan/sierra/high sierra with recent brew tools (and with recent I mean updated in the last two years from August 2018)

@pazos
Copy link
Member

pazos commented Dec 6, 2018

A fun twist of messing with otool is that we can create a mac app bundle just by loading shared libraries from @executable_path. AFAICT just sdl2 is needed on top of koreader libs.

I followed the guide to build an osx app bundle - the ugly way and copied everything under the emulator (using cp -LR to avoid soft links) to koreader.app/Contents/Resources and place a shell script in koreader.app/Contents/MacOS/koreader with

#!/bin/bash
cd "${0%/*}/../Resources"
./reader.lua

And funny enough it seems to be ready to rock!
kapture 2018-12-06 at 2 35 07

tried packaging the emulator with debug info and got an app of 180MB. I guess without debugging symbols I will get about 80MB reduction. (dmg file of about 40MB)

@pazos pazos self-assigned this Dec 6, 2018
@Frenzie
Copy link
Member

Frenzie commented Dec 6, 2018

should affect any other mac running el capitan/sierra/high sierra with recent brew tools (and with recent I mean updated in the last two years)

@pazos I had no issues when I fixed the MuPDF build on Mac earlier this year and that would've been a brand new brew.

@pazos
Copy link
Member

pazos commented Dec 6, 2018

@Frenzie: You're right! It seems that transiction to luarocks3 has happened in August 2018. Homebrew/homebrew-core@799af87

@pazos
Copy link
Member

pazos commented Dec 27, 2018

Pinging @Dollyn, let us know if you find issues now. Thanks

@pazos pazos removed their assignment Dec 27, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants