Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup Travis-CI for build AppImage #10

Closed
Symbian9 opened this issue May 4, 2018 · 68 comments
Closed

Setup Travis-CI for build AppImage #10

Symbian9 opened this issue May 4, 2018 · 68 comments

Comments

@Symbian9
Copy link

@Symbian9 Symbian9 commented May 4, 2018

Cast @probonopd

@probonopd
Copy link

@probonopd probonopd commented May 4, 2018

First thing to do: Get it to build on Travis CI on Ubuntu 14.04.

Ubuntu 14.04 needs a more recent harfbuzz than is included, but more recent Ubuntu should work well.

This is worrisome, as bundling Harfbuzz inside the AppImage is known to be problematic (see probonopd/linuxdeployqt#261 (comment) and probonopd/linuxdeployqt#157 (comment)).

Options:

  • Remove the dependency on Harfbuzz
  • Lower the dependency on Harfbuzz to the one in 14.04
  • Statically link Harfbuzz
@probonopd
Copy link

@probonopd probonopd commented May 4, 2018

Tried it at https://github.com/probonopd/laidout/blob/patch-1/.travis.yml. As was to be expected, running into

captioninterface.cc:428:49: error: ‘hb_ft_face_create_referenced’ was not declared in this scope
   hb_face = hb_ft_face_create_referenced(ft_face); 

https://travis-ci.org/probonopd/laidout/builds/374735928#L718

This code is too new to work on the oldest still-supported version of Ubuntu, which currently is 14.04. The source code needs to be changed so that it can build on 14.04. Otherwise the AppImage will not run on all still-supported versions of the major distributions (which one usually wants). With other libraries, the solution would be to just bundle the library in question, but this does not work properly with libharfbuzz for some reason.

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented May 4, 2018

Laidout 0.096 doesn't really need very current harfbuzz stuff, I don't recall this being insurmountable a couple years ago, meaning it shouldn't be all that difficult to make it work on default 14.04. I'm travelling now, but I'll look into it when I get home in a few days, unless you can patch it! However, I no longer have a 14.04 to test.

FYI I hope to have a 0.097 release within a month or so, and I'll be testing on Ubuntu 16.04, but it will likely be very difficult to make it work with default 14.04 harfbuzz.

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented May 18, 2018

Been low on free time, I would expect an AppImage that works on 14.04 to be successful with the "stable" Laidout 0.096 version as that does not depend on harfbuzz at all. Immediately after 0.096, I started using harfbuzz, and newer harfbuzz provides too many improvements to justify supporting 14.04 at this point. But I'd be happy to set up a separate branch if someone has a patch!

@Symbian9
Copy link
Author

@Symbian9 Symbian9 commented May 18, 2018

I started using harfbuzz, and newer harfbuzz provides too many improvements

Which version of harfbuzz you mean?

But I'd be happy to set up a separate branch if someone has a patch!

Need ask Trusty-backports team

There are many other guys that provide backports for Trusty on Launchpad, so will try cast some of them

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented May 18, 2018

For the upcoming Laidout 0.097, I'll make it work on default Ubuntu 16.04. My text tools are very much works in progress, and my goal is to support as many writing systems as I can. I'm certainly no expert, but it seems like the most current harfbuzz as possible is important for that. Balancing new development against old system compatibility is not easy.

For backporting, I would think the first step is getting any version of Laidout in a Debian or Ubuntu repository at all, currently it's not, as far as I know. I would very much welcome that, but I don't personally have enough free time to take that on.

@probonopd
Copy link

@probonopd probonopd commented Jun 2, 2019

Tried building Laidout, but running into issues:

./configure --laxkit=`pwd`/laxkit/lax --prefix=/usr --nogl --disable-sqlite --force
Checking for x11......ok
Checking for xext......ok
Checking for freetype2......ok
Checking for libssl......ok
Checking for imlib2......ok
Checking for cairo......ok
Checking for harfbuzz......ok
Checking for gegl-0.4......You may need to install the development package of gegl-0.4.
Checking for sqlite3......ok
Checking for readline......ok
Checking for cups......ok

Too many problems to proceed!
Use --force to override. You may have to adjust CPPFLAGS and LDFLAGS in Makefiles,
or pass in includes with --extra-cppflags.

On debian based systems, you might try installing all dependencies with this:
    apt-get install g++ pkg-config libpng12-dev libreadline-dev libx11-dev libxext-dev libxi-dev libxft-dev libcups2-dev libimlib2-dev libfontconfig-dev libfreetype6-dev libssl-dev xutils-dev libgraphicsmagick++1-dev mesa-common-dev libglu1-mesa-dev libftgl-dev libcairo2-dev libharfbuzz-dev

Here is what I am doing:
https://github.com/probonopd/laidout/blob/dcf0dd519a43f95dc15f077b722de859aff24f87/.travis.yml

What am I doing wrong?

Full build log:
https://api.travis-ci.org/v3/job/540384607/log.txt

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 2, 2019

Part of it is It's trying to use Gegl 0.4 but not finding it. On some recent ubuntu, for instance, there's only gegl 0.3, if that's the case you can pass in ./configure --gegl-version=gegl-0.3. If you are sure you have gegl 0.4, then you can specifiy where the headers and libs are with --extra-cppflags="-I/path/to/headers" and --extra-ldflags="/path/to/libs".

I'm surprised the --force option isn't working, it should ignore check results if it is given.

Also, I see you are passing --disable-sqlite to laidout's configure.. if that's true, you should also pass in the same thing to Laxkit's configure.

When I get done traveling perhaps I can work on the build scripts some more. I had been trying to keep the compile system very simple with plain Makefiles, but I keep adding new dependencies, and my configure script gets a bit cumbersome!

probonopd added a commit to probonopd/laidout that referenced this issue Jun 2, 2019
@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 9, 2019

Oops, I mixed up bash and Makefile syntax, had written $(LAXDIR) when it should have just been $LAXDIR.. Fresh pull should fix.

@probonopd
Copy link

@probonopd probonopd commented Jun 9, 2019

Can you support DESTDIR?

./configure --prefix=/usr (...)
sudo make install DESTDIR=$(readlink -f appdir) ; find appdir/
(...)
install -D -m711 laxinput/laxinput /usr/local/bin/laxinput && \
	echo '/usr/local/bin/laxinput' >> install.log
find: ‘appdir/’: No such file or directory

I would have expected

appdir/usr/bin/laidout
appdir/usr/share/applications/laidout.desktop
...
@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 9, 2019

Hmm.. try with ./configure --finalprefix=$(readlink -f appdir)
finalprefix is left over from dealing with a similar (i think) issue building .debs.

probonopd added a commit to probonopd/laidout that referenced this issue Jun 9, 2019
@probonopd
Copy link

@probonopd probonopd commented Jun 9, 2019

Unrecognized option --finalprefix /home/travis/build/probonopd/laidout/laxkit/appdir

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 10, 2019

Try with the '=' and no spaces:
--finalprefix=/home/...

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 10, 2019

I can't seem to get it to fail with --finalprefix with or without the '='.. Do you have the full output?

@probonopd
Copy link

@probonopd probonopd commented Jun 10, 2019

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 10, 2019

Ok, that's just weird.
finalprefix is parsed no differently than prefix or anything else. The only difference here is using $(readlink -f appdir). Even if it's an unrecognized option, it shouldn't exit 1. If finalprefix is always known, as a last resort, you can hardcode it in configure, line 12. Does it also fail if you use backticks instead of $(...)?

probonopd added a commit to probonopd/laidout that referenced this issue Jun 10, 2019
@probonopd
Copy link

@probonopd probonopd commented Jun 10, 2019

$ ./configure --finalprefix=`pwd`/appdir --laxkit=`pwd`/laxkit/lax --nogl --disable-sqlite --force --gegl-version=gegl-0.3
Unrecognized option --finalprefix /home/travis/build/probonopd/laidout/laxkit/appdir
probonopd added a commit to probonopd/laidout that referenced this issue Jun 10, 2019
@probonopd
Copy link

@probonopd probonopd commented Jun 10, 2019

$ ./configure --prefix=`pwd`/appdir --laxkit=`pwd`/laxkit/lax --nogl --disable-sqlite --force --gegl-version=gegl-0.3
Unrecognized option --laxkit /home/travis/build/probonopd/laidout/laxkit/laxkit/lax

Help!

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 11, 2019

Hmm... is that configure being called from laidout/laxkit, or from laidout/ ? I see a cd to laxkit, but maybe not a cd back out? As it happens, the laxkit/configure aborts on unrecognized options, but Laidout's does not.

@probonopd
Copy link

@probonopd probonopd commented Jun 11, 2019

Heureka! That was it. Sorry for not having noticed earlier.

Now getting

g++ -std=c++11  -Wall -g -gdwarf-2 -I/home/travis/build/probonopd/laidout/laxkit/lax/.. `pkg-config --cflags freetype2` -I/home/travis/build/probonopd/laidout/src -I/usr/include/GraphicsMagick/  -c -o singleseditor.o singleseditor.cc
polyptychwindow.cc:254:1: error: expected declaration before ‘}’ token
 } // namespace Laidout
 ^
<builtin>: recipe for target 'polyptychwindow.o' failed
make[2]: *** [polyptychwindow.o] Error 1
make[2]: Leaving directory '/home/travis/build/probonopd/laidout/src/impositions'
Makefile:180: recipe for target 'laidout' failed
make[1]: *** [laidout] Error 2
make[1]: Leaving directory '/home/travis/build/probonopd/laidout/src'
Makefile:29: recipe for target 'laidout' failed
make: *** [laidout] Error 2
The command "make -j$(nproc)" exited with 2.

https://api.travis-ci.org/v3/job/544056189/log.txt

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 11, 2019

That one's easy (i think). Fixed in master.
Resulting from the --nogl option and a badly nested #ifndef. It's possible there's other minor snags from not building without the opengl parts. Is there a particular reason you are using --nogl?

@probonopd
Copy link

@probonopd probonopd commented Jun 12, 2019

Is there a particular reason you are using --nogl?

I generally try to build things with the minimal amount of dependencies and then go from there. What for would an imposing software need OpenGL?

@probonopd
Copy link

@probonopd probonopd commented Jun 12, 2019

Getting there, but running into a compile error:

polyptychwindow.cc:99:1: error: ‘PolyptychWindow’ does not name a type
 PolyptychWindow::PolyptychWindow(NetImposition *imp, anXWindow *parnt,unsigned long owner,const char *sendmes)
 ^

Full log:
https://api.travis-ci.org/v3/job/544573113/log.txt

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 12, 2019

Fixed in master.
About gl not useful in layout software, what are you talking about!? Without gl, you can't do this: https://www.flickr.com/photos/tomlechner/3838153212/in/album-72157604986347329/
Though I confess that unwrapper as it exists in Laidout currently is a bit unsatisfactory and buggy. I do need to spend more time on the 3-d stuff.

probonopd added a commit to probonopd/laidout that referenced this issue Jun 16, 2019
@probonopd
Copy link

@probonopd probonopd commented Jun 16, 2019

Is the geglnodes.so there, but not loading, or not there at all?

It is there but its dependencies were not deployed yet, working on it.

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 16, 2019

So I made this repository for prebuilt icons: https://github.com/Laidout/laidout-icons
Before laidout make install, clone the repo and: cp laidout-icons/target-24/* src/icons

probonopd added a commit to probonopd/laidout that referenced this issue Jun 16, 2019
@probonopd
Copy link

@probonopd probonopd commented Jun 16, 2019

Now running into

GeglNodesPlugin initializing...
babl-extension.c:204 babl_extension_load()
	dlopen() failed:
	/usr/lib/x86_64-linux-gnu/babl-0.1/CIE.so: undefined symbol: babl_space_from_xyz
When loading /usr/lib/x86_64-linux-gnu/babl-0.1/HCY.so:
	babl-model.c:175 babl_model_new()
	unhandled argument 'alpha' for babl_model 'HCYA'
Could not attach to process.  If your uid matches the uid of the target
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
again as the root user.  For more details, see /etc/sysctl.d/10-ptrace.conf
ptrace: Operation not permitted.
/tmp/babl.gdb:1: Error in sourced command file:
No stack.

Apparently the plugin gets loaded, but then it seems to attempt dlopen() stuff from /usr which is not a good idea at all. Any way to prevent that from happening and only dlopen() stuff from locations relative to the main executable?

The GIMP AppImage seems to use a custom AppRun launcher script that sets

#!/bin/bash
DIR="`dirname \"$0\"`" 
DIR="`( cd \"$DIR\" && readlink -f $(pwd) )`"
export BABL_PATH=$(readlink -f "$DIR/usr/lib/babl-0.1")
export GEGL_PATH=$(readlink -f "$DIR/usr/lib/gegl-0.4")

Maybe we need to do that too.

probonopd added a commit to probonopd/laidout that referenced this issue Jun 16, 2019
@probonopd
Copy link

@probonopd probonopd commented Jun 16, 2019

Is any of the usr/lib/x86_64-linux-gnu/babl-0.1/ actually needed? Then we need to bundle those and their dependencies as well.

@probonopd
Copy link

@probonopd probonopd commented Jun 16, 2019

Please try now @tomlechner

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 16, 2019

Besides the initial geglnodes.so, I'm not doing any dlopen, I presume any other library loading is internal to gegl, which absolutely depends on babl. Which is what gimps export BABL_Path.. is supposed to account for, I guess?

When I load on my system, my geglnodes.so seems to load, but somehow it doesn't seem to pull in anything that's from the actual gegl library, as if gegl isn't being initialized or something, and subsequent library calls just return nothing. I might need to put in more error checking for this kind of thing.

Otherwise, looks like icons work now!

@probonopd
Copy link

@probonopd probonopd commented Jun 16, 2019

When I load on my system, my geglnodes.so seems to load, but somehow it doesn't seem to pull in anything that's from the actual gegl library, as if gegl isn't being initialized or something, and subsequent library calls just return nothing. I might need to put in more error checking for this kind of thing.

Maybe because the stuff from usr/lib/x86_64-linux-gnu/gegl-0.4/ and its dependencies would need to be bundled? Which of those are needed? All?

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 16, 2019

Potentially all, yes. I imagine they could all be accessible in Laidout's node tool.

@probonopd
Copy link

@probonopd probonopd commented Jun 17, 2019

Is it better now? How do I test?

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 17, 2019

Still missing the gegl nodes. To test, start a new document, shortcut with laidout -n letter, select Node Tool from tool menu lower left, right cilck -> Add Node, under Gegl, there should be thirty or so nodes, not just GeglBounds, GeglToImage, ImageToGegl.

In the debug output to the console, there's a line that's "gegl operations:", which should be followed by numerous lines showing each gegl operation. Since there's not, the gegl library is not returning any known operations. Not quite sure what to do about that. I'm not seeing any bad linking messages here.

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 20, 2019

In the travis.yml, Laidout is being configured and built against gegl-0.3, but there's a bunch of lines referring to gegl-0.4? -executable=appdir/usr/lib/x86_64-linux-gnu/gegl-0.4/

How much of Gimp's apprun is appimage magic, and how much is gimp magic? Laidout should need no particular setup, apparently other than helping gegl and babl find their .so files.

probonopd added a commit to probonopd/laidout that referenced this issue Jun 20, 2019
gegl-0.3/
@probonopd
Copy link

@probonopd probonopd commented Jun 20, 2019

Spot on, that was my mistake.

@probonopd
Copy link

@probonopd probonopd commented Jun 20, 2019

So I guess this time it's looking better?

By the way, there seems to be so much more in Laidout that meets the eye initially, I wish there was a good video to learn it all.

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 20, 2019

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 22, 2019

I'm not seeing all the gegl nodes in Laidout-57cb006-x86_64.AppImage, only the 3 gegl glue nodes. So a custom apprun would just be a script that sets environment variables for babl and gegl then calls the actual binary?

For reference, when fully successful, the gegl menu should look like this:
geglnodes

@probonopd
Copy link

@probonopd probonopd commented Jun 23, 2019

I have no clue why it is not loading these. Are there additional environment variables missing? You can run --appimage-extract to inspect/change its contents, including AppRun, if you'd like to experiment.

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 23, 2019

I see in the AppRun, the GEGL_PATH is still set to 0.4, not 0.3.

@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 23, 2019

If I make changes to the extracted stuff to experiment, what's the reverse of --appimage-extract?

probonopd added a commit to probonopd/laidout that referenced this issue Jun 23, 2019
@probonopd
Copy link

@probonopd probonopd commented Jun 23, 2019

If I make changes to the extracted stuff to experiment, what's the reverse of --appimage-extract?

wget -c "https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-x86_64.AppImage"
chmod +x ./appimagetool-x86_64.AppImage
./appimagetool-x86_64.AppImage ./squashfs-root/

But just executing AppRun in the extracted directory should be sufficient to see if it works.

probonopd added a commit to probonopd/laidout that referenced this issue Jun 23, 2019
@tomlechner
Copy link
Contributor

@tomlechner tomlechner commented Jun 23, 2019

Woohoo, success!!

@probonopd
Copy link

@probonopd probonopd commented Jul 5, 2019

I tried to send a PR but it shows too many changes. I am a bit puzzled.

@tomlechner tomlechner closed this Jul 12, 2019
@probonopd
Copy link

@probonopd probonopd commented Jul 12, 2019

Thank you very much @tomlechner . Well done! 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants