Distribution: Raw AppImage builder using Docker #912

henry701 · 2022-01-30T03:50:46Z

AppImage builder from scratch using Docker.

Closes #804.

PR here just for tracking, so far. It's still not working because Asbrú will need some code changes in order to invoke external executables without the environment variable mangling (that was done in order to run the AppImage with the correct relative paths of the embedded libs). More about the progress in the comments below.

To build the AppImage, you will need Docker installed and accessible from your POSIX shell. Then, just cd to the root directory of the project, and run: ./dist/appimage-raw/make_appimage.sh. It will be available on ./dist/appimage-raw/build.

TODO: Fix Perl include paths

…some linking issues left to iron out

gfrenoy · 2022-02-07T20:38:11Z

Thanks a lot Henrique !

I gave it a try and was able to build the AppImage but when executing it, I have the following message:

execv error: Permission denied

Is this what you mean with "invoke external executables without the environment variable mangling" or did I do something wrong ?

henry701 · 2022-02-07T21:36:29Z

Thanks a lot Henrique !

I gave it a try and was able to build the AppImage but when executing it, I have the following message:

execv error: Permission denied

Is this what you mean with "invoke external executables without the environment variable mangling" or did I do something wrong ?

I'm usually having no such file or directory instead of permission denied, strange...

Anyways, this PR will take some more time to be ready, because of a lot of complexity:

Asbru uses VTE to invoke itself and there's some issues arising with that due to the customized env vars.
Every time we invoke a command which executable lies outside of the AppImage the original env vars have to be set.
[I'm currently working on this] Local Shell is not working due to the X11 Perl module for some reason not detecting that it's running inside X11, in asbru_conn's window resizing code.
There's some places (local shell directory configuration, so far) where we change the current working directory, but this will have to be changed for some other alternate method because changing cwd breaks AppImages
- Changing CWD breaks AppImages because the ELF header of all executables are patched to use relative directories so that they invoke the embedded Shared Objects, not the system ones

So there's a LOT of stuff to iron out and make flexible, and there's a lot of code that's going to have to be changed to match.

Then after I'm done there will be a lot of testing both inside and outside AppImage to ensure the changed pieces are working fine.

About the permission denied problem, something somewhere is probably not marked as executable, I'm using WSL so maybe that has something to do with it, probably Windows "forces" execution even without the executable flag...

…ame Perl interpreter and same AppImage env

henry701 · 2022-02-08T00:00:11Z

The X11 problem didn't occur today even though I changed nothing locally, very strange. At least now the shell finally opens :)

Just need to iron out this, current commit ignores the configuration option:

There's some places (local shell directory configuration, so far) where we change the current working directory, but this will have to be changed for some other alternate method because changing cwd breaks AppImages

Changing CWD breaks AppImages because the ELF header of all executables are patched to use relative directories so that they invoke the embedded Shared Objects, not the system ones

And then more extensive testing, of course.

Also, sorry for not even saying on the PR how to build the docker image. I will edit it with the information.

henry701 · 2022-02-08T00:21:26Z

Does the AppImage work for you after these last changes?

gfrenoy · 2022-02-08T09:22:49Z

Does the AppImage work for you after these last changes?

No joy :( Here is what I see if I run it on the same machine:

$ dist/appimage-raw/build/Asbru-CM.AppImage 
Gtk-Message: 10:15:04.368: Not loading module "atk-bridge": The functionality is provided by GTK natively. Please try to not load it.
syntax error at /tmp/.mount_Asbru-HlZ4og/opt/asbru-cm/lib/PACUtils.pm line 343, near "return"
syntax error at /tmp/.mount_Asbru-HlZ4og/opt/asbru-cm/lib/PACUtils.pm line 368, near "return"
BEGIN not safe after errors--compilation aborted at /tmp/.mount_Asbru-HlZ4og/opt/asbru-cm/lib/PACUtils.pm line 3812.
Compilation failed in require at /tmp/.mount_Asbru-HlZ4og/opt/asbru-cm/lib/PACMain.pm line 57.
BEGIN failed--compilation aborted at /tmp/.mount_Asbru-HlZ4og/opt/asbru-cm/lib/PACMain.pm line 57.
Compilation failed in require at /tmp/.mount_Asbru-HlZ4og/opt/asbru-cm/asbru-cm line 96.
BEGIN failed--compilation aborted at /tmp/.mount_Asbru-HlZ4og/opt/asbru-cm/asbru-cm line 96.

It looks like we are not using the right version of perl or using a "strict" mode or something ; I'm lost at this stage...

henry701 · 2022-02-08T15:31:41Z

I actually forgot a ; after logging some extra things and didn't run Asbrú before committing 🤦‍♂️ will fix this later

gfrenoy · 2022-02-08T15:51:34Z

I actually forgot a ; after logging some extra things and didn't run Asbrú before committing man_facepalming will fix this later

Oh yes, of course ... I see ; did not realize you made that many changes and that it was actually a "real" error ;)

So, I did fix it locally, tried again and ... it works ! That looks like a great news :) I'll play around and we'll also need to see how we can automate this build in our Travis script.

Thanks a lot.

henry701 · 2022-02-08T15:59:20Z

Good, nice to hear that it works there as well! I'll have the other TODOs patched up sometime this week probably. And documentation :x

…l commands as well

henry701 · 2022-02-13T05:10:44Z

Both "shell directory" and "use login shell" settings work properly now. Still need to test more and check for strings used on anything that spawns a subprocess so that every one of those is checked for env variable prefixing, depending on what it does and what it invokes.

… improve restart on config import

henry701 · 2022-03-25T16:10:25Z

Seems stable and complete now. Missing docs for now

henry701 · 2022-04-10T18:58:31Z

Docs done 👀

gfrenoy · 2022-04-17T15:52:20Z

Hi Henrique,

Thanks again for the terrific work to get this working ! I made a test with the latest loki release and was able to create an AppImage with the latest loki code.

It's working fine for me. The only little glitch I see is the following message for every terminal you want to start:

Gtk-Message: 17:35:50.555: Not loading module "atk-bridge": The functionality is provided by GTK natively. Please try to not load it.

Any idea why I see this ? Can we do something about it ?

I now need to see how we can build this automatically when we start our automated build.

henry701 · 2022-04-20T03:04:52Z

I couldn't reproduce that here, it seems some GTK module outside the AppImage is being picked up.

From searching around about GTK module loading I saw some documentation about GTK_IM_MODULE and GTK_IM_MODULE_FILE environment variables, I'll adjust the AppRun script to override them as well and see.

henry701 · 2022-04-20T03:29:43Z

Try now on the branch, that atk-bridge module was on the gtk-2.0 folder, so I pointed GTK_PATH to the gtk-3.0 folder instead (and that's what Ásbrú uses, so whoops). Diff: henry701@b95b324

gfrenoy · 2022-05-16T20:41:16Z

Henrique, still searching for a way to publish the AppImage from our automated build...

In the meantime, I created an entry on AppImageHub and uploaded manually:
https://www.appimagehub.com/p/1796061

henry701 · 2022-06-09T16:41:30Z

Henrique, still searching for a way to publish the AppImage from our automated build...

In the meantime, I created an entry on AppImageHub and uploaded manually: https://www.appimagehub.com/p/1796061

Is using a separate build orchestration an option, or does it need to be on the same Travis CI? I can try to integrate with Travis CI or GitHub Actions and post the resulting code, if it would help :)

Hi Henrique,

Thanks again for the terrific work to get this working ! I made a test with the latest loki release and was able to create an AppImage with the latest loki code.

It's working fine for me. The only little glitch I see is the following message for every terminal you want to start:
Gtk-Message: 17:35:50.555: Not loading module "atk-bridge": The functionality is provided by GTK natively. Please try to not load it.
Any idea why I see this ? Can we do something about it ?

I now need to see how we can build this automatically when we start our automated build.

Could you cherry-pick henry701@b95b324 and try again? It probably solves the issue you described here.

gfrenoy · 2022-06-11T18:30:39Z

Is using a separate build orchestration an option,

Yes, certainly.

I can try to integrate with Travis CI or GitHub Actions and post the resulting code, if it would help :)

Well, adding this to Travis CI is fairly trivial if I would have a way to upload the build to AppImageHub (or anything else...).
I did not really find an API or a way to automate this :(

Could you cherry-pick henry701/asbru-cm@b95b324 and try again?

Sure, will do.

Thanks again for this great feature !

gfrenoy · 2022-06-11T19:05:05Z

Could you cherry-pick henry701@b95b324 and try again?

It works like a charm :) Thanks a lot !

I uploaded "apha-2" on https://www.appimagehub.com/p/1796061

Do not use external env XDG_DATA_DIRS and LD_LIBRARY_PATH as fallback

henry701 added 3 commits January 16, 2022 21:30

feat: Create raw appimage directory using Docker

a30ce41

TODO: Fix Perl include paths

fix: [Raw AppImage] Perl now executes and loads Ásbrú, but there are …

b809b2f

…some linking issues left to iron out

fix!: Add more env vars for relocation, patch loaders.cache

79bc3f4

henry701 added 3 commits February 7, 2022 20:53

fix: Add executable flag to generated AppImage

c22738f

chore: Invoke external executables with old env, and sub-perls with s…

2e23a36

…ame Perl interpreter and same AppImage env

Merge branch 'loki' into loki-feat-raw-appimage

d2e6c6d

fix: Add execution permissions to AppRun file

6555cb8

henry701 added 5 commits February 9, 2022 01:16

fix!: Missing semicolon. Classic.

37211c9

chore: Remove extraneous verbose log

8705714

fix!: Another missing semicolon, oops

130420d

fix: More syntax fixes

45b06ab

feat!: Fix cwd changing on asbru_conn, add env var prefix for interna…

aef7165

…l commands as well

henry701 added 2 commits February 27, 2022 13:16

fix!: Use -e flag on ps for desktop detection

57f5518

fix: Add missing ASBRU_ENV_FOR_EXTERNAL env usages, remove old TODOs,…

aaed279

… improve restart on config import

henry701 marked this pull request as ready for review March 25, 2022 16:10

henry701 changed the title ~~[DRAFT] Raw AppImage builder using Docker~~ Distribution: Raw AppImage builder using Docker Mar 25, 2022

henry701 added 2 commits April 10, 2022 15:57

docs: Document necessary AppImage adjustments

9907764

chore: Add '/out' (mkdocs dir) to .gitignore

291a99f

gfrenoy merged commit 2c2bba1 into asbru-cm:loki Apr 17, 2022

henry701 mentioned this pull request Jun 9, 2022

Request for Appimage #804

Closed

gfrenoy added a commit that referenced this pull request Jun 11, 2022

🚨 Fix for "atk-bridge" message when starting AppImage (#912)

6c748bb

gfrenoy added a commit that referenced this pull request Jun 18, 2022

🚨 Fix for AppImage (#912)

d5aaecc

Do not use external env XDG_DATA_DIRS and LD_LIBRARY_PATH as fallback

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Distribution: Raw AppImage builder using Docker #912

Distribution: Raw AppImage builder using Docker #912

henry701 commented Jan 30, 2022 •

edited

Loading

gfrenoy commented Feb 7, 2022

henry701 commented Feb 7, 2022 •

edited

Loading

henry701 commented Feb 8, 2022

henry701 commented Feb 8, 2022

gfrenoy commented Feb 8, 2022

henry701 commented Feb 8, 2022

gfrenoy commented Feb 8, 2022

henry701 commented Feb 8, 2022

henry701 commented Feb 13, 2022 •

edited

Loading

henry701 commented Mar 25, 2022

henry701 commented Apr 10, 2022

gfrenoy commented Apr 17, 2022

henry701 commented Apr 20, 2022

henry701 commented Apr 20, 2022 •

edited

Loading

gfrenoy commented May 16, 2022

henry701 commented Jun 9, 2022

gfrenoy commented Jun 11, 2022

gfrenoy commented Jun 11, 2022

Distribution: Raw AppImage builder using Docker #912

Distribution: Raw AppImage builder using Docker #912

Conversation

henry701 commented Jan 30, 2022 • edited Loading

gfrenoy commented Feb 7, 2022

henry701 commented Feb 7, 2022 • edited Loading

henry701 commented Feb 8, 2022

henry701 commented Feb 8, 2022

gfrenoy commented Feb 8, 2022

henry701 commented Feb 8, 2022

gfrenoy commented Feb 8, 2022

henry701 commented Feb 8, 2022

henry701 commented Feb 13, 2022 • edited Loading

henry701 commented Mar 25, 2022

henry701 commented Apr 10, 2022

gfrenoy commented Apr 17, 2022

henry701 commented Apr 20, 2022

henry701 commented Apr 20, 2022 • edited Loading

gfrenoy commented May 16, 2022

henry701 commented Jun 9, 2022

gfrenoy commented Jun 11, 2022

gfrenoy commented Jun 11, 2022

henry701 commented Jan 30, 2022 •

edited

Loading

henry701 commented Feb 7, 2022 •

edited

Loading

henry701 commented Feb 13, 2022 •

edited

Loading

henry701 commented Apr 20, 2022 •

edited

Loading