Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate distributed Linux builds to snap #6112

Open
janisozaur opened this issue Aug 2, 2017 · 75 comments
Open

Migrate distributed Linux builds to snap #6112

janisozaur opened this issue Aug 2, 2017 · 75 comments
Assignees
Labels

Comments

@janisozaur
Copy link
Member

@janisozaur janisozaur commented Aug 2, 2017

OS: Linux

Right now the Linux builds are distributed as binary tarballs. I have investigated snaps a little and they seem to offer broader compatibility. Until recently I have feared the snap package would weigh hundreds of megabytes, but a full build of OpenRCT2 weighed only around 30MiB with symbols(!).

@janisozaur janisozaur self-assigned this Aug 2, 2017
@rwjuk

This comment has been minimized.

Copy link
Member

@rwjuk rwjuk commented Aug 2, 2017

Just to clarify, this wouldn't stop us from distributing through distro package managers (e.g. Pacman, apt, etc.) right?

@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented Aug 2, 2017

Correct. This concerns only the things we build on travis.

@rwjuk

This comment has been minimized.

Copy link
Member

@rwjuk rwjuk commented Aug 2, 2017

In that case I'm all for it.

@jansegre

This comment has been minimized.

Copy link
Contributor

@jansegre jansegre commented Aug 23, 2017

I'd suggest flatpak too. From what I understand (I may be wrong, can't find sources to back it up) it basically boils down to snap/snappy centralizing the packages on a single repository (snapcraft?) while flatpak doesn't, which mostly means a flatpak package could be attached to the github release the same way targzs are, while a snap package would have to be published to snapcraft, which is not bad per se.

Also while snappy comes by default in recent Ubuntu, other distros haven't embraced it too much (source from a couple months ago).

@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented Aug 23, 2017

Interesting. I know there are at least a few contending formats: flatpak, snappy, appimage, to some extent docker, maybe more.

I have only tried snappy so far and was quite impressed by results and the ease with which I was able to produce an openrct2.snap, at least on a Ubuntu machine. The supported distros also looks OK: https://snapcraft.io/ has logos for Arch, Debian, Gentoo, Fedora, SUSE with some installation instructions (which I haven't checked).

What do you mean by "centralizing"? In my very brief experience with snap, I noticed it downloaded a "core" image, which allows reducing amount of userspace libraries I need to install in snap itself, but other than that the snap archive looked pretty self-contained.

Are you any familiar with flatpak? Can you provide more arguments for using it? How much userspace do I need to include in resulting file, what's the size of it?

@janisozaur janisozaur mentioned this issue Aug 30, 2017
0 of 2 tasks complete
@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented Aug 31, 2017

#6232 should be taken into account when doing this.

@ykgmfq

This comment has been minimized.

Copy link

@ykgmfq ykgmfq commented Sep 7, 2017

I have managed to build a Flatpak from the latest release. There are still some issues that need to be looked at before it is functional though. The code and documentation can be found here. If someone had some ideas about how to fix the problems, that would be great!

@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented Sep 7, 2017

@ykgmfq

The dialogue for the game file selection does not work. It can be circumvented by setting game path in ~/.var/app/org.openrct2.openrct2/config/OpenRCT2/config.ini to the correct directory.

Does flatpak archive access native file system sandboxed one? What if user has kdialog or zenity installed, does it still not show up in the flatpak'd $PATH?

See ./openrct2 --help for how you can have it set game_path for you.

There seems to be a problem with resolving the data path. This leads to an immediate crash.

Does it crash or exit with error code? If former, that's a bug and please file it. I don't know flatpaks that well, but is it feasible to embed data/ inside the archive? You'd have to copy data/ from root of the repo, do make g2 and move resulting g2.dat into data/.

The build system cannont download the title sequence because it won't resolve the host name. To get around this the download has been deactivated with a cmake option.

Likewise, you can just embed title sequences in data/title/

@ykgmfq

This comment has been minimized.

Copy link

@ykgmfq ykgmfq commented Sep 7, 2017

My configuration has full host access with "--filesystem=host" for now.
It crashes with this:
ERROR[/run/build/openrct2/src/openrct2/localisation/LanguagePack.cpp:106 (FromFile)]: Unable to open /language/en-GB.txt: Unable to open '/language/en-GB.txt'

The language files are as expected in ../share/openrct2/langauges respective to the executable. I think platform_posix_sub_resolve_openrct_data_path() might not work correctly with Flatpak. Unfortunately I have not yet found out how to attach a debugger to a Flatpak process.

The download failure seems to be a Flatpak + cmake bug as it works when buidling directly on the host.

I will try your steps with modifying the package. I am aware of --openrct-data-path, but it is parsed by flatpak run itself!
$ flatpak run org.openrct2.openrct2 --openrct-datapath="/home/dmp/workspace/rctflat/files/share" Unknown option: --openrct-datapath

@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented Sep 7, 2017

You're missing a dash there. datapath vs data-path. What does --version say, flatpak or OpenRCT2 version?

If data directory is next to executable, it will be picked up automatically too.

@ykgmfq

This comment has been minimized.

Copy link

@ykgmfq ykgmfq commented Sep 7, 2017

Damn, that's embarassing. Yeah, version does show OpenRCT2. And with data-path the game actually works.

@ykgmfq

This comment has been minimized.

Copy link

@ykgmfq ykgmfq commented Sep 7, 2017

The title sequences are now included in the manifest and work as expected.

@Luraktinus

This comment has been minimized.

Copy link

@Luraktinus Luraktinus commented Feb 17, 2018

why not AppImage ?

for snap or flatpak the users have to install said tools
(snap is not centralized btw, snaps can be added to the github release page too @jansegre )

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented May 11, 2018

Building an AppImage is ridiculously easy using the binary tarball you provide.

Check out the release page of the repo I created tonight: https://github.com/TheAssassin/openrct2-appimage/

Edit: the AppImage works only on distros released after Ubuntu xenial. This is due to the version of libstdc++ the tarball's binaries are linked to.

@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented May 11, 2018

@TheAssassin that's very nice. Can you comment on snap vs flatpak vs appimage?

I have yet to test this on my Arch system, but my previous encounters with AppImage were successful, much more so than snap (which had issues obtaining GL context and required building on Ubuntu).

the AppImage works only on distros released after Ubuntu xenial. This is due to the version of libstdc++ the tarball's binaries are linked to.

Is it something dependent on the system used to build the AppImage? If I were to switch the builder to Arch, it should still work or would I need to watch out for some things? The release version still uses pre-C++-17 compiler, I think, we've updated since then and that required passing -static-libstdc++ -static-libgcc.

I still have to look into what gets provided in the embedded file system, but that looks like a viable option to me. @TheAssassin would you be interested in submitting a PR to package our develop builds into AppImages?

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented May 11, 2018

@janisozaur I am an AppImage developer (that is, I develop the apps to make AppImages), therefore I am always in favor of AppImages. They're single files which can just be copied from one system to another. They don't need any root permissions to install, they "just run" (well, in 95+% of all cases). And with tools like AppImageLauncher, they're really nice to use on the Linux desktop.

Regarding snaps: I have to disagree with @jansegre. Snaps are centralized. Always. And that's by design (but that's a story for another day). Even if you upload a built snap to GitHub, it still depends on files managed by a central entity (most likely the Canonical maintained snap store), e.g., the core image(s). They're not self contained in the way AppImages are.

I haven't tried Flatpak very much, but in contrary to AppImages, where it's a simple "make it executable" (unless you use AppImageLauncher), you need to perform quite a few more steps in order to get them up and running.

See also https://appimage.org, https://github.com/AppImage/AppImageUpdate for updating, and https://github.com/TheAssassin/AppImageLauncher for desktop integration.

Regarding the PR resp. the C++17: The general rule of thumb for AppImages is: build on the oldest OS you want to support. For most people, this is Ubuntu trusty at the moment. The reason is that we bundle libraries based on a blacklist, which is primarily used to save space in the AppImage by not bundling libraries which are available on all the distros according to our tests (e.g., libc, libstdc++, etc.) (or which are producing incompatibilities). If you need to build on a fairly recent system, you need to add some additional work, e.g., by using tools like https://github.com/darealshinji/AppImageKit-checkrt which allow you to bundle a libstdc++, and load it on demand on older systems.

We should be able to develop such an AppImage build script. I can't promise I'll get to it, but I'll put it on my TODO list. Maybe I have a few minutes during the next weekend.

@IntelOrca

This comment has been minimized.

Copy link
Contributor

@IntelOrca IntelOrca commented May 11, 2018

Other than libc, are all the other libraries such as libpng, curl, jansson etc. bundled in the app image?

@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented May 11, 2018

@IntelOrca

/tmp/.mount_OpenRCYoiIcX
├── [  16]  AppRun -> usr/bin/openrct2
├── [ 117]  openrct2.desktop
├── [ 26K]  openrct2.png
└── [   0]  usr
    ├── [   0]  bin
    │   ├── [   0]  data
    │   │   ├── [ 89K]  g2.dat
    │   │   ├── [   0]  language
    │   │   │   ├── [219K]  ar-EG.txt
    │   │   │   ├── [221K]  ca-ES.txt
    │   │   │   ├── [324K]  cs-CZ.txt
    │   │   │   ├── [214K]  de-DE.txt
    │   │   │   ├── [222K]  en-GB.txt
    │   │   │   ├── [111K]  en-US.txt
    │   │   │   ├── [210K]  es-ES.txt
    │   │   │   ├── [160K]  fi-FI.txt
    │   │   │   ├── [217K]  fr-FR.txt
    │   │   │   ├── [ 12K]  hu-HU.txt
    │   │   │   ├── [210K]  it-IT.txt
    │   │   │   ├── [266K]  ja-JP.txt
    │   │   │   ├── [363K]  ko-KR.txt
    │   │   │   ├── [196K]  nb-NO.txt
    │   │   │   ├── [284K]  nl-NL.txt
    │   │   │   ├── [293K]  pl-PL.txt
    │   │   │   ├── [240K]  pt-BR.txt
    │   │   │   ├── [319K]  ru-RU.txt
    │   │   │   ├── [204K]  sv-SE.txt
    │   │   │   ├── [208K]  zh-CN.txt
    │   │   │   └── [200K]  zh-TW.txt
    │   │   ├── [   0]  shaders
    │   │   │   ├── [ 201]  applypalette.frag
    │   │   │   ├── [ 182]  applypalette.vert
    │   │   │   ├── [ 702]  applytransparency.frag
    │   │   │   ├── [ 182]  applytransparency.vert
    │   │   │   ├── [  95]  drawline.frag
    │   │   │   ├── [ 581]  drawline.vert
    │   │   │   ├── [1.9K]  drawrect.frag
    │   │   │   └── [1.4K]  drawrect.vert
    │   │   └── [   0]  title
    │   │       ├── [4.0M]  openrct2.parkseq
    │   │       ├── [ 972]  rct1aall.parkseq
    │   │       ├── [ 798]  rct1aa.parkseq
    │   │       ├── [ 501]  rct1.parkseq
    │   │       └── [ 264]  rct2.parkseq
    │   ├── [   0]  doc
    │   │   ├── [ 38K]  changelog.txt
    │   │   ├── [6.6K]  contributors.md
    │   │   ├── [ 34K]  licence.txt
    │   │   └── [7.1K]  readme.txt
    │   ├── [   0]  include
    │   │   ├── [ 602]  discord_register.h
    │   │   └── [2.6K]  discord_rpc.h
    │   ├── [5.2M]  openrct2
    │   └── [ 24K]  openrct2-cli
    ├── [   0]  lib
    │   ├── [685K]  libasn1.so.8
    │   ├── [ 26K]  libasyncns.so.0
    │   ├── [ 83K]  libbsd.so.0
    │   ├── [2.3M]  libcrypto.so.1.0.0
    │   ├── [451K]  libcurl.so.4
    │   ├── [320K]  libdbus-1.so.3
    │   ├── [290K]  libdiscord-rpc.so
    │   ├── [ 34K]  libffi.so.6
    │   ├── [481K]  libFLAC.so.8
    │   ├── [905K]  libgcrypt.so.20
    │   ├── [524K]  libgmp.so.10
    │   ├── [1.2M]  libgnutls.so.30
    │   ├── [306K]  libgssapi_krb5.so.2
    │   ├── [272K]  libgssapi.so.3
    │   ├── [207K]  libhcrypto.so.4
    │   ├── [ 63K]  libheimbase.so.1
    │   ├── [ 39K]  libheimntlm.so.0
    │   ├── [212K]  libhogweed.so.4
    │   ├── [316K]  libhx509.so.5
    │   ├── [207K]  libidn.so.11
    │   ├── [ 54K]  libjansson.so.4
    │   ├── [ 47K]  libjson-c.so.2
    │   ├── [188K]  libk5crypto.so.3
    │   ├── [575K]  libkrb5.so.26
    │   ├── [863K]  libkrb5.so.3
    │   ├── [ 47K]  libkrb5support.so.0
    │   ├── [ 63K]  liblber-2.4.so.2
    │   ├── [332K]  libldap_r-2.4.so.2
    │   ├── [139K]  liblzma.so.5
    │   ├── [226K]  libnettle.so.6
    │   ├── [ 38K]  libogg.so.0
    │   ├── [ 17M]  libopenrct2.so
    │   ├── [450K]  libpcre.so.3
    │   ├── [153K]  libpng12.so.0
    │   ├── [518K]  libpulsecommon-8.0.so
    │   ├── [321K]  libpulse.so.0
    │   ├── [ 93K]  libroken.so.18
    │   ├── [117K]  librtmp.so.1
    │   ├── [111K]  libsasl2.so.2
    │   ├── [1.1M]  libSDL2-2.0.so.0
    │   ├── [135K]  libselinux.so.1
    │   ├── [408K]  libsndfile.so.1
    │   ├── [ 59K]  libsndio.so.6.1
    │   ├── [ 79K]  libspeexdsp.so.1
    │   ├── [858K]  libsqlite3.so.0
    │   ├── [434K]  libssl.so.1.0.0
    │   ├── [537K]  libsystemd.so.0
    │   ├── [ 78K]  libtasn1.so.6
    │   ├── [677K]  libvorbisenc.so.2
    │   ├── [176K]  libvorbis.so.0
    │   ├── [ 63K]  libwayland-client.so.0
    │   ├── [ 34K]  libwayland-cursor.so.0
    │   ├── [9.2K]  libwayland-egl.so.1
    │   ├── [166K]  libwind.so.0
    │   ├── [ 38K]  libwrap.so.0
    │   ├── [ 17K]  libXau.so.6
    │   ├── [ 43K]  libXcursor.so.1
    │   ├── [ 26K]  libXdmcp.so.6
    │   ├── [ 76K]  libXext.so.6
    │   ├── [ 26K]  libXfixes.so.3
    │   ├── [ 13K]  libXinerama.so.1
    │   ├── [ 67K]  libXi.so.6
    │   ├── [256K]  libxkbcommon.so.0
    │   ├── [ 47K]  libXrandr.so.2
    │   ├── [ 43K]  libXrender.so.1
    │   ├── [ 18K]  libXss.so.1
    │   ├── [ 26K]  libXxf86vm.so.1
    │   └── [ 79K]  libzip.so.4
    └── [   0]  share
        ├── [   0]  applications
        │   └── [  92]  openrct2.desktop
        ├── [   0]  doc
        │   ├── [   0]  libasn1-8-heimdal
        │   │   └── [8.0K]  copyright
        │   ├── [   0]  libasyncns0
        │   │   └── [1.3K]  copyright
        │   ├── [   0]  libbsd0
        │   │   └── [ 23K]  copyright
        │   ├── [   0]  libcurl3
        │   │   └── [ 11K]  copyright
        │   ├── [   0]  libdbus-1-3
        │   │   └── [ 19K]  copyright
        │   ├── [   0]  libffi6
        │   │   └── [3.4K]  copyright
        │   ├── [   0]  libflac8
        │   │   └── [8.7K]  copyright
        │   ├── [   0]  libgcrypt20
        │   │   └── [ 15K]  copyright
        │   ├── [   0]  libgmp10
        │   │   └── [2.0K]  copyright
        │   ├── [   0]  libgnutls30
        │   │   └── [ 21K]  copyright
        │   ├── [   0]  libgssapi3-heimdal
        │   │   └── [8.0K]  copyright
        │   ├── [   0]  libgssapi-krb5-2
        │   │   └── [ 57K]  copyright
        │   ├── [   0]  libhcrypto4-heimdal
        │   │   └── [8.0K]  copyright
        │   ├── [   0]  libheimbase1-heimdal
        │   │   └── [8.0K]  copyright
        │   ├── [   0]  libheimntlm0-heimdal
        │   │   └── [8.0K]  copyright
        │   ├── [   0]  libhx509-5-heimdal
        │   │   └── [8.0K]  copyright
        │   ├── [   0]  libidn11
        │   │   └── [ 17K]  copyright
        │   ├── [   0]  libjansson4
        │   │   └── [1.8K]  copyright
        │   ├── [   0]  libjson-c2
        │   │   └── [1.9K]  copyright
        │   ├── [   0]  libk5crypto3
        │   │   └── [ 57K]  copyright
        │   ├── [   0]  libkrb5-26-heimdal
        │   │   └── [8.0K]  copyright
        │   ├── [   0]  libkrb5-3
        │   │   └── [ 57K]  copyright
        │   ├── [   0]  libkrb5support0
        │   │   └── [ 57K]  copyright
        │   ├── [   0]  libldap-2.4-2
        │   │   └── [ 20K]  copyright
        │   ├── [   0]  liblzma5
        │   │   └── [ 15K]  copyright
        │   ├── [   0]  libnettle6
        │   │   └── [ 13K]  copyright
        │   ├── [   0]  libogg0
        │   │   └── [3.6K]  copyright
        │   ├── [   0]  libpcre3
        │   │   └── [2.7K]  copyright
        │   ├── [   0]  libpng12-0
        │   │   └── [4.6K]  copyright
        │   ├── [   0]  libpulse0
        │   │   └── [ 21K]  copyright
        │   ├── [   0]  libroken18-heimdal
        │   │   └── [8.0K]  copyright
        │   ├── [   0]  librtmp1
        │   │   └── [2.7K]  copyright
        │   ├── [   0]  libsasl2-2
        │   │   └── [3.0K]  copyright
        │   ├── [   0]  libsdl2-2.0-0
        │   │   └── [ 19K]  copyright
        │   ├── [   0]  libselinux1
        │   │   └── [3.1K]  copyright
        │   ├── [   0]  libsndfile1
        │   │   └── [4.5K]  copyright
        │   ├── [   0]  libsndio6.1
        │   │   └── [2.0K]  copyright
        │   ├── [   0]  libspeexdsp1
        │   │   └── [2.0K]  copyright
        │   ├── [   0]  libsqlite3-0
        │   │   └── [1.2K]  copyright
        │   ├── [   0]  libssl1.0.0
        │   │   └── [6.4K]  copyright
        │   ├── [   0]  libsystemd0
        │   │   └── [7.6K]  copyright
        │   ├── [   0]  libtasn1-6
        │   │   └── [3.3K]  copyright
        │   ├── [   0]  libvorbis0a
        │   │   └── [2.2K]  copyright
        │   ├── [   0]  libvorbisenc2
        │   │   └── [2.2K]  copyright
        │   ├── [   0]  libwayland-client0
        │   │   └── [1.6K]  copyright
        │   ├── [   0]  libwayland-cursor0
        │   │   └── [1.6K]  copyright
        │   ├── [   0]  libwayland-egl1-mesa
        │   │   └── [7.2K]  copyright
        │   ├── [   0]  libwind0-heimdal
        │   │   └── [8.0K]  copyright
        │   ├── [   0]  libwrap0
        │   │   └── [1.3K]  copyright
        │   ├── [   0]  libxau6
        │   │   └── [1.2K]  copyright
        │   ├── [   0]  libxcursor1
        │   │   └── [2.7K]  copyright
        │   ├── [   0]  libxdmcp6
        │   │   └── [1.2K]  copyright
        │   ├── [   0]  libxext6
        │   │   └── [ 10K]  copyright
        │   ├── [   0]  libxfixes3
        │   │   └── [2.3K]  copyright
        │   ├── [   0]  libxi6
        │   │   └── [4.3K]  copyright
        │   ├── [   0]  libxinerama1
        │   │   └── [3.6K]  copyright
        │   ├── [   0]  libxkbcommon0
        │   │   └── [3.5K]  copyright
        │   ├── [   0]  libxrandr2
        │   │   └── [3.6K]  copyright
        │   ├── [   0]  libxrender1
        │   │   └── [2.2K]  copyright
        │   ├── [   0]  libxss1
        │   │   └── [2.6K]  copyright
        │   ├── [   0]  libxxf86vm1
        │   │   └── [1.3K]  copyright
        │   └── [   0]  libzip4
        │       └── [2.1K]  copyright
        └── [   0]  icons
            └── [   0]  hicolor
                └── [   0]  256x256
                    └── [   0]  apps
                        └── [ 26K]  openrct2.png

78 directories, 178 files
@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented May 11, 2018

Binary delta updates, e.g., for continuous builds (only download the binary diff) using AppImageUpdate

(https://github.com/AppImage/AppImageKit/wiki#for-application-developers and https://github.com/AppImage/AppImageUpdate)

This is very interesting. @TheAssassin I'm liking AppImages more and more. I will certainly look more into this whenever I have some time. Is your demo sufficitent for linuxdeployqt to enable binary diffs? Is there anything needed on the side of the host which provides the AppImages? We ship with some assets that rarely change, can this prior knowledge be utilised to further reduce diffs?

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented May 11, 2018

@janisozaur you don't need to generate much, actually, our tools will just generate a single file called .zsync file which can be used to update any file to the one the .zsync file belongs to. There's no "I need to generate a binary delta file for every possible version in advance" process involved, like it's needed for other update systems. Read more on it here: https://github.com/AppImage/zsync2 https://github.com/AppImage/AppImageUpdate

What you need to provide to have our tools generate this .zsync file is so-called update information. When it is passed to appimagetool, it generates such a file, and lets you use it. Unfortunately, instead of using linuxdeployqt's -appimage option, one has to download appimagetool and pass it manually to it. But that's not really a big issue. On Travis CI, the update information can be guessed even. I'm not sure why it didn't get uploaded previously, but I'll try to fix that soon.

Edit: .zsync files are available now. You can try AppImageUpdate now with my OpenRCT2 AppImage.

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented May 11, 2018

I forgot to answer your questions.

Is there anything needed on the side of the host which provides the AppImages?

That's the key feature of zsync2: All you need is an HTTP server (or whatever else system we support, e.g., GitHub releases) serving the two files (AppImage and zsync file).

We ship with some assets that rarely change, can this prior knowledge be utilised to further reduce diffs?

Not really. AppImages are merely squashfs images with an executable header. These are the compressed. But in squashfs, all files are compressed separately. Also, they're block aligned, and the block sizes of the squashfs image and the zsync file are compatible, therefore single file changes should cause zsync2 to download only the blocks that have changed, leaving the rest alone.

By the way: regarding the homepage of OpenRCT2: See https://doesmysiteneedhttps.com/.

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented May 11, 2018

Oh, another point: You may visit us on IRC (#appimage on Freenode) in order to kick off the "upstream OpenRCT2 AppImage project". I can't really provide you full-time maintenance of an AppImage, but I'd rather show you how to do it. That'd yield a usable result quickly, I'd say.

@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented May 11, 2018

By the way: regarding the homepage of OpenRCT2: See https://doesmysiteneedhttps.com/.

I'm not sure what you mean. https://openrct2.io/ is the only way to reach it, and also:

$ LANG=C wget --spider http://openrct2.io/
Spider mode enabled. Check if remote file exists.
--2018-05-11 23:37:06--  http://openrct2.io/
Resolving openrct2.io (openrct2.io)... 104.27.184.176, 104.27.185.176, 2400:cb00:2048:1::681b:b8b0, ...
Connecting to openrct2.io (openrct2.io)|104.27.184.176|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://openrct2.io/ [following]
Spider mode enabled. Check if remote file exists.
--2018-05-11 23:37:06--  https://openrct2.io/
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Connecting to openrct2.io (openrct2.io)|104.27.184.176|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Remote file exists and could contain further links,
but recursion is disabled -- not retrieving.
@qwertychouskie

This comment has been minimized.

Copy link
Contributor

@qwertychouskie qwertychouskie commented May 11, 2018

openrct2.org supports HTTPS, but doesn't force it. Also, why are there two sites in the first place?

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented Aug 29, 2018

I'm bringing my AppImage feature branch up to date, and will send a PR shortly. Then, we'll have an AppImage build script that builds from source and creates an AppImage which won't work < Ubuntu 18.04.

@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented Aug 29, 2018

I read that part and found nothing related to C++17 or any other standard, but as C++17 was the mostly discussed issue here, I thought it was the blocking issue. Support for current LTS release of Ubuntu is enough.

@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented Aug 29, 2018

It appears from your branch that wget libfuse2 are required for AppImage to work? I can add them to our base docker image, it's much faster in CI this way.

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented Aug 29, 2018

Feel free to push to my branch, you have write access. I've sent a PR.

@jansegre

This comment has been minimized.

Copy link
Contributor

@jansegre jansegre commented Aug 29, 2018

Let me add my 2¢. From what I gather from https://docs.appimage.org/introduction/concepts.html#build-on-old-systems-run-on-newer-systems, building on other systems is considered best practice because it lowers the chance for an AppImage to break on older systems. Some cases require libs to go to an exclude list, so that the AppImage can use the lib provided by the system (distro) instead of bundling it.

That means that if C++17 (or whatever version) can bundle its dependencies (mostly libstdc++ or libc++ I think), and those are not on the exclude list, there should be no obvious reason for the image not to work on older distros. I see that libstdc++.so.6 is on the exclude list, but I'm not sure what version of the lib C++17 is built with, either way libc++ is not on the exclude list, so if building with that (Clang only?), it will get bundled and most likely work on older distros.

So, theoretically an AppImage for OpenRTC2 built with libc++ should work on Ubuntu 16.04 (and probably previous versions too). This sounds great. The original link only refers to this as a best practice, probably to maximize compatibility.

So, support for older distros can be unofficial/best-effort? I really don't see what's wrong with wanting to jump to C++20 when it's out, even if older distros cannot compile it, they can most likely run it.

@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented Aug 29, 2018

Building with clang does not imply using libc++, at least not on most Linuxes.

@jansegre

This comment has been minimized.

Copy link
Contributor

@jansegre jansegre commented Aug 29, 2018

I mean the other way, using libc++ requires building with Clang. I'm not really sure though.

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented Aug 29, 2018

@jansegre well it's not as easy to "bundle everything". That document in the docs is still a bit WIP, so here's the short version:

Bundling libraries like libc or libstdc++ is not as easy as it might seem at first. Some libraries just cannot be bundled but depend on these. Graphics drivers are an example for that. If a system graphics driver links to an older version of libstdc++, you can not combine it with the newer version, as changes are really high that the ABI of the newer libstdc++ has changed in an incompatible way.

Therefore we recommend to try to target the oldest system you want to support. You chose to ignore half of the Linux users (< Ubuntu 18.04), and that's okay, it's your choice. However, we consider it an anti-pattern to always try to use the most recent versions of $stuff, unless you really need to use that. Big projects such as LibreOffice or Firefox have recognized that, and target really old systems (CentOS 6 resp. 5). Their binaries work on virtually any Linux released after 2011 therefore.

It's not even as hard as it might seem at first. For instance, I am using boost::filesystem in linuxdeploy, which really helps in working with the filesystem. I wouldn't want to miss it. By using it, I can safely keep using C++11, linking boost::filesystem and the dependencies statically into our official binaries. Our binaries are built on trusty, and work for almost all AppImage builds.

Developers should really try to avoid breaking backwards compatibility in general. As said, it's completely your choice, but please beware of the consequences.

Again, as some don't seem to have understood it yet: AppImage does not have a problem with C++17. But your AppImage won't work for a majority of users on older systems.

Edit: This "bundle everything" stuff is being looked into by us constantly. For non-GUI applications, it's even doable as of now (we're even thinking about a linuxdeploy plugin for the purpose, see linuxdeploy/linuxdeploy#5). But in some cases, it's just not possible. We don't recommend it even if it works, as it adds a lot of bloat. AppImages aren't supposed to ship an entire distribution.

@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented Aug 29, 2018

But your AppImage won't work for a majority of users on older systems.

We won't reach an agreement here. If a user stays on an outdated system, they should not expect to get all the shiny new stuff and I feel no obligation to provide it for them, especially when it costs us limiting ourselves to old versions of software, whether compiler or libraries.

Edit: I see the point you're making regarding Firefox or LibreOffice, but I would consider them vastly more important compared to our project and they make different choices from ours, with more resources to spend on their DevOps. I don't see it as fair comparison.

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented Aug 29, 2018

@janisozaur again: I don't care what you do, as long as you are aware of the consequences. I'm not requesting any sort of backport, just trying to make you keep this in mind for the next time a new release of C++ is available.

And define "outdated". I disagree these people use outdated systems. I mean, their systems are still supported.

@pizza2004

This comment has been minimized.

Copy link
Contributor

@pizza2004 pizza2004 commented Aug 29, 2018

Personally, I feel that any time a new OS version releases that's free and someone stays on an old version past the first month of waiting for bug fixes or whatever despite being /able/ to upgrade is using outdated software and shouldn't expect to get any updates to anything. It's nice to be able to support older machines that can't run new software, like supporting a few years worth of macOS, but it's another thing entirely to do this weird nonsense of just sitting on half a decade old versions of libraries because people can't be bothered to update.

The only time I ever didn't update was when they removed RSS from the Mail app on OS X and I spent a year or two putting it off since I rarely used that machine anymore (it was my iMac and I typically used my MacBook Pro) and I used RSS in Mail basically daily. That should be even less of a problem in Linux though because an open platform means that it should be possible to add back any lost functionality yourself.

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented Aug 29, 2018

Personally, I feel that any time a new OS version releases that's free and someone stays on an old version past the first month of waiting for bug fixes or whatever despite being /able/ to upgrade is using outdated software and shouldn't expect to get any updates to anything.

That's a quite ignorant statement. Look at what e.g., Ubuntu do. They stabilize the upgrade process, and publish it together with the first point release. That's ~5 months of waiting period. Telling these people "it's your fault" is unfair.

Then there's university labs, workspaces etc., where older, still supported distros are used. Telling these people "your own fault, you can upgrade your computer" is ignorant.

You're quite arrogant if you treat your users like that, @pizza2004.

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented Aug 29, 2018

Anyway, all of this is off-topic, and I don't want to discuss it in here.

@probonopd

This comment has been minimized.

Copy link

@probonopd probonopd commented Aug 30, 2018

Corporate and institutional users stay on LTS releases for a long time. They want to use a mature operating system that doesn't change (and needs to be re-validated) every year.

But the users using those systems still want the latest applications.

@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented Aug 30, 2018

Corporate and institutional users stay on LTS releases for a long time.

I get that and I already said it makes sense in context of projects such as Firefox or LibreOffice, but corporate and institutional users are not a target of OpenRCT2, a game engine. If the project was of other nature, I would probably weigh the arguments at hand differently.

But the users using those systems still want the latest applications.

But you just said they (whether them personally or through some higher entity) deliberately stay on LTS, which means support, not necessarily features. See the example you provided: Firefox ESR ships security patches for extended periods of time, but it doesn't add new features.

They want to use a mature operating system that doesn't change

This is in direct contradiction to the previous quote.

"Still supported" does not mean "not outdated". We are free to decide that maintaining support is too burdensome for us, but as we are an open source project, users have the same freedom to backport all the new features to their outdated system so they can use our project, but it is simply too much work for us.

@probonopd

This comment has been minimized.

Copy link

@probonopd probonopd commented Aug 31, 2018

Operating system (stable, LTS) != applications running on top of it (latest releases). Think Windows 7 with the latest apps. But anyhow, I take your point that you don't intend to target all users of still-supported distributions.

@IntelOrca

This comment has been minimized.

Copy link
Contributor

@IntelOrca IntelOrca commented Aug 31, 2018

For Windows we statically link all dependencies including the C++ runtime, this maximises the compatibility and does not require the user to install any redistributable. I thought that Snap or App Image would allow the same thing for Linux?

@Ryder17z

This comment has been minimized.

Copy link

@Ryder17z Ryder17z commented Aug 31, 2018

I've seen linking to .so files but I don't know what good it is in this case. (windows dll equivalent)

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented Aug 31, 2018

@IntelOrca if you manage to link to libstdc++ statically using your build system, it should work fine. But I suspect unless you explicitly link to an older libc at the same time, at least on Linux, there will be issues.

I could imagine you build your own compiler and libstdc++ on e.g., trusty, where an older libc is the default, link libstdc++ statically, and hope for the best.

But, again, let's ask: is it worth the effort? I was already working on that, and decided it's not. Please let's stop discussing hypothetical issues, and work on the AppImage. I've sent you a build script in a PR a couple of days ago. All you need to do is integrate it into your own CI/CD system, and maybe optimize it a bit.

@probonopd

This comment has been minimized.

Copy link

@probonopd probonopd commented Sep 2, 2018

For Windows we statically link all dependencies including the C++ runtime, this maximises the compatibility and does not require the user to install any redistributable. I thought that Snap or App Image would allow the same thing for Linux?

For AppImage, you can bundle all libraries with the application that you can't expect to be "there" in the default installation of all still-supported Linux distributions in a recent enough version. The only rule is to compile on the oldest system (Linux distribution) you are still targeting (recommendation: target all still-supported distributions).

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented Sep 29, 2018

So... will you ever release AppImages (even if they won't work on Ubuntu <18.04)...?

@IntelOrca

This comment has been minimized.

Copy link
Contributor

@IntelOrca IntelOrca commented Sep 29, 2018

@TheAssassin If I can get them working we will add to release and app image repo. See #7957.

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented Dec 21, 2018

Did you manage to integrate AppImage builds in your release toolchain?

@janisozaur

This comment has been minimized.

Copy link
Member Author

@janisozaur janisozaur commented Dec 21, 2018

There was no effort spent on that.

@probonopd

This comment has been minimized.

Copy link

@probonopd probonopd commented Dec 21, 2018

What would it take to make you spending effort on that @IntelOrca @Gymnasiast? 💯

@Gymnasiast

This comment has been minimized.

Copy link
Member

@Gymnasiast Gymnasiast commented Dec 21, 2018

Time, mostly. We have a limited amount of it, so we have to set priorities.

@TheAssassin

This comment has been minimized.

Copy link
Contributor

@TheAssassin TheAssassin commented Dec 21, 2018

The script still "just runs" (it's standalone), just needs to be called somewhere.

@baryluk

This comment has been minimized.

Copy link

@baryluk baryluk commented Oct 22, 2019

Yeah, snap or AppImage would be a great for distributing via webpage. I personally like AppImage for it being dead easy and requiring zero dependencies in the host system basically, but snap could be an option too probably. The default build system should still be made to make distros and packaging easy without them tho.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.