Flatpak support

[Eonfge]

Intro

Flatpak is a framework for distributing desktop applications on Linux. It has been created by developers who have a long history of working on the Linux desktop, and is run as an independent open source project.

Introduction: Reasons to use Flatpak

Reason

In short, this: https://repology.org/metapackage/onionshare/versions

A good 80% of all Linux versions out there is running an older, possibly insecure version of Onionshare. As such, it would be preferable to move current deployment effort over to Flatpak. This would initially take a bit more effort, but in the long run it would ensure that all distributions always have the latest version of your application.

[micahflee]

I agree, the Linux packaging situation is a mess. How does Flatpak compare with Snapcraft? https://snapcraft.io/

[RoestVrijStaal]

I'd recommend AppImage, since at user's end, it doesn't require any "package manager", nor (activation of a) "runtime" nor (elevated) commands to run the application, in contrast with Flatpak or Snappy.

https://docs.appimage.org/introduction/index.html

[Eonfge]

Uhm. Flatpak vs Snap is still an unsettled battle. It's clear now that AppImage doesn't seem to cut it because it misses a centralized upgrading and distribution mechanism, but as far as FlatPak and Snap goes, all I can do, is point to research of others.

Large overview by AppImage themselves:
https://github.com/AppImage/AppImageKit/wiki/Similar-projects#comparison
In my point of view there is some bias in this breakdown, but most elements are accurate. Reader discretion advices.

Comparison Flatpak and Snap cross distro support (2018):
https://kamikazow.wordpress.com/2018/06/08/adoption-of-flatpak-vs-snap-2018-edition/

Theming and desktop integration (2017):
https://aozoeky4dglp5sh0-zippykid.netdna-ssl.com/wp-content/uploads/2017/07/corebird-app-theming.jpg

Theming and desktop integration (2018):
https://www.youtube.com/watch?v=V_3e_VWCGlU

Why did I initialized this conversation with asking about Flatpak? because it's strikes a healthy balance between vendor neutrality, functionality and popularity. Most FOSS/Libre applications right now are on FlatHub (Red Hat's repo) but you could host your own repo. Most proprietary applications are on the snapcraft.io/store though, which is a platform primarily controlled by Canonical. I appreciate Canonical for trying to make a linux store work (this is like their forth attempt), but I'm also hesitant to support them because most of their new ideas die quite quickly.

As for AppImage. It's fallen a bit by the wayside. It's uptake is slow, mostly because it has no system in place for updating existing users and it offers little support for packaging dependencies. Of the three, it's apparently the worst to work with.

As such, I tend to mostly use Flatpak packaged applications right now, but it can go both ways. Considering the FOSS/Libre nature Onion Share, it would make a little more sense to have it on Flatpak, but if that is the winning horse in the long run, I can't tell.

[edoardopigaiani]

Hi all, I stumbled upon this issue while taking a look at the repository (I'm joining as a translator) and I have a couple of questions.

Since Onionshare's critical point is privacy and security, and Snap recently had a problem with privilege escalation, is there a comparison from the security point of view of these two solutions?

[Eonfge]

@itschap
It's a bit to early to tell. Simply because Snap had a bug discovered, doesn't mean it's less secure then Flatpak. As far as security goes, both are quite new so the amount code reviewing will be quite limited. The benefit of something like .deb packages and APT is that they are both well tested. Still not fully secure of cause (only last month, somebody found an attack surface by changing headers in the http communication).

Both support sandboxing and have quite an extensive system of permissions to control those sandboxes. On paper, there is not really any difference as far as security is concerned. Of cause, there is still the discussion between SELinux and App Armor, but that is a bit beyond the discussion.

@micahflee
Thought about it some more. For me, the critical difference would be the platform independence of Flatpak. You, the TOR Project or Guardian Project could host their own Flatpak repository with software, even if that software is banned in some places. With Snaps, you are stuck with Canonical, who might decide that some marketshare is more important then the political values of your product. This is a serious consideration, since companies like Google, Apple and Microsoft have already decided to ban VPN's in countries who's economic growth is more important.

How to about hosting Flatpak:
http://docs.flatpak.org/en/latest/hosting-a-repository.html

So, in summary, I strongly advice you to use Flatpak. It's not provider bound and it ensures that all users have a fast, reliable way of upgrading their application.

[micahflee]

After looking into this a bit and talking to some Linux nerd friends, I agree with you @Eonfge that Flatpak is the way to go.

Also, I like the idea of cross-platform Linux sandboxing -- right now we do ship an AppArmor profile, but it's only enabled in Debian, not in Ubuntu or other distros because it doesn't quite work there.

[micahflee]

I added this to the 2.1 milestone, but it occurs to me it might be a lot more complicated than I first thought, specifically about how we deal with dependencies (like tor), which right now are installed diretly from apt/dnf package repos. I haven't taken a stab at making a package yet, but if it turns out to be complicated I might push this off to version 2.2.

[micahflee]

I'm starting to look into this. It seems like we should probably use the org.kde.Platform runtime and org.kde.Sdk SDK, because these come with Qt.

Here's relevant docs for shipping python pip dependencies:
http://docs.flatpak.org/en/latest/python.html

Except that PyQt5 won't work using that, so we need to compile that another way:
flatpak/flatpak-builder-tools#26

I'm still not sure how to go about bundling tor binaries. Maybe the best way is to build tor from source, actually...

It looks like we'll need a "Portal" for allowing access to files that are dragged into the window or selected from a file open dialog. This might mean that the Linux version (or at least Flatpak version) of OnionShare will need two separate "Add Files" and "Add Folder" buttons in share mode so they can use default system file selecte dialogs, just like the macOS version does for its sandbox. Apparently Qt5 portals work "transparently":
http://docs.flatpak.org/en/latest/sandbox-permissions.html

This is perhaps too big a project to finish in the next few days for the 2.1 release. Maybe we can make it though. Once it's successful, I'd be into updating onionshare.org and other documentation basically making this the preferred way to install OnionShare in Linux, and then stop focusing on any other Linux packaging (except for special cases like Tails).

[Eonfge]

@micahflee Sounds good. I did look a bit into flatpakking myself but on the whole I don't have enough experience in it to help you. Then again, it's cool to see my humble proposal take of.

Don't feel to pressured to complete it all in the first go, with 2.1. This is a more fundamental shift in distribution and it would be unwise to rush it.

[x80486]

Hello there! Hey, @micahflee, let me know if you need a hand. Point me to the branch that you are working on and I'll try to help if I can. It looks like this one should be easier than, for instance, Electron-based apps.

[micahflee]

Thank you @x80486, I'd gladly accept help with this. OnionShare is a fairly complicated python/PyQt5 project and has a bunch of dependencies, including Tor and Tor pluggable transports which will have to be compiled separately.

When I started working on this the first barrier I hit was just packaging a PyQt5 app at all. We can't install PyQt5 from PyPi because that only distributes binaries, so we have to compile it from source, and also compile Qt5 from source.

So I made a much simpler example PyQt5 project, along with flatpak packaging, here: https://github.com/micahflee/flatpak-example

I figure once I have that working, doing the rest of the work to figure out how to package OnionShare should be simpler. I still haven't gotten just my simple PyQt5 example to work yet though, but I haven't touched it in a few weeks. So if you have time, please go ahead and see if you can get it working!

[x80486]

OK, I started it already. It looks like the build might be straightforward, but I'm seeing this issue:

========================================================================
Building module onionshare in /home/x80486/Workshop/flathub/.flatpak-builder/build/onionshare-4
========================================================================
Running: python3 setup.py install --prefix=${FLATPAK_DEST}
/usr/lib/python3.7/distutils/dist.py:274: UserWarning: Unknown distribution option: 'include_package_data'
  warnings.warn(msg)
running install
running build
running build_py
creating build
creating build/lib
creating build/lib/onionshare
copying onionshare/__init__.py -> build/lib/onionshare
...
changing mode of /app/bin/onionshare to 755
changing mode of /app/bin/onionshare-gui to 755
running install_data
copying install/onionshare.desktop -> /usr/share/applications
error: could not create '/usr/share/applications/onionshare.desktop': Read-only file system
Error: module onionshare: Child process exited with code 1

My guess would be: the path /usr/share/applications is hard-coded, it should obey the --prefix value also.

I think the AppData XML file might need some very minor tweaks along with the application's ID, currently onionshare.desktop, but it must follow a reverse-DNS scheme, consisting of {tld}.{vendor}.{product}. I left the AppData XML that I started doing as a reference (will delete it later), you might want to merge what you have with some parts of this one. The same for the .desktop file.

I've also seen that renaming the AppData XML file and icon(s) to follow the application's ID is more convenient. And the last one, I started using the latest dependencies for PyQt5 and SIP, so if you know for sure a combination of those two that works, that might save some time down the path.

Let me know what you think.

[micahflee]

Nice, this is looking great!

I've never packaged anything for flatpak before. Does it make sense for all the packaging code to live in the github repo flathub/org.onionshare.OnionShare, or should it be part of this repo, so we can update the packaging as-needed with each release -- for example, what if we add or remove a dependency? If it's in flathub's github account, who owns that repo and approves PRs into it?

I'm assuming that onionshare will need some minor changes to work with flatpak -- probably the AppData XML and the desktop file, and possibly others stuff, like how onionshare finds the path of local resources in the share folder. Would you be willing to make a PR for onionshare with these changes, and once the flatpak packaging works with your onionshare branch, I can merge that PR in as well?

I notice your flathub.json(https://github.com/x80486/flathub/blob/org.onionshare.OnionShare/flathub.json) file says "only-arches": ["x86_64"]. OnionShare should work with any architecture so long as we can compile Tor for that architecture.

And I think we'll need to add Tor as a module too, as well as pluggable transports obfs4 and meek_lite. Right now for macOS and Windows packaging, the build script downloads Tor Browser and extracts the binaries from it into the OnionShare package (see here for example). For flatpak, I think we'll need to build these from source.

I also notice that the URL for downloading the OnionShare source code is from the github archive. Starting with version 2.1, we now release PGP-signed source packages that you can get directly from the website: https://onionshare.org/dist/2.1/ . The source packages from onionshare.org and from github should be the same (but with different hashes, because of different timestamps when creating them), except when I generate the one for onionshare.org, I make sure to PGP-verify the git tag using this script, so I'd prefer to pull source packages from there.

Thanks for working on this! Let me know if there's anything I can do to help.

[micahflee]

Oh, another tricky thing that you will run into is the browse dialog to add files and folders to share. OnionShare uses a Qt dialog that allows you to select both files and folders when you click the "Add" button in share mode.

But since flatpak will add a sandbox, and the sandbox probably can't access everything in the home folder, it probably needs to use a native browse dialog to choose files outside the sandbox, instead of the Qt one. I ran into this same problem when enabling the macOS sandbox, and so the macOS version actually has two separate buttons, an "Add Files" and "Add Folder" button, which use native dialogs.

[x80486]

I've never packaged anything for flatpak before. Does it make sense for all the packaging code to live in the github repo flathub/org.onionshare.OnionShare, or should it be part of this repo, so we can update the packaging as-needed with each release -- for example, what if we add or remove a dependency? If it's in flathub's github account, who owns that repo and approves PRs into it?

The manifest would be effectively the same. I think it's better to have it in Flathub since it's a widely known and reputable store...but it's really up to what you prefer. It's also worth to mention that reusing their infrastructure for building (and publishing) is a plus.

The repository is owned by Flathub, but you would have write access, so it would be like if you would own it...sort of.

I'm assuming that onionshare will need some minor changes to work with flatpak -- probably the AppData XML and the desktop file, and possibly others stuff, like how onionshare finds the path of local resources in the share folder. Would you be willing to make a PR for onionshare with these changes, and once the flatpak packaging works with your onionshare branch, I can merge that PR in as well?

I can make a pull request for the AppData XML and the desktop file, but I don't know enough Python to adventure on tweaking that. I'm more on the Erlang / Java side.

I notice your flathub.json(https://github.com/x80486/flathub/blob/org.onionshare.OnionShare/flathub.json) file says "only-arches": ["x86_64"]. OnionShare should work with any architecture so long as we can compile Tor for that architecture.

Yeah, but I always start with x86_64 builds since that's what works on my desktop. Later on I use Flathub's infrastructure to give it a try with the rest of the (supported) architectures.

And I think we'll need to add Tor as a module too, as well as pluggable transports obfs4 and meek_lite. Right now for macOS and Windows packaging, the build script downloads Tor Browser and extracts the binaries from it into the OnionShare package (see here for example). For flatpak, I think we'll need to build these from source.

I'll add those as well.

I also notice that the URL for downloading the OnionShare source code is from the github archive. Starting with version 2.1, we now release PGP-signed source packages that you can get directly from the website: https://onionshare.org/dist/2.1/ . The source packages from onionshare.org and from github should be the same (but with different hashes, because of different timestamps when creating them), except when I generate the one for onionshare.org, I make sure to PGP-verify the git tag using this script, so I'd prefer to pull source packages from there.

Noted, I'll modify that to pull the sources from that location.

[micahflee]

It took me forever but I finally merged #999

[4jNsY6fCVqZv]

There is a problem with publishing via Flathub. Flathub is a platform that also offers proprietary packages. Since I switched to Linux to only find free software in my Software Center, installing the Onionshare Flatpak unfortunately takes me away from this goal. When I install an app via Flathub, the proprietary apps I don't want are suddenly displayed in my Software Center. I don't want that. Therefore I ask you to offer an official version as a download e.g. via your website or another safe place as an alternative for people who are looking for 100% Free Software.

[Eonfge]

@4jNsY6fCVqZv I understand your frustration, as it is indeed clumsy to see non-libre software on the same page as proprietary software.

Still, this might be better addressed in the GNOME Software Center. It would be better if the Software Center has a search filtering where users can choose if they want to see free and/or non-free packages. Same as with age ratings. Some libre games on Flathub are 18+ for violence, and users might want to exclude those as well.

So in short, head over to the GNOME Project and see if there is an issue to address this legitimate concern: https://gitlab.gnome.org/GNOME/gnome-software/issues

[4jNsY6fCVqZv]

@Eonfge Thank you for your understanding!

I see the responsibility in the first place with the offering platform itself. And hiding does not mean that it is no longer there in the sources.
There is already an idea for a solution, on which however so far no public work is done.
flathub/flathub#691

As for your rating point, both Flathub and GNOME Software should support OARS.
https://hughsie.github.io/oars/

The responsibility for the implementation in the context of Flathub probably is with the maintainers of the Flathub packages themselves, if I understand this issue correctly.
flathub/flathub#1081

So do you see a solution that Onionshare can offer?

[micahflee]

I agree that it would be nice if Flathub maintained separate repositories for FLOSS and for proprietary packages. But that's an issue for Flathub to solve, and I don't think it's necessarily a good reason for OnionShare not to use Flathub.

[4jNsY6fCVqZv]

Thanks for your feedback!
Please take a look at my original comment, it's not that Onionshare should not use Flathub.
Since Flathub does not seem to act actively in the direction to maintain a separate repository for FLOSS packages, I would be glad if you offer an official version as a download e.g. via your website or another safe place as an alternative for people who are looking for 100% Free Software. So my request is about an Flatpak alternative in addition to Flathub. At least for the transition until Flathub also works for users like me.
I know it's extra work and I'm very grateful for what you're doing now!
Nevertheless, my wish remains that you, as a developer of Onionshare, not only appreciate my use case, but also actively support my attitude wherever possible.
What do you think of that?

[colorchestra]

Hi! Don't mean to bother but is there a rough guesstimate as to when a Flatpak package might become available? I have a few non-technical users on my hands that need to use Onionshare, but some of them are on Ubuntu LTS, on which Onionshare is not usable without manually building (which they cannot do) because of #1052.

I would also not mind helping if there is anything specific to work on to get a Flatpak package.

Thank you very much!

[micahflee]

@4jNsY6fCVqZv ahh I think I see what you mean. I'm still learning the ins and outs of flatpak, and I haven't used any repositories other than flathub myself. But maybe we can add a step to our release system to also publish the flathub package to https://onionshare.org/dist/, which is where we publish Windows and Mac installers.

And @colorchestra this issue has been idle for some time, but I'm hoping to finally include it in the 2.3 release. We don't have any sort of release date, but I'm hoping within the next few months.

I've finally started working on this for real. You can see my progress (and test it) in this repo: https://github.com/micahflee/org.onionshare.OnionShare

Making the flathub package also requires some changes to the OnionShare source itself to make it work, so I'm making those changes in this branch: https://github.com/micahflee/onionshare/tree/910_flatpak

Any help is appreciated.

[4jNsY6fCVqZv]

@micahflee Thank you very much for your great and important work on Onionshare!
If you can set up your publishing system so that the Flatpak is available at https://onionshare.org/dist/, that would be very supportive and a solution to my original ethical dilemma!

[colorchestra]

That would be perfect. So non-tech-savvy users on Linux (which exist) can find and install it right away, they shouldn't have to know what Flatpak is.

[micahflee]

@x80486 hey, I hope you're doing well!

I've finally been working on the flatpak package but I'm hitting an issue. In order for Tor bridges to work, we need to build a dependency, obfs4proxy, but it's implemented in golang and I'm just not sure how to proceed.

Here is the issue: https://github.com/micahflee/org.onionshare.OnionShare/issues/1

I'm wondering if you could help?

[x80486]

Hey, how is it going! It's nice that you are working on it.

I'll see if I can do something, but I have very little time these days. I think it could be straightforward: use the org.freedesktop.Sdk.Extension.golang Sdk extension and make a reference to the package in question. Have a look at this manifest.

As a side note, if you have plans to publish OnionShare to Flathub (recommended), I would say: send a Draft Pull Request to Flathub and there will be a lot of people that will help you with anything and everything.

[micahflee]

Thank you so much! This is very helpful.

[micahflee]

Version 2.3 will have a flatpak package! I'll still need to do more packaging work as part of the release.

[4jNsY6fCVqZv]

@micahflee Thanks a lot! You still support the idea of a direct download of the .flatpakref? As a possibility for users who are looking for 100% FLOSS and unfortunately can't find it at Flathub just yet.