Support building Flatpak application bundles #16169

aperezdc · 2016-11-28T19:56:15Z

This adds support for building Flatpak application bundles, which should be runnable on any Linux distribution with Flatpak support.

New Gulp tasks are added to prepare a directory with the needed files (vscode-linux-${arch}-prepare-flatpak), assembling the Flatpak bundles (vscode-linux-${arch}-flatpak), and for cleaning the build directories (clean-vscode-linux-${arch}-flatpak). This mimics how the Debian and RPM package creation works. Hence, building an application bundle is done with:

$ gulp vscode-linux-x64-flatpak
[...]
$ ls *.flatpak
com.visualstudio.code.oss-x86_64.flatpak

Installing and running the application is achieved with:

$ flatpak --user install --bundle com.visualstudio.code.oss-x86_64.flatpak
$ flatpak run com.visualstudio.code.oss

(Removing --user would install the application system-wide.)

The org.freedesktop.Platform runtime needs to be installed for running the application, and for making builds the org.freedesktop.Sdk runtime is needed as well. There are installation instructions in the Flatpak website; the short version is:

wget https://sdk.gnome.org/keys/gnome-sdk.gpg
flatpak --user remote-add --gpg-import=gnome-sdk.gpg gnome https://sdk.gnome.org/repo/
flatpak --user install org.freedesktop.Platform 1.4
flatpak --user install org.freedesktop.Sdk 1.4

The flatpak-bundler development dependency provides an asynchronous API to invoke flatpak-builder. The gulp-image-resize module is used to
incorporate scaling of the application icon to multiple sizes, and uses ImageMagick (or GraphicsMagick) under the hood, which is commonly available on most GNU/Linux distributions.

Instead of building the Electron dependencies, this reuses the io.atom.electron.BaseApp (source) bundle as base application. The contents of the base application are imported at build time, and then Electron and the prepared files added on top. The prebuilt base application will be fetched automatically using the URL specified in the baseFlatpakref attribute, and it is kindly hosted at Amazon S3 by Endless Computers.

Merging this would tackle issue #7112.

msftclas · 2016-11-28T19:56:18Z

Hi @aperezdc, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla.microsoft.com.

TTYL, MSBOT;

mattdangerw · 2016-11-28T20:17:01Z

build/gulpfile.vscode.linux.js

+	],
+	finishArgs: [
+		'--share=ipc', '--socket=x11',  // Allow showing X11 windows.
+		'--filesystem=home',            // Allow access to home directory.


I think for a text editor, exposing as much of the filesystem as possible would be a good idea. So use --filesystem=host instead. That's what gnome-builder does, for example.

mattdangerw · 2016-11-28T20:22:37Z

build/gulpfile.vscode.linux.js

+	finishArgs: [
+		'--share=ipc', '--socket=x11',  // Allow showing X11 windows.
+		'--filesystem=home',            // Allow access to home directory.
+		'--device=dri',                 // Allow OpenGL rendering.


Other finish args to consider

--socket=pulseaudio if the app or any app extensions need audio output

--share=network to install extensions using the network connection

--talk-name=org.freedesktop.Notifications if this even uses libnotify for desktop notifications

Lastly a bit weird, but adding --filesystem=/tmp will make sure you get the system tmp directory, which chromium/electron is using for its single check. https://github.com/electron/electron/blob/master/chromium_src/chrome/browser/process_singleton_posix.cc#L12

I think we can add all of these:

Indeed --share=network is a good idea, somehow I missed the fact that the application goes online to fetch extensions. Not to mention that it may want to push/pull from Git repositories!

As for notifications, VSCode itself does not use them, and checking the popular extensions page I do not see how any of them would like to use them, and searching for “notification” brings up zero results... but anyway it's a small allowance, but it doesn't hurt having it.

Funnily enough, there's a few extensions which use audio devices, so let's add that, too!

aperezdc · 2016-11-28T22:22:01Z

Pushed an updated version with @mattdangerw's suggestions applied.

aperezdc · 2016-11-28T23:37:12Z

...and updated again to symlink /app/bin/code-oss to the launcher shell script at /app/share/code-oss/bin/code-oss instead of symlinking to the Electron binary. This enables passing command line arguments normally, and things like the following do work:

flatpak run com.microsoft.CodeOSS --wait --goto file.js:10:3

(Probably people using the Flatpak will want to do alias code='flatpak run com.microsoft.CodeOSS' as a shortcut 😉)

Tyriar · 2016-11-29T19:52:30Z

@aperezdc we can't automatically install a reliable bin command due to the lack of sudo right? Is this typical for flatpak usage, to have a long command to run?

aperezdc · 2016-11-30T16:54:25Z

@Tyriar Sorry, I do not understand what do you mean exactly... Are you wondering about the flatpak binary and how to get it into the CI? If that is what you are worried about, there is a documented way which doesn't require sudo:

addons:
  apt:
    packages:
      - imagemagick
      - flatpak

I'll try to add the needed bits to the .travis.yml file myself, and one more commit to PR — it seems like a good idea to have it done already as part of this set of changes.

Or do you mean something else?

aperezdc · 2016-11-30T17:06:59Z

...but I see that .travis.yml does not call any of the tasks that build Linux packages, my previous comment was quite dumb.

@Tyriar I think now I get what you meant. And no, we cannot install things in the host machine when using Flatpak, because Flatpak bundles are self-contained and the applications run in a sandbox. Not even the libraries from the host are used: the ones from the runtime/version specified at build time are used, which ensures the application will keep running regardless of what changes in the host. This is by design. On the upside, notice how the .desktop file, icons, and a few other things around are renamed to contain com.microsoft.CodeOSS: Flatpak will export those files out of the bundle when installing, and place them in a location where the desktop shell will notice them. Even if we don't install a launcher script under /usr/bin the user will see the launcher being added to their applications list in any desktop environment which follows the XDG base applications spec (that includes GNOME, XFCE, Cinnamon, Mate, Budgie, Enlightenment, LXDE, and many others).

Tyriar · 2016-11-30T19:42:42Z

@aperezdc I was mainly just wondering is that's typical for using applications installed via flatpak, I have my answer now 😄

I'll look at merging this after the November/December end game happens as things are a bit busy currently. Could you sign the CLA in #16169 (comment) in the meantime? Thanks again!

aperezdc · 2016-11-30T23:27:25Z

I have signed the CLA a few minutes ago. Does @msftclas notice automatically or do we have to poke it somehow?

Tyriar · 2016-12-01T00:44:35Z

@aperezdc it normally happens automatically I think, I'll close and reopen the PR and see if that makes a difference.

msftclas · 2016-12-01T00:44:43Z

Hi @aperezdc, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!

The agreement was validated by Microsoft and real humans are currently evaluating your PR.

TTYL, MSBOT;

Tyriar · 2016-12-01T00:44:59Z

There we go 😄

Tyriar

Most of the 1.8 release stuff is out of the way so I can look at this again 😄

Tyriar · 2016-12-13T19:26:20Z

build/gulpfile.vscode.linux.js

+const linuxPackageRevision = Math.floor(new Date().getTime() / 1000);
+
+const flatpakManifest = {
+	appId: 'com.microsoft.CodeOSS',


Our application id in the other packages is code[-insiders].desktop, should this be the same or is this a new ID we would be introducing? This should be sourced from https://github.com/Microsoft/vscode/blob/master/product.json

Flatpak application identifiers need to be reverse-domain style. Now that I come back to review this, I see that we could reuse com.visualstudio.code.oss (it is the value for darwinBundleIdentifier).

Yeah reusing that should be fine for now.

Tyriar · 2016-12-13T19:27:47Z

build/gulpfile.vscode.linux.js

+	appId: 'com.microsoft.CodeOSS',
+	sdk: 'org.freedesktop.Sdk',
+	runtime: 'org.freedesktop.Sdk',
+	runtimeVersion: '1.4',


What version is this for?

This is the version of the runtime bundle that the application needs. It is used both for running (org.freedesktop.Platform) and building (org.freedesktop.Sdk). The current version is 1.4. Setting this ensures that the application will be always ran using an userland which conforms exactly to the ABI exposed by the 1.4 version of the runtime.

Tyriar · 2016-12-13T19:28:22Z

build/gulpfile.vscode.linux.js

+	runtimeVersion: '1.4',
+	base: 'io.atom.electron.BaseApp',
+	baseFlatpakref: 'https://s3-us-west-2.amazonaws.com/electron-flatpak.endlessm.com/electron-base-app-master.flatpakref',
+	command: "code-oss",


command: product.applicationName

Tyriar · 2016-12-13T19:28:39Z

build/gulpfile.vscode.linux.js

+	baseFlatpakref: 'https://s3-us-west-2.amazonaws.com/electron-flatpak.endlessm.com/electron-base-app-master.flatpakref',
+	command: "code-oss",
+	symlinks: [
+		['/share/code-oss/bin/code-oss', '/bin/code-oss'],


This should use product.applicationName instead of code-oss

Tyriar · 2016-12-13T19:31:01Z

build/gulpfile.vscode.linux.js

+	const destination = '.build/linux/flatpak/' + flatpakArch;
+
+	return function () {
+		const all = [16, 24, 32, 48, 64, 128, 192, 256, 512].map(function (size) {


Are all these sizes necessary? The smaller ones in particular would probably look pretty crappy being scaled like this?

Fortunately the icon has clean, flat shapes and it scales down quite well. This is how the 16x16 px version looks:

24x24 px:

and 32x32 px:

Tyriar · 2016-12-13T19:39:29Z

resources/linux/code.appdata.xml

+			<image>https://code.visualstudio.com/home/home-screenshot-linux-lg.png</image>
+			<caption>Editing a TypeScript file and searching for extensions.</caption>
+		</screenshot>
+		<screenshot>


Let's just keep the default screenshot for now, I've created #17131 to follow this up.

Updated to have only the default screenshot.

Tyriar · 2016-12-13T19:40:20Z

resources/linux/code.appdata.xml

+	<summary>Code editor for developers supporting integration with existing tools</summary>
+	<description>
+		<p>
+			Visual Studio Code is a lightweight but powerful source code editor which


Let's go with the standard description now as defined in #17130, also could you put it all one line? <p>Visual Studio...

Using the description from #17130 results in:

vscode % appstream-util validate resources/linux/code.appdata.xml resources/linux/code.appdata.xml: FAILED: • tag-missing : <translation> not specified • tag-missing : <update_contact> is not present • style-invalid : <p> cannot contain a hyperlink [Visual Studio Code is a new choice of tool that combines the simplicity of a code editor with what developers need for the core edit-build-debug cycle. See https://code.visualstudio.com/docs/setup/linux for installation instructions and FAQ.]

Using validate-relax still passes. Should we keep the URL in the description, or can I just remove the last sentence?

Removing the last line would be great. I'll fix up <translation> later and can you add this:

<update_contact>vscode-linux@microsoft.com</update_contact>

Tyriar · 2016-12-13T19:40:49Z

resources/linux/code.appdata.xml

+<?xml version="1.0" encoding="UTF-8"?>
+<component type="desktop">
+	<id>@@NAME@@.desktop</id>
+	<metadata_license>CC-BY-SA-3.0</metadata_license>


@chrisdias do we have a preferred license for metadata files like these?

Tyriar · 2016-12-13T19:43:48Z

resources/linux/code.appdata.xml

+	<project_license>MIT</project_license>
+	<name>@@NAME_LONG@@</name>
+	<url type="homepage">https://code.visualstudio.com</url>
+	<summary>Code editor for developers supporting integration with existing tools</summary>


As in #17130, let's make this "Code editing. Redefined." for now

Tyriar · 2016-12-13T19:45:38Z

resources/linux/code.appdata.xml

+<component type="desktop">
+	<id>@@NAME@@.desktop</id>
+	<metadata_license>CC-BY-SA-3.0</metadata_license>
+	<project_license>MIT</project_license>


This will need to change to be based on product.json but I just realized we don't do it for the rpm package so this is work I'll need to follow up 😄 #17133

Picking the value from product.json once that PR is landed will be just perfect 👍

Tyriar · 2016-12-13T20:04:50Z

resources/linux/code.appdata.xml

@@ -0,0 +1,46 @@
+<?xml version="1.0" encoding="UTF-8"?>


Could you check that this validates against:

appstream-util validate code.appdata.xml

Checked, and fixed a couple of issues. Right now it says:

vscode % appstream-util validate resources/linux/code.appdata.xml resources/linux/code.appdata.xml: FAILED: • tag-missing : <translation> not specified • tag-missing : <update_contact> is not present Validation of files failed

These are optional tags, and validate-relax succeds:

vscode % appstream-util validate-relax resources/linux/code.appdata.xml resources/linux/code.appdata.xml: OK vscode %

This adds support for building Flatpak application bundles, which should be runnable on any Linux distribution with Flatpak support. See the page at http://flatpak.org/getting.html New Gulp tasks are added to prepare a directory with the needed files (vscode-linux-${arch}-prepare-flatpak), assembling the Flatpak bundles (vscode-linux-${arch}-flatpak), and for cleaning the build directories (clean-vscode-linux-${arch}-flatpak). This mimics how the Debian and RPM package creation works. Hence, building an application bundle is done with: $ gulp vscode-linux-x64-flatpak [...] $ ls *.flatpak com.visualstudio.code.oss-x86_64.flatpak Installing and running the application is achieved with: $ flatpak --user install --bundle com.visualstudio.code.oss-x86_64.flatpak $ flatpak run com.visualstudio.code.oss (Removing "--user" would install the application system-wide. Also note that the "org.freedesktop.Platform" runtime needs to be installeed, check the Flatpak website for instructions.) The "flatpak-bundler" development dependency provides an asynchronous API to invoke "flatpak-builder". The "gulp-image-resize" module is used to incorporate scaling of the application icon to multiple sizes, and uses ImageMagick (or GraphicsMagick) under the hood, which is commonly available on most GNU/Linux distributions. Instead of building the Electron dependencies ourselves, this uses the io.atom.electron.BaseApp bundle as base application. Instead of building the Electron dependencies, this reuses the "io.atom.electron.BaseApp" bundle as base application. The contents of the base application are imported at build time, and then Electron and the prepared files added on top. The prebuilt base application will be fetched automatically using the URL specified in the "baseFlatpakref" attribute of the manifest, and it is kindly hosted at Amazon S3 by Endless Computers (https://endlessm.com). The sources for the base application can be found at: https://github.com/endlessm/electron-flatpak-base-app/

aperezdc · 2016-12-14T14:53:12Z

@Tyriar: I have pushed again, this time only with the commit that adds the initial support (the rest of the commits were stashed to a flatpak-follow-up branch, to be submitted later). All the issues mentioned in the review (thanks!) were addressed.

Tyriar · 2016-12-14T18:06:53Z

@aperezdc thanks for the great work 😄 we probably won't distribute flatpaks just yet, but they should now be available for OSS builds! Looking forward to getting appdata.xml in as well.

msftclas added the cla-required label Nov 28, 2016

mattdangerw reviewed Nov 28, 2016

View reviewed changes

Tyriar self-assigned this Nov 28, 2016

Tyriar added this to the January 2017 milestone Nov 30, 2016

Tyriar closed this Dec 1, 2016

Tyriar reopened this Dec 1, 2016

msftclas added the cla-already-signed label Dec 1, 2016

Tyriar removed the cla-required label Dec 1, 2016

nerijus mentioned this pull request Dec 12, 2016

Support building Flatpak application bundles yakyak/yakyak#507

Closed

Tyriar mentioned this pull request Dec 13, 2016

Add screenshots to Linux packages #17131

Closed

Tyriar requested changes Dec 13, 2016

View reviewed changes

Tyriar reviewed Dec 13, 2016

View reviewed changes

Tyriar merged commit f087eea into microsoft:master Dec 14, 2016

This was referenced Dec 16, 2016

Add an AppData XML data file to Linux builds #17369

Merged

Add a few niceties for making Flatpak builds of VSCode #17370

Merged

xaiki mentioned this pull request Dec 20, 2016

support flatpak butterproject/butter-desktop#558

Closed

xaiki mentioned this pull request Dec 20, 2016

Support Flatpak (for Linux) butterproject/butter-desktop#559

Closed

github-actions bot locked and limited conversation to collaborators Mar 27, 2020

Support building Flatpak application bundles #16169

Support building Flatpak application bundles #16169

Conversation

aperezdc commented Nov 28, 2016 • edited

msftclas commented Nov 28, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aperezdc commented Nov 28, 2016

aperezdc commented Nov 28, 2016

Tyriar commented Nov 29, 2016 • edited

aperezdc commented Nov 30, 2016 • edited

aperezdc commented Nov 30, 2016 • edited

Tyriar commented Nov 30, 2016

aperezdc commented Nov 30, 2016

Tyriar commented Dec 1, 2016

msftclas commented Dec 1, 2016

Tyriar commented Dec 1, 2016

Tyriar left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aperezdc commented Dec 14, 2016

Tyriar commented Dec 14, 2016

aperezdc commented Nov 28, 2016 •

edited

Tyriar commented Nov 29, 2016 •

edited

aperezdc commented Nov 30, 2016 •

edited

aperezdc commented Nov 30, 2016 •

edited

Tyriar left a comment •

edited