GIMP's file dialogs not showing bookmarks #328

Open
Jehan opened this Issue Sep 26, 2016 · 27 comments

Comments

Projects
None yet
5 participants
@Jehan

Jehan commented Sep 26, 2016

Hi,

So I don't know if the problem is with the Flatpak of GIMP or with Flatpak in general. But the file dialogs don't show the usual bookmarks which are usually seen across GTK+ applications.

Here is GIMP master from my dev environment:
gimp-dev

And here the nightly Flatpak-ed GIMP:
gimp-flatpak

@matthiasclasen

This comment has been minimized.

Show comment
Hide comment
@matthiasclasen

matthiasclasen Sep 26, 2016

Collaborator

For that to work, gimp needs to be ported to GTK+3 and use the GtkFileChooserNative api

Collaborator

matthiasclasen commented Sep 26, 2016

For that to work, gimp needs to be ported to GTK+3 and use the GtkFileChooserNative api

@Jehan

This comment has been minimized.

Show comment
Hide comment
@Jehan

Jehan Sep 26, 2016

For that to work

You mean in flatpak? Because in non-flatpak GIMP (GTK+2), it already works. We are planning the port to GTK+3, but that's in 2 major release.

Shouldn't this be just some kind of permission thing? (like letting the software in the flatpak environment access the right config file)

Jehan commented Sep 26, 2016

For that to work

You mean in flatpak? Because in non-flatpak GIMP (GTK+2), it already works. We are planning the port to GTK+3, but that's in 2 major release.

Shouldn't this be just some kind of permission thing? (like letting the software in the flatpak environment access the right config file)

@matthiasclasen

This comment has been minimized.

Show comment
Hide comment
@matthiasclasen

matthiasclasen Sep 26, 2016

Collaborator

Sure, if you give the app sufficient filesystem access, that will work too.

Collaborator

matthiasclasen commented Sep 26, 2016

Sure, if you give the app sufficient filesystem access, that will work too.

@alexlarsson

This comment has been minimized.

Show comment
Hide comment
@alexlarsson

alexlarsson Sep 27, 2016

Member

It seems like your issue is that you can't see gvfs mounts.
Can you try something like
flatpak run --talk-name=org.gtk.vfs --talk-name=org.gtk.vfs.* org.gimp.GimpDevel

Member

alexlarsson commented Sep 27, 2016

It seems like your issue is that you can't see gvfs mounts.
Can you try something like
flatpak run --talk-name=org.gtk.vfs --talk-name=org.gtk.vfs.* org.gimp.GimpDevel

@alexlarsson

This comment has been minimized.

Show comment
Hide comment
@alexlarsson

alexlarsson Sep 27, 2016

Member

and if that works you can make it permanent using flatpak override, or if you want we can add it to the build always.

Member

alexlarsson commented Sep 27, 2016

and if that works you can make it permanent using flatpak override, or if you want we can add it to the build always.

@Jehan

This comment has been minimized.

Show comment
Hide comment
@Jehan

Jehan Sep 27, 2016

Running:

flatpak run --talk-name=org.gtk.vfs --talk-name=org.gtk.vfs.* org.gimp.GimpDevel

I can now see my 2 remote owncloud mounts, but I still don't see all the bookmarks which I normally see.

Jehan commented Sep 27, 2016

Running:

flatpak run --talk-name=org.gtk.vfs --talk-name=org.gtk.vfs.* org.gimp.GimpDevel

I can now see my 2 remote owncloud mounts, but I still don't see all the bookmarks which I normally see.

@matthiasclasen

This comment has been minimized.

Show comment
Hide comment
@matthiasclasen

matthiasclasen Sep 27, 2016

Collaborator

gtk2 is looking for bookmarks in two places:

~/.config/gtk-3.0/bookmarks or
~/.gtk-bookmarks

Collaborator

matthiasclasen commented Sep 27, 2016

gtk2 is looking for bookmarks in two places:

~/.config/gtk-3.0/bookmarks or
~/.gtk-bookmarks

@alexlarsson

This comment has been minimized.

Show comment
Hide comment
@alexlarsson

alexlarsson Sep 27, 2016

Member

Gimp has --filesystem=host, so it should see everything though...

Member

alexlarsson commented Sep 27, 2016

Gimp has --filesystem=host, so it should see everything though...

@matthiasclasen

This comment has been minimized.

Show comment
Hide comment
@matthiasclasen

matthiasclasen Sep 27, 2016

Collaborator

The problem is that we respect XDG_CONFIG_HOME here.
So gimp tries to open
~/.var/app/org.gimp.GimpDevel/config/gtk-3.0/bookmarks
When the actual bookmarks file lives in
~/.config/gtk-3.0/bookmarks

Collaborator

matthiasclasen commented Sep 27, 2016

The problem is that we respect XDG_CONFIG_HOME here.
So gimp tries to open
~/.var/app/org.gimp.GimpDevel/config/gtk-3.0/bookmarks
When the actual bookmarks file lives in
~/.config/gtk-3.0/bookmarks

@Jehan

This comment has been minimized.

Show comment
Hide comment
@Jehan

Jehan Sep 27, 2016

Does flatpak set XDG_CONFIG_HOME? That means that a flatpaked GIMP won't even use the user's config files. That's not very good.

Also why ~/.var? Even if you wanted to have a separate environment by default for flatpak, I would expect it to be either under the user's $XDG_CONFIG_HOME/flatpak or $XDG_DATA_HOME/flatpak (so the directory above would be by default: ~/.config/flatpak/app/org.gimp.GimpDevel/config/gtk-3.0/bookmarks or ~/.local/share/flatpak/app/org.gimp.GimpDevel/config/gtk-3.0/bookmarks).

But then a permission for a flatpak app to use the actual user's XDG environment would be good.

Jehan commented Sep 27, 2016

Does flatpak set XDG_CONFIG_HOME? That means that a flatpaked GIMP won't even use the user's config files. That's not very good.

Also why ~/.var? Even if you wanted to have a separate environment by default for flatpak, I would expect it to be either under the user's $XDG_CONFIG_HOME/flatpak or $XDG_DATA_HOME/flatpak (so the directory above would be by default: ~/.config/flatpak/app/org.gimp.GimpDevel/config/gtk-3.0/bookmarks or ~/.local/share/flatpak/app/org.gimp.GimpDevel/config/gtk-3.0/bookmarks).

But then a permission for a flatpak app to use the actual user's XDG environment would be good.

@alexlarsson

This comment has been minimized.

Show comment
Hide comment
@alexlarsson

alexlarsson Sep 27, 2016

Member

Generally flatpak apps don't have any access to $HOME, so we set the xdg vars to make it write to the only writable location that is guaranteed to be there, which is ~/.var/app/$app-id. It also means all the files from each app is generally in one location so you can easily delete the files if you uninstall it, rather than scattered over the entire filesystem.

Member

alexlarsson commented Sep 27, 2016

Generally flatpak apps don't have any access to $HOME, so we set the xdg vars to make it write to the only writable location that is guaranteed to be there, which is ~/.var/app/$app-id. It also means all the files from each app is generally in one location so you can easily delete the files if you uninstall it, rather than scattered over the entire filesystem.

@Jehan

This comment has been minimized.

Show comment
Hide comment
@Jehan

Jehan Sep 27, 2016

I'm not sure I like this part very much. It goes against all the freedesktop specs and logics of where an app should save its config, data, cache, etc.

GIMP users would just lose all their configuration, and we'd have to ask them to move it by hand. Not user-friendly (and prone to all issues of user-moved files). GIMP also manages configuration update (like from 2.8 to 2.10, etc.) so you can't just move a GIMP 2.8 config directory to 2.10 config dir. GIMP searches for an existing 2.8 configuration and migrates it if it finds one. And that means adding this new logics will mess up with this as well.

Couldn't there be some kind of permission in flatpak which could allow the software to create or write into a specific directory? (which would be $XDG_CONFIG_HOME/GIMP/2.9/ for GIMP dev) This would be set by the flatpak package creator and the app would not have access to other directories (if you don't want to give access, though as you said, in GIMP case, you made it see everything).

Jehan commented Sep 27, 2016

I'm not sure I like this part very much. It goes against all the freedesktop specs and logics of where an app should save its config, data, cache, etc.

GIMP users would just lose all their configuration, and we'd have to ask them to move it by hand. Not user-friendly (and prone to all issues of user-moved files). GIMP also manages configuration update (like from 2.8 to 2.10, etc.) so you can't just move a GIMP 2.8 config directory to 2.10 config dir. GIMP searches for an existing 2.8 configuration and migrates it if it finds one. And that means adding this new logics will mess up with this as well.

Couldn't there be some kind of permission in flatpak which could allow the software to create or write into a specific directory? (which would be $XDG_CONFIG_HOME/GIMP/2.9/ for GIMP dev) This would be set by the flatpak package creator and the app would not have access to other directories (if you don't want to give access, though as you said, in GIMP case, you made it see everything).

@Jehan

This comment has been minimized.

Show comment
Hide comment
@Jehan

Jehan Sep 27, 2016

Oh wait. Here it needs to see some other $XDG_DATA_HOME data, not GIMP's.

Well still it shows there is a problem more global. For proper integration with the desktop, you can't just lose all your folder bookmarks. And I'm sure we would find other issues. Basically I don't believe it is a good idea to override XDG variables. This defeats the purpose of this spec.

Jehan commented Sep 27, 2016

Oh wait. Here it needs to see some other $XDG_DATA_HOME data, not GIMP's.

Well still it shows there is a problem more global. For proper integration with the desktop, you can't just lose all your folder bookmarks. And I'm sure we would find other issues. Basically I don't believe it is a good idea to override XDG variables. This defeats the purpose of this spec.

@matthiasclasen

This comment has been minimized.

Show comment
Hide comment
@matthiasclasen

matthiasclasen Sep 27, 2016

Collaborator

Well still it shows there is a problem more global. For proper integration with the desktop, you can't
just lose all your folder bookmarks. And I'm sure we would find other issues. Basically I don't believe it is a good idea to override XDG variables. This defeats the purpose of this spec.

The proper desktop integration for sandboxed apps is via portals, in this case the file chooser portal. The bookmarks belong to the user / session and will be available just fine on the portal side.

Collaborator

matthiasclasen commented Sep 27, 2016

Well still it shows there is a problem more global. For proper integration with the desktop, you can't
just lose all your folder bookmarks. And I'm sure we would find other issues. Basically I don't believe it is a good idea to override XDG variables. This defeats the purpose of this spec.

The proper desktop integration for sandboxed apps is via portals, in this case the file chooser portal. The bookmarks belong to the user / session and will be available just fine on the portal side.

@alexlarsson

This comment has been minimized.

Show comment
Hide comment
@alexlarsson

alexlarsson Sep 28, 2016

Member

I think its a somewhat of a problem that $XDG_DATA_HOME contains both shared (desktop files, mimetypes, thumbnails, bookmarks, recent files) and per-app configuration, because is a more separated "app" world we really do want the per-app data to be isolated, but on the way there apps are still going to be given homedir access, which means they could use at some of the shared configuration.

We can't use all of though. For instance, the desktop files and mime files are inherently tied to the host /usr, and for things like recent files sharing gets complicated because apps may see different subsets of the filesystem. Also, anything that stores binaries or plugins there will not work because the dependencies in that may be incompatible with the one in the runtime.

Some libraries also store config in ~/.config. Such configuration can easily break as you change library versions (i.e. its often compatible going forward, but not backwards). This also breaks with flatpak, which would run multiple library versions, maybe even in parallel, against a shared config dir.

So, I think the current way flatpak works is the least problematic as a general solution.
We could add library specific workarounds though, so that you can use the shared config for something you have verified is safe to share. For instance, we could have the app specify that ~/.config/gtk-3.0 should be mounted as ~/.var/app/$app/config/gtk-3.0. Of course, that would mean guaranteeing that these config files are both forward and backwards compat for all gtk3 versions forever. And also that they are self-contained and don't rely on other things from the host. For instance, what if the settings.ini file in that directory refers to an icon theme in /usr that the app can't see.

Member

alexlarsson commented Sep 28, 2016

I think its a somewhat of a problem that $XDG_DATA_HOME contains both shared (desktop files, mimetypes, thumbnails, bookmarks, recent files) and per-app configuration, because is a more separated "app" world we really do want the per-app data to be isolated, but on the way there apps are still going to be given homedir access, which means they could use at some of the shared configuration.

We can't use all of though. For instance, the desktop files and mime files are inherently tied to the host /usr, and for things like recent files sharing gets complicated because apps may see different subsets of the filesystem. Also, anything that stores binaries or plugins there will not work because the dependencies in that may be incompatible with the one in the runtime.

Some libraries also store config in ~/.config. Such configuration can easily break as you change library versions (i.e. its often compatible going forward, but not backwards). This also breaks with flatpak, which would run multiple library versions, maybe even in parallel, against a shared config dir.

So, I think the current way flatpak works is the least problematic as a general solution.
We could add library specific workarounds though, so that you can use the shared config for something you have verified is safe to share. For instance, we could have the app specify that ~/.config/gtk-3.0 should be mounted as ~/.var/app/$app/config/gtk-3.0. Of course, that would mean guaranteeing that these config files are both forward and backwards compat for all gtk3 versions forever. And also that they are self-contained and don't rely on other things from the host. For instance, what if the settings.ini file in that directory refers to an icon theme in /usr that the app can't see.

@Jehan

This comment has been minimized.

Show comment
Hide comment
@Jehan

Jehan Sep 29, 2016

I understand wanting to isolate per-app data, but I really can't say I like the hijacking of the XDG directories to create another directory with the same purpose. It took years to finally have most applications follow this specification and getting away from the every-app-with-custom-config-dir pain, and now breaking this is like a step back. Applications are not supposed to redefine these variables (if set, they are user-defined).

Some libraries also store config in ~/.config. Such configuration can easily break as you change library versions (i.e. its often compatible going forward, but not backwards).

Well IMO, that's up to these libraries to deal with this, for instance by using versionned directory names. That's what we do with GIMP (you'll notice that our config dir is $XDG_CONFIG_HOME/GIMP/{version}). This way, we have no compatibility problem forward nor backward.

For instance, we could have the app specify that ~/.config/gtk-3.0 should be mounted as ~/.var/app/$app/config/gtk-3.0.

Can this be done also for the software config dir? Also since you redefine $XDG_CONFIG_HOME, it would be good if the old value is kept around. So in your previous example $XDG_CONFIG_HOME/gtk-3.0/ (i.e. the old value of $XDG_CONFIG_HOME) should be mounted as ~/.var/app/$app/config/gtk-3.0/.

Jehan commented Sep 29, 2016

I understand wanting to isolate per-app data, but I really can't say I like the hijacking of the XDG directories to create another directory with the same purpose. It took years to finally have most applications follow this specification and getting away from the every-app-with-custom-config-dir pain, and now breaking this is like a step back. Applications are not supposed to redefine these variables (if set, they are user-defined).

Some libraries also store config in ~/.config. Such configuration can easily break as you change library versions (i.e. its often compatible going forward, but not backwards).

Well IMO, that's up to these libraries to deal with this, for instance by using versionned directory names. That's what we do with GIMP (you'll notice that our config dir is $XDG_CONFIG_HOME/GIMP/{version}). This way, we have no compatibility problem forward nor backward.

For instance, we could have the app specify that ~/.config/gtk-3.0 should be mounted as ~/.var/app/$app/config/gtk-3.0.

Can this be done also for the software config dir? Also since you redefine $XDG_CONFIG_HOME, it would be good if the old value is kept around. So in your previous example $XDG_CONFIG_HOME/gtk-3.0/ (i.e. the old value of $XDG_CONFIG_HOME) should be mounted as ~/.var/app/$app/config/gtk-3.0/.

@Jehan

This comment has been minimized.

Show comment
Hide comment
@Jehan

Jehan Sep 29, 2016

P.S.: and therefore $XDG_CONFIG_HOME/GIMP/ should be mounted as ~/.var/app/$app/config/GIMP/ too.

Jehan commented Sep 29, 2016

P.S.: and therefore $XDG_CONFIG_HOME/GIMP/ should be mounted as ~/.var/app/$app/config/GIMP/ too.

@Jehan

This comment has been minimized.

Show comment
Hide comment
@Jehan

Jehan Oct 12, 2016

So I have been playing with flatpak a few days ago, making a flatpak for GIMP since I would like us to have an official flatpak for our releases. But this issue is really blocking me because that's not good user experience. I could actually sum it up in 2 issues:

1/ Access to desktop data, for instance user bookmarks, thumbnails… I guess this should likely be solved by portals as @matthiasclasen was noting.

2/ Access to its own data as specified in the Freedesktop specs (and in particular the XDG directories). This one, I am very annoyed because I completely disagree with "hijacking" the XDG variables and bypassing the user data (if one had brushes, plugins, etc. they would all be lost if one used a distribution package, then later a flatpak. Well they are not actually lost, but from the user's point of view, that's searching on the web, asking on forums until finally discovering one has to manually move files from a hidden directory to another => bad experience).

The GEGL cache is another very good example of why hijacking XDG variable is wrong. GEGL could theoretically share its buffers between applications. Such usage doesn't exist yet but it has been explained as one of the possible future. For instance you'd want to share a buffer between GIMP and Darktable, allowing to work and see edits in real time in both software in same time. If GEGL now loads its data from different directories, this goes against the whole logics of GEGL.
Similarly, checking other contents under ~/.var/app/org.gimp.GimpDevel/, I can also see the fontconfig cache being duplicated instead of shared. That's bad too.

Could you please reconsider this whole overriding-XDG logics? Or at the very least propose an option so that a flatpak packager asks for a package to not override the XDG variables? This could require a --filesystem list of directories where a given module has the right to look up (which can be relative to a XDG environment variable, or to $HOME, or other common variables). Then this is not complete open-door, you don't give access to the whole $XDG_CONFIG_HOME, but to $XDG_CONFIG_HOME/GIMP/2.9/ for instance (and a few others, like the GEGL and fontconfig caches, etc.).

Jehan commented Oct 12, 2016

So I have been playing with flatpak a few days ago, making a flatpak for GIMP since I would like us to have an official flatpak for our releases. But this issue is really blocking me because that's not good user experience. I could actually sum it up in 2 issues:

1/ Access to desktop data, for instance user bookmarks, thumbnails… I guess this should likely be solved by portals as @matthiasclasen was noting.

2/ Access to its own data as specified in the Freedesktop specs (and in particular the XDG directories). This one, I am very annoyed because I completely disagree with "hijacking" the XDG variables and bypassing the user data (if one had brushes, plugins, etc. they would all be lost if one used a distribution package, then later a flatpak. Well they are not actually lost, but from the user's point of view, that's searching on the web, asking on forums until finally discovering one has to manually move files from a hidden directory to another => bad experience).

The GEGL cache is another very good example of why hijacking XDG variable is wrong. GEGL could theoretically share its buffers between applications. Such usage doesn't exist yet but it has been explained as one of the possible future. For instance you'd want to share a buffer between GIMP and Darktable, allowing to work and see edits in real time in both software in same time. If GEGL now loads its data from different directories, this goes against the whole logics of GEGL.
Similarly, checking other contents under ~/.var/app/org.gimp.GimpDevel/, I can also see the fontconfig cache being duplicated instead of shared. That's bad too.

Could you please reconsider this whole overriding-XDG logics? Or at the very least propose an option so that a flatpak packager asks for a package to not override the XDG variables? This could require a --filesystem list of directories where a given module has the right to look up (which can be relative to a XDG environment variable, or to $HOME, or other common variables). Then this is not complete open-door, you don't give access to the whole $XDG_CONFIG_HOME, but to $XDG_CONFIG_HOME/GIMP/2.9/ for instance (and a few others, like the GEGL and fontconfig caches, etc.).

@alexlarsson

This comment has been minimized.

Show comment
Hide comment
@alexlarsson

alexlarsson Oct 13, 2016

Member

Sharing things like the gegl cache is inherently very non-sandboxed though. For instance, you can probably easily write something evil there to cause the other app to crash, and thus exploit it. Also, the other app can easily be running a different gegl version that is not compatible with the one you're using, which can cause all sorts of issues. Basically, the fundamentals of flatpak is to separate things from the host and from other apps to minimize the risks of things breaking each other.

Also, some things are hard to share. For instance, the font config caches you talk about must be separate, because they index different things. If the font cache contains an absolute path like "/app/share/fonts/foo.ttf" or "/usr/share/fonts/foo.ttf" then these will resolve to different things for different applications. Same for anything else that has absolute pathnames in it. Additionally the font caches are tied to the version of fontconfig in use, and the different apps will use different versions of fontconfig, so they will stomp on each other.

So, I will never just make it just use the host XDG dirs, because that will break in several ways. However, being able to specify explicit sharing is more interesting. For instance you could have something like --share-xdg-config=GIMP which would make ~/.var/app/org.gimp.Gimp/config/GIMP be a bind mount of the host side $XDG_APP_DIR/GIMP, thus making $XDG_APP_DIR/GIMP in the sandbox access the same thing as on the host. This could work because the app author can express (and fulfil) the guarantees required for this to work in the current and future versions of the app.

There are some questions about this though:

This breaks the design of flatpak in that you can't easily uninstall an application without having to "untangle" it from the rest of the system. Do we want to make users aware of this application property, and be able to opt out/in of it? I can see this causing confusion in the future on a system where most apps are flatpaks and some are standalone while others are mixed together.

What if the source dir doesn't exist? For instance, if the user never ran a non-flatpaked gimp. Do we create it, or do we use a non-shared config?

What if the user ran a flatpaked gimp before so the non-shared dir exists. Do we override that with the shared host copy?

Member

alexlarsson commented Oct 13, 2016

Sharing things like the gegl cache is inherently very non-sandboxed though. For instance, you can probably easily write something evil there to cause the other app to crash, and thus exploit it. Also, the other app can easily be running a different gegl version that is not compatible with the one you're using, which can cause all sorts of issues. Basically, the fundamentals of flatpak is to separate things from the host and from other apps to minimize the risks of things breaking each other.

Also, some things are hard to share. For instance, the font config caches you talk about must be separate, because they index different things. If the font cache contains an absolute path like "/app/share/fonts/foo.ttf" or "/usr/share/fonts/foo.ttf" then these will resolve to different things for different applications. Same for anything else that has absolute pathnames in it. Additionally the font caches are tied to the version of fontconfig in use, and the different apps will use different versions of fontconfig, so they will stomp on each other.

So, I will never just make it just use the host XDG dirs, because that will break in several ways. However, being able to specify explicit sharing is more interesting. For instance you could have something like --share-xdg-config=GIMP which would make ~/.var/app/org.gimp.Gimp/config/GIMP be a bind mount of the host side $XDG_APP_DIR/GIMP, thus making $XDG_APP_DIR/GIMP in the sandbox access the same thing as on the host. This could work because the app author can express (and fulfil) the guarantees required for this to work in the current and future versions of the app.

There are some questions about this though:

This breaks the design of flatpak in that you can't easily uninstall an application without having to "untangle" it from the rest of the system. Do we want to make users aware of this application property, and be able to opt out/in of it? I can see this causing confusion in the future on a system where most apps are flatpaks and some are standalone while others are mixed together.

What if the source dir doesn't exist? For instance, if the user never ran a non-flatpaked gimp. Do we create it, or do we use a non-shared config?

What if the user ran a flatpaked gimp before so the non-shared dir exists. Do we override that with the shared host copy?

@Jehan

This comment has been minimized.

Show comment
Hide comment
@Jehan

Jehan Oct 18, 2016

Sharing things like the gegl cache is inherently very non-sandboxed though. For instance, you can probably easily write something evil there to cause the other app to crash, and thus exploit it.

Only GEGL is supposed to access this cache. Applications would only access/share this cache through GEGL. Now I also understand what you mean of course: if you give read/write access to the sandboxed GEGL, the application which was packaged together could also access this data.
Maybe that's something to think about: could there be access given only to a specific module, and not to all modules packaged in the same flatpak?

I don't know how such buffer sharing would be actually implemented (Øyvind Kolås would be the one to ask) in a hypothetical future, but sharing data between separate processes is not an unusual thing nowadays. Whatever for simple speed-improving caching, for runtime data sharing or other reason, this should not be forgotten.
When the day comes when we can actually work on a common image buffer on several programs, if flatpak prevents this killer-feature usage, then it will make this packaging system useless IMO.

Also, the other app can easily be running a different gegl version that is not compatible with the one you're using, which can cause all sorts of issues.

I already answered a similar hypothesis (that was about the config files, now it is about communication, but ultimately that's the same issue) in a previous comment: if an app has such a problem, then there is definitely a design bug. And this is not flatpak's role to block the right usage from all the applications which are properly designed, just to circumvent the ones which are not. If a library is planning inter-process communication, it will have to plan for incompatible GEGL speaking together. And maybe in some particular cases, it won't be able to. But then I expect flatpak not to prevent all the cases where it could have worked just because the not-working case may happen.

Also, some things are hard to share. For instance, the font config caches […]

Ok I don't know enough of the internals of fontconfig so pardon me if I say something stupid…

In GIMP, we had some issues with fontconfig cache which can take a long long time to be built when users had a lot of fonts (people using graphics software tend to have/need a lot of fonts). Well this was particularly a problem under Windows because the cache used to be saved in a temporary folder which got wiped out at each reboot. But still it means that the first time flatpak GIMP/Inkscape/other will run, the whole fontconfig cache will have to be rebuilt even if they use only the same system fonts (and no flatpak-embedded fonts), right?
Well it's not as bad as it used to be on Windows obviously (every reboot vs. first run only), yet it feels wasteful. For information, we had reports of people with crazy number of fonts, and GIMP would hang for a very long time at startup. I'm speaking sometimes several dozens of minutes! Fortunately until now this issue was mostly reported on non-Linux systems (Windows and OSX mainly) where fontconfig was not initialized. Now I'm wondering if we will start receiving reports from flatpak users too if GIMP gets widely distributed with it. :-/

Our current workaround is simply to pulse the start UI regularly so that it at least does not appear as having frozen: https://bugzilla.gnome.org/show_bug.cgi?id=449848
Check out the number of duplicates and people following the report: this is a pretty common issue and very bad user experience (even if happening only the first time).

So I understand the /usr/ and /app/ absolute path issue, since you remount these against the SDK and the package inside the sandbox environment. But I wonder if there could some better way which does not end up with the user waiting forever for a cache which already exists on one's system.
Maybe a font portal?

So, I will never just make it just use the host XDG dirs, because that will break in several ways. However, being able to specify explicit sharing is more interesting. For instance you could have something like --share-xdg-config=GIMP which would make ~/.var/app/org.gimp.Gimp/config/GIMP be a bind mount of the host side $XDG_APP_DIR/GIMP, thus making $XDG_APP_DIR/GIMP in the sandbox access the same thing as on the host. This could work because the app author can express (and fulfil) the guarantees required for this to work in the current and future versions of the app.

Looks good.

This breaks the design of flatpak in that you can't easily uninstall an application without having to "untangle" it from the rest of the system. Do we want to make users aware of this application property, and be able to opt out/in of it? I can see this causing confusion in the future on a system where most apps are flatpaks and some are standalone while others are mixed together.

This looks like a possibility to me. On the GUI, I could imagine a small "Advanced" tab when installing a software, where you could opt-out of sharing XDG config, therefore making your flatpak GIMP 100% sandboxed from the rest of the system if you absolutely wanted to. Non-advanced user won't even bother checking the "Advanced" tabs, so confusion would be minimal.

What if the source dir doesn't exist? For instance, if the user never ran a non-flatpaked gimp. Do we create it, or do we use a non-shared config?

Well that's a common use case (there is always a first time you run an app) and normally the application creates the directory. Unless you are saying that flatpak might forbid the application from creating the directory it was given access to (maybe because it doesn't have write access on the parent dir)?

What if the user ran a flatpaked gimp before so the non-shared dir exists. Do we override that with the shared host copy?

I would do nothing. That's a development version and changes are meant to happen. I would not add code exceptions (in GIMP or flatpak) for things changing in a dev version. And if you mean that the user specifically opted out of the shared config (cf. your question 1), then it is an advanced user and we expect him to find the solution (ask on forums or IRC, maybe read a release note or blog posts where we could write these kind of details for people migrating from a dev version, etc.).
Same as the fact we have environment variables allowing a user to set a custom config directory, but then if someone uses this, it is up to this person to handle migration (our config migration code cannot guess every possible case).

Jehan commented Oct 18, 2016

Sharing things like the gegl cache is inherently very non-sandboxed though. For instance, you can probably easily write something evil there to cause the other app to crash, and thus exploit it.

Only GEGL is supposed to access this cache. Applications would only access/share this cache through GEGL. Now I also understand what you mean of course: if you give read/write access to the sandboxed GEGL, the application which was packaged together could also access this data.
Maybe that's something to think about: could there be access given only to a specific module, and not to all modules packaged in the same flatpak?

I don't know how such buffer sharing would be actually implemented (Øyvind Kolås would be the one to ask) in a hypothetical future, but sharing data between separate processes is not an unusual thing nowadays. Whatever for simple speed-improving caching, for runtime data sharing or other reason, this should not be forgotten.
When the day comes when we can actually work on a common image buffer on several programs, if flatpak prevents this killer-feature usage, then it will make this packaging system useless IMO.

Also, the other app can easily be running a different gegl version that is not compatible with the one you're using, which can cause all sorts of issues.

I already answered a similar hypothesis (that was about the config files, now it is about communication, but ultimately that's the same issue) in a previous comment: if an app has such a problem, then there is definitely a design bug. And this is not flatpak's role to block the right usage from all the applications which are properly designed, just to circumvent the ones which are not. If a library is planning inter-process communication, it will have to plan for incompatible GEGL speaking together. And maybe in some particular cases, it won't be able to. But then I expect flatpak not to prevent all the cases where it could have worked just because the not-working case may happen.

Also, some things are hard to share. For instance, the font config caches […]

Ok I don't know enough of the internals of fontconfig so pardon me if I say something stupid…

In GIMP, we had some issues with fontconfig cache which can take a long long time to be built when users had a lot of fonts (people using graphics software tend to have/need a lot of fonts). Well this was particularly a problem under Windows because the cache used to be saved in a temporary folder which got wiped out at each reboot. But still it means that the first time flatpak GIMP/Inkscape/other will run, the whole fontconfig cache will have to be rebuilt even if they use only the same system fonts (and no flatpak-embedded fonts), right?
Well it's not as bad as it used to be on Windows obviously (every reboot vs. first run only), yet it feels wasteful. For information, we had reports of people with crazy number of fonts, and GIMP would hang for a very long time at startup. I'm speaking sometimes several dozens of minutes! Fortunately until now this issue was mostly reported on non-Linux systems (Windows and OSX mainly) where fontconfig was not initialized. Now I'm wondering if we will start receiving reports from flatpak users too if GIMP gets widely distributed with it. :-/

Our current workaround is simply to pulse the start UI regularly so that it at least does not appear as having frozen: https://bugzilla.gnome.org/show_bug.cgi?id=449848
Check out the number of duplicates and people following the report: this is a pretty common issue and very bad user experience (even if happening only the first time).

So I understand the /usr/ and /app/ absolute path issue, since you remount these against the SDK and the package inside the sandbox environment. But I wonder if there could some better way which does not end up with the user waiting forever for a cache which already exists on one's system.
Maybe a font portal?

So, I will never just make it just use the host XDG dirs, because that will break in several ways. However, being able to specify explicit sharing is more interesting. For instance you could have something like --share-xdg-config=GIMP which would make ~/.var/app/org.gimp.Gimp/config/GIMP be a bind mount of the host side $XDG_APP_DIR/GIMP, thus making $XDG_APP_DIR/GIMP in the sandbox access the same thing as on the host. This could work because the app author can express (and fulfil) the guarantees required for this to work in the current and future versions of the app.

Looks good.

This breaks the design of flatpak in that you can't easily uninstall an application without having to "untangle" it from the rest of the system. Do we want to make users aware of this application property, and be able to opt out/in of it? I can see this causing confusion in the future on a system where most apps are flatpaks and some are standalone while others are mixed together.

This looks like a possibility to me. On the GUI, I could imagine a small "Advanced" tab when installing a software, where you could opt-out of sharing XDG config, therefore making your flatpak GIMP 100% sandboxed from the rest of the system if you absolutely wanted to. Non-advanced user won't even bother checking the "Advanced" tabs, so confusion would be minimal.

What if the source dir doesn't exist? For instance, if the user never ran a non-flatpaked gimp. Do we create it, or do we use a non-shared config?

Well that's a common use case (there is always a first time you run an app) and normally the application creates the directory. Unless you are saying that flatpak might forbid the application from creating the directory it was given access to (maybe because it doesn't have write access on the parent dir)?

What if the user ran a flatpaked gimp before so the non-shared dir exists. Do we override that with the shared host copy?

I would do nothing. That's a development version and changes are meant to happen. I would not add code exceptions (in GIMP or flatpak) for things changing in a dev version. And if you mean that the user specifically opted out of the shared config (cf. your question 1), then it is an advanced user and we expect him to find the solution (ask on forums or IRC, maybe read a release note or blog posts where we could write these kind of details for people migrating from a dev version, etc.).
Same as the fact we have environment variables allowing a user to set a custom config directory, but then if someone uses this, it is up to this person to handle migration (our config migration code cannot guess every possible case).

@alexlarsson

This comment has been minimized.

Show comment
Hide comment
@alexlarsson

alexlarsson Nov 15, 2016

Member

Git master of flatpak now has support for e.g. --filesystem=xdg-config/gimp which will give the app (read-write) access to ~/.config/gimp (or wherever you pointed it at on the host). Additionally this will make the gimp subdirectory of ~/.var/app/org.gimp.Gimp/config be a mirror of that directory, so anything respecting XDG_CONFIG_HOME in the sandbox will read/write to the same files as things outside the sandbox.

I still don't recommend doing this, because a large part of the point of flatpak is that we want applications to act as standalone entities in the system which you can easily (and automatically) remote when you want without having to figure out where in the global system it dropped its files.

Member

alexlarsson commented Nov 15, 2016

Git master of flatpak now has support for e.g. --filesystem=xdg-config/gimp which will give the app (read-write) access to ~/.config/gimp (or wherever you pointed it at on the host). Additionally this will make the gimp subdirectory of ~/.var/app/org.gimp.Gimp/config be a mirror of that directory, so anything respecting XDG_CONFIG_HOME in the sandbox will read/write to the same files as things outside the sandbox.

I still don't recommend doing this, because a large part of the point of flatpak is that we want applications to act as standalone entities in the system which you can easily (and automatically) remote when you want without having to figure out where in the global system it dropped its files.

@Jehan

This comment has been minimized.

Show comment
Hide comment
@Jehan

Jehan Nov 24, 2016

Thanks, I will proceed with my tests in the coming month and will report if needed. I hope this will be available in Fedora 25 with this new option soon though.

Jehan commented Nov 24, 2016

Thanks, I will proceed with my tests in the coming month and will report if needed. I hope this will be available in Fedora 25 with this new option soon though.

@alexlarsson

This comment has been minimized.

Show comment
Hide comment
@alexlarsson

alexlarsson Nov 25, 2016

Member

I hope to have a release out soonish.

Member

alexlarsson commented Nov 25, 2016

I hope to have a release out soonish.

@Jehan

This comment has been minimized.

Show comment
Hide comment
@Jehan

Jehan Jan 26, 2017

Hmmm… So I finally tested. Using flatpak 0.8.1 (Fedora 25), and running flatpak build-finish (yes, I used the command lines because I'm just testing right now; obviously we will use the flatpak-builder in the end) with --filesystem=xdg-config/GIMP, it doesn't use my locale XDG config dir, and I can see it still creates ~/.var/app/org.gimp.Gimp/config/GIMP/2.9/.

I see bug #463 has been created by someone else (who used the builder) so maybe discussion on this not working as expected should shift there.

Jehan commented Jan 26, 2017

Hmmm… So I finally tested. Using flatpak 0.8.1 (Fedora 25), and running flatpak build-finish (yes, I used the command lines because I'm just testing right now; obviously we will use the flatpak-builder in the end) with --filesystem=xdg-config/GIMP, it doesn't use my locale XDG config dir, and I can see it still creates ~/.var/app/org.gimp.Gimp/config/GIMP/2.9/.

I see bug #463 has been created by someone else (who used the builder) so maybe discussion on this not working as expected should shift there.

@alexlarsson

This comment has been minimized.

Show comment
Hide comment
@alexlarsson

alexlarsson Jan 27, 2017

Member

Fixed in master, will release a 0.8.2 with this soon

Member

alexlarsson commented Jan 27, 2017

Fixed in master, will release a 0.8.2 with this soon

@meghprkh

This comment has been minimized.

Show comment
Hide comment
@meghprkh

meghprkh Oct 7, 2017

This happens in the firefox dev edition flatpak too, which I got from here. Dont know if this is a bug with flatpak or the flatpak build. But native firefox detects bookmark and uses GTK3 file dialog afaik. (if you need any more info please ask, I am an user so dont know much :) )

meghprkh commented Oct 7, 2017

This happens in the firefox dev edition flatpak too, which I got from here. Dont know if this is a bug with flatpak or the flatpak build. But native firefox detects bookmark and uses GTK3 file dialog afaik. (if you need any more info please ask, I am an user so dont know much :) )

@glaubersm

This comment has been minimized.

Show comment
Hide comment
@glaubersm

glaubersm Nov 17, 2017

I have the same problem with gtk3 file dialogs on Arch Linux, flatpak 0.10 even when I use "flatpak --user override".

I have the same problem with gtk3 file dialogs on Arch Linux, flatpak 0.10 even when I use "flatpak --user override".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment