New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Portable Apps and their settings: Portable Home #4

Open
pcx862 opened this Issue Jul 14, 2016 · 16 comments

Comments

Projects
None yet
4 participants
@pcx862

pcx862 commented Jul 14, 2016

Hi, lead developer of ORB Applications here.

We have developed a "standard" called Portable Home, designed for the portability of data used by Portable Apps.

I was wondering it there are plans to have "Portable AppImages" (Portable Apps)?
Is there interest in adopting something like this "standard"?

Nothing is set in stone regarding this standard, so it can be changed, if necessary.

Feel free to comment on the Github page or here.

@probonopd

This comment has been minimized.

Show comment
Hide comment
@probonopd

probonopd Jul 14, 2016

Member

How about:
If there is a directory with the same name as the AppImage or orb plus some ending (e.g., .home), then use that as the $HOME, $XDG_DATA_HOME, and/or $XDG_DATA_HOME.

This way, we would have one app = one file plus one app's settings = one directory.

Member

probonopd commented Jul 14, 2016

How about:
If there is a directory with the same name as the AppImage or orb plus some ending (e.g., .home), then use that as the $HOME, $XDG_DATA_HOME, and/or $XDG_DATA_HOME.

This way, we would have one app = one file plus one app's settings = one directory.

@pcx862

This comment has been minimized.

Show comment
Hide comment
@pcx862

pcx862 Jul 15, 2016

I think that is an excellent idea. I will add it to the spec and to the scripts (in addition to the existing behaviour)
It will also make sandboxing much easier.

The biggest challenge I foresee is the handling of updates. For example:

  1. firefox_46.elf stores data into firefox_46.elf.home
  2. firefox_46.elf is replaced with firefox_47.elf (either manually by the user, or by some updater)
  3. firefox_47.elf does not see the "old data" folder, as it expects firefox_47.elf.home

pcx862 commented Jul 15, 2016

I think that is an excellent idea. I will add it to the spec and to the scripts (in addition to the existing behaviour)
It will also make sandboxing much easier.

The biggest challenge I foresee is the handling of updates. For example:

  1. firefox_46.elf stores data into firefox_46.elf.home
  2. firefox_46.elf is replaced with firefox_47.elf (either manually by the user, or by some updater)
  3. firefox_47.elf does not see the "old data" folder, as it expects firefox_47.elf.home
@probonopd

This comment has been minimized.

Show comment
Hide comment
@probonopd

probonopd Jul 15, 2016

Member

That is intended - users can have multiple versions (including experimental versions in parallel) which should not mess up each other's settings. It's easy to rename a directory if the user wants to.

Member

probonopd commented Jul 15, 2016

That is intended - users can have multiple versions (including experimental versions in parallel) which should not mess up each other's settings. It's easy to rename a directory if the user wants to.

@pcx862

This comment has been minimized.

Show comment
Hide comment
@pcx862

pcx862 Jul 17, 2016

Good point, I haven't really thought about beta/experimental software.

I've now added it to our code.
The ORB code now also defines the variable $XDG_DATA_HOME

By the way, I think I sort-of fixed the permission problems (files being owned by another user) by automatically running this after the App/payload terminates:
chmod -R a+rwx "$PORTABLE_HOME"

pcx862 commented Jul 17, 2016

Good point, I haven't really thought about beta/experimental software.

I've now added it to our code.
The ORB code now also defines the variable $XDG_DATA_HOME

By the way, I think I sort-of fixed the permission problems (files being owned by another user) by automatically running this after the App/payload terminates:
chmod -R a+rwx "$PORTABLE_HOME"

@probonopd

This comment has been minimized.

Show comment
Hide comment
@probonopd

probonopd Jul 18, 2016

Member

Not sure whether chmod -R a+rwx "$PORTABLE_HOME" is a good idea at all - what about the security implications?

Member

probonopd commented Jul 18, 2016

Not sure whether chmod -R a+rwx "$PORTABLE_HOME" is a good idea at all - what about the security implications?

@pcx862

This comment has been minimized.

Show comment
Hide comment
@pcx862

pcx862 Jul 18, 2016

Based on my testing other users can't read the $PORTABLE_HOME, if the $PORTABLE_HOME is on a directory which itself can't be read by other users (eg. /media/peter/USB/ )
Of course, if it is placed on a directory readable by all, the $PORTABLE_HOME will also be readable by all (but that is expected)
As such, I do not foresee any significant security implications.

But I agree, I think a better/improved solution will be to use some sort of encrypted file, maybe using LUKS or VeraCrypt and automatically prompt the user for the unlock password before running the App

pcx862 commented Jul 18, 2016

Based on my testing other users can't read the $PORTABLE_HOME, if the $PORTABLE_HOME is on a directory which itself can't be read by other users (eg. /media/peter/USB/ )
Of course, if it is placed on a directory readable by all, the $PORTABLE_HOME will also be readable by all (but that is expected)
As such, I do not foresee any significant security implications.

But I agree, I think a better/improved solution will be to use some sort of encrypted file, maybe using LUKS or VeraCrypt and automatically prompt the user for the unlock password before running the App

@probonopd

This comment has been minimized.

Show comment
Hide comment
@probonopd

probonopd Jul 18, 2016

Member

@shoogle what are your thoughts here?

Member

probonopd commented Jul 18, 2016

@shoogle what are your thoughts here?

@shoogle

This comment has been minimized.

Show comment
Hide comment
@shoogle

shoogle Jul 19, 2016

Contributor

@probonopd I like the idea, but I'm not sure if creating a new environment variable is the best solution for two main reasons:

  1. Every program would have to support it.
  2. This variable would have to be manually set by the user every time they move to a new computer.

I think the detection should be done by the packaging so that the application inside doesn't have to be rewritten. I would then overwrite an existing environment variable like $HOME or $XDG_DATA_HOME and I would look for a special portable configuration file to decide whether to do it.

Let's say the USB drive is mounted at /media/user/USB_drive/ and has the following structure:

/media/user/USB_drive/*              # various files (maybe some AppImages here too)
/media/user/USB_drive/Documents
/media/user/USB_drive/Pictures
/media/user/USB_drive/PortableApps   # most AppImages stored here

This structure seems pretty typical and more or less resembles $HOME anyway:

/home/user/*
/home/user/Documents
/home/user/Pictures
/home/user/bin

So it seems to me that sensible places to look for a configuration file would be . and .. relative to the AppImage. The configuration file could something like this:

/media/user/USB_drive/PortableApps/portable_config.sh

#!/bin/bash
export HOME=".."
export XDG_DATA_HOME="$HOME"

This is a bash script that can simply be sourced by AppRun, though a custom syntax might be safer:

/media/user/USB_drive/PortableApps/portable.config

use_custom_home: true
custom_home: ".."

AppRun would then expand this value to a full absolute path and overwrite $HOME and $XDG_DATA_HOME with it so that the application inside the AppImage doesn't have to be rewritten to support $PORTABLE_HOME.

AppRun could even detect that it's path contains /media and offer to create a configuration file.

Contributor

shoogle commented Jul 19, 2016

@probonopd I like the idea, but I'm not sure if creating a new environment variable is the best solution for two main reasons:

  1. Every program would have to support it.
  2. This variable would have to be manually set by the user every time they move to a new computer.

I think the detection should be done by the packaging so that the application inside doesn't have to be rewritten. I would then overwrite an existing environment variable like $HOME or $XDG_DATA_HOME and I would look for a special portable configuration file to decide whether to do it.

Let's say the USB drive is mounted at /media/user/USB_drive/ and has the following structure:

/media/user/USB_drive/*              # various files (maybe some AppImages here too)
/media/user/USB_drive/Documents
/media/user/USB_drive/Pictures
/media/user/USB_drive/PortableApps   # most AppImages stored here

This structure seems pretty typical and more or less resembles $HOME anyway:

/home/user/*
/home/user/Documents
/home/user/Pictures
/home/user/bin

So it seems to me that sensible places to look for a configuration file would be . and .. relative to the AppImage. The configuration file could something like this:

/media/user/USB_drive/PortableApps/portable_config.sh

#!/bin/bash
export HOME=".."
export XDG_DATA_HOME="$HOME"

This is a bash script that can simply be sourced by AppRun, though a custom syntax might be safer:

/media/user/USB_drive/PortableApps/portable.config

use_custom_home: true
custom_home: ".."

AppRun would then expand this value to a full absolute path and overwrite $HOME and $XDG_DATA_HOME with it so that the application inside the AppImage doesn't have to be rewritten to support $PORTABLE_HOME.

AppRun could even detect that it's path contains /media and offer to create a configuration file.

@pcx862

This comment has been minimized.

Show comment
Hide comment
@pcx862

pcx862 Jul 19, 2016

@shoogle Oh, you are right, the best way certainly is to re-define $HOME and $XDG_DATA_HOME
The "$PORTABLE_HOME" variable is internal to the ORB code and not intended as an environment variable,
I've just used "$PORTABLE_HOME" to mean "the folder used as $HOME for the Apps" (sorry if that was not more clear)

I also really like the idea of a configuration file defining the location of the folder to be used as $HOME

I would just like to add a small suggestion:
The use_custom_home: true line can be eliminated: If we have just one line defining the custom_home, isn't it enough both for human users and for the AppRun? Simpler for users and for any software interpreting the config file, right?

pcx862 commented Jul 19, 2016

@shoogle Oh, you are right, the best way certainly is to re-define $HOME and $XDG_DATA_HOME
The "$PORTABLE_HOME" variable is internal to the ORB code and not intended as an environment variable,
I've just used "$PORTABLE_HOME" to mean "the folder used as $HOME for the Apps" (sorry if that was not more clear)

I also really like the idea of a configuration file defining the location of the folder to be used as $HOME

I would just like to add a small suggestion:
The use_custom_home: true line can be eliminated: If we have just one line defining the custom_home, isn't it enough both for human users and for the AppRun? Simpler for users and for any software interpreting the config file, right?

@shoogle

This comment has been minimized.

Show comment
Hide comment
@shoogle

shoogle Jul 19, 2016

Contributor

@pcx862 sorry, I must have missed that. I've read the spec fully now and I see that you look for a directory called "portable_home". I prefer the config file approach for these reasons:

  1. The config file can be hidden
  2. It can be expanded to include other settings (use_custom_home: true was just an example of this)
  3. Users are free to name their directories however they like

Now I have read the full spec it seems to me that Orbs and AppImages are pretty much the same thing, so perhaps you and @probonopd could come to some kind of arrangement to merge the projects and pool resources? Some ways the arrangement could work:

  1. Full project merge. You guys can discuss copyright, naming and commit rights, etc.
  2. Separate projects for now, but Orbs are added to AppImageSpec as valid AppImages and AppImages to Orb spec as valid Orbs. Possible full merge in "Version 2" of the specifications.
  3. Separate projects that both conform to a jointly maintained "Portable Application Specification", but are free to offer additional features on top of that specification.

I'll leave it to you guys to sort out the details, perhaps somewhere more private if you prefer, but if you like then I'm happy to weigh in both as a user and as a developer who maintains a portable package of a popular application. Having two competing but more-or-less identical projects out there isn't great for users or developers. This is open source - surely we can do better than that!

Contributor

shoogle commented Jul 19, 2016

@pcx862 sorry, I must have missed that. I've read the spec fully now and I see that you look for a directory called "portable_home". I prefer the config file approach for these reasons:

  1. The config file can be hidden
  2. It can be expanded to include other settings (use_custom_home: true was just an example of this)
  3. Users are free to name their directories however they like

Now I have read the full spec it seems to me that Orbs and AppImages are pretty much the same thing, so perhaps you and @probonopd could come to some kind of arrangement to merge the projects and pool resources? Some ways the arrangement could work:

  1. Full project merge. You guys can discuss copyright, naming and commit rights, etc.
  2. Separate projects for now, but Orbs are added to AppImageSpec as valid AppImages and AppImages to Orb spec as valid Orbs. Possible full merge in "Version 2" of the specifications.
  3. Separate projects that both conform to a jointly maintained "Portable Application Specification", but are free to offer additional features on top of that specification.

I'll leave it to you guys to sort out the details, perhaps somewhere more private if you prefer, but if you like then I'm happy to weigh in both as a user and as a developer who maintains a portable package of a popular application. Having two competing but more-or-less identical projects out there isn't great for users or developers. This is open source - surely we can do better than that!

@probonopd

This comment has been minimized.

Show comment
Hide comment
@probonopd

probonopd Jul 19, 2016

Member

Well said @shoogle - my preference would be to start with what we have (AppImageSpec) and go from there; discuss and possibly add everything that might be missing as long as it does not violate the stated objectives of AppImage.

There might be cases in which Orb does something that is not mandatory as per the AppImageSpec (which intends to be lean), but ideally it could still be done in a way that would expand upon the AppImageSpec.

Member

probonopd commented Jul 19, 2016

Well said @shoogle - my preference would be to start with what we have (AppImageSpec) and go from there; discuss and possibly add everything that might be missing as long as it does not violate the stated objectives of AppImage.

There might be cases in which Orb does something that is not mandatory as per the AppImageSpec (which intends to be lean), but ideally it could still be done in a way that would expand upon the AppImageSpec.

@shoogle

This comment has been minimized.

Show comment
Hide comment
@shoogle

shoogle Jul 20, 2016

Contributor

@pcx862 what do you think? Are your objectives for Orbs in line with those of AppImages or do you see them diverging in the future?

Contributor

shoogle commented Jul 20, 2016

@pcx862 what do you think? Are your objectives for Orbs in line with those of AppImages or do you see them diverging in the future?

@probonopd probonopd referenced this issue Mar 4, 2017

Closed

Portable $HOME #368

@probonopd

This comment has been minimized.

Show comment
Hide comment
@probonopd

probonopd Jun 18, 2017

Member

AppImage/AppImageKit@d17fdef implements the following:

  • If there is a directory with the same name as the AppImage plus .home, then export $HOME.
  • If there is a directory with the same name as the AppImage plus .config, then export $XDG_CONFIG_HOME.

Depending on how well this works in practice, it may become part of the spec.

Example on how to use this

Imagine you want to use the Leafpad text editor, but carry its settings around with the executable.

You can do the following:

# Download Leafpad AppImage and make it executable
wget -c "https://bintray.com/probono/AppImages/download_file?file_path=Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage" -O Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage
chmod a+x Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage

# Create a directory with the same name as the AppImage plus the ".config" extension
# in the same directory as the AppImage
mkdir Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage.config

# Run Leafpad, change some setting (e.g., change the default font size)
# then close Leafpad
./Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage

# Now, check where the settings were written:
linux@linux:~> find Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage.config
(...)
Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage.config/leafpad/leafpadrc

Notice that Leafpad has written its configuration file leafpadrc into the directory we had created.

If it does not work with your AppImage, make sure it was generated with the latest appimagetool continuous build as of today. If your application does not honor $XDG_CONFIG_HOME (it should), then please file a bug with the application developer and use a directory with the same name as the AppImage plus the .home extension in the meantime.

Member

probonopd commented Jun 18, 2017

AppImage/AppImageKit@d17fdef implements the following:

  • If there is a directory with the same name as the AppImage plus .home, then export $HOME.
  • If there is a directory with the same name as the AppImage plus .config, then export $XDG_CONFIG_HOME.

Depending on how well this works in practice, it may become part of the spec.

Example on how to use this

Imagine you want to use the Leafpad text editor, but carry its settings around with the executable.

You can do the following:

# Download Leafpad AppImage and make it executable
wget -c "https://bintray.com/probono/AppImages/download_file?file_path=Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage" -O Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage
chmod a+x Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage

# Create a directory with the same name as the AppImage plus the ".config" extension
# in the same directory as the AppImage
mkdir Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage.config

# Run Leafpad, change some setting (e.g., change the default font size)
# then close Leafpad
./Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage

# Now, check where the settings were written:
linux@linux:~> find Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage.config
(...)
Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage.config/leafpad/leafpadrc

Notice that Leafpad has written its configuration file leafpadrc into the directory we had created.

If it does not work with your AppImage, make sure it was generated with the latest appimagetool continuous build as of today. If your application does not honor $XDG_CONFIG_HOME (it should), then please file a bug with the application developer and use a directory with the same name as the AppImage plus the .home extension in the meantime.

@JonasT

This comment has been minimized.

Show comment
Hide comment
@JonasT

JonasT Aug 8, 2017

@probonopd suggestion: it should be possible to force a setting when creating the AppImage to use such a local home even if the user hasn't already created it (with the AppImage attempting to create it on launch with a reasonable error if the permissions don't allow it). This would allow safely packaging applications in a portable way that guarantees to not interfere/break with system-wide installs of the same application, and to keep settings portability without relying on the user thinking of creating those folders.

JonasT commented Aug 8, 2017

@probonopd suggestion: it should be possible to force a setting when creating the AppImage to use such a local home even if the user hasn't already created it (with the AppImage attempting to create it on launch with a reasonable error if the permissions don't allow it). This would allow safely packaging applications in a portable way that guarantees to not interfere/break with system-wide installs of the same application, and to keep settings portability without relying on the user thinking of creating those folders.

@probonopd

This comment has been minimized.

Show comment
Hide comment
@probonopd

probonopd Aug 8, 2017

Member

That's easy to do already: Put in a start script (e.g., AppRun) that does mkdir -p "$APPIMAGE.data" before executing the payload application. But you need to check whether the location is writeable and fall back to something else if it is not.

Member

probonopd commented Aug 8, 2017

That's easy to do already: Put in a start script (e.g., AppRun) that does mkdir -p "$APPIMAGE.data" before executing the payload application. But you need to check whether the location is writeable and fall back to something else if it is not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment