Feature: Autoupdate

[overheadhunter]

If possible in a multi-platform manner, it would be nice, if the application can update itself. However there might be security considerations. To be discussed.

[jellemdekker]

I'm interested in working on this feature. What kind of implementation is preferred? I propose the following options:

1. Instead of opening https://cryptomator.org/, the following code is expanded to open the appropriate download URL for the current operating system's latest version. Example for macOS: opening https://bintray.com/cryptomator/cryptomator/download_file?file_path=Cryptomator-1.3.2.dmg.
2. Instead of opening browser windows, Cryptomator downloads the appropriate installation file and presents it to the user by showing the folder where it was downloaded to (opening a window in Windows Explorer, Finder, Linux file browsers, or simply linking to the file's location in a dialog box).
3. Expanding on option 2: open the downloaded installation file and close Cryptomator, so the user can proceed with installing the latest version (might be more complex because of platform dependent code for starting an installation, also needs to account for portable installations).

Please let me know what your thoughts are on these options and feel free to make suggestions for different approaches.

[overheadhunter]

@jellemdekker Thanks for your interest in helping us here!

To be honest, I no longer prefer a multi-platform solution. Instead I want to get rid of the update check within the JavaFX application and would propose the following:

• Sparkle for Cryptomator for macOS
• AppImageUpdate for Cryptomator for Linux
• Something else (maybe WinSparkle?) for Cryptomator for Windows

[albertopasqualetto]

I think the biggest problem for now is that when I download and open the installer it doesn't just update or uninstall the previous version and then install the updated version, but first of all it asks to uninstall Cryptomator manually...

[infeo]

This is due to a change in technology (we migrated from exe installer to msi installer) and is not related to a possible auto update.

[albertopasqualetto]

I understand👍🏻

[purejava]

Hi,

I thought I'd play around with AppImageUpdate and found that everything required in this repo to enable AppImageUpdate already seems to be in place:

cryptomator/.github/workflows/appimage.yml

Lines 137 to 139 in 46745d0

	./squashfs-root/AppRun Cryptomator.AppDir cryptomator-${{ steps.versions.outputs.semVerStr }}-x86_64.AppImage
	-u 'gh-releases-zsync|cryptomator|cryptomator|latest|cryptomator-*-x86_64.AppImage.zsync'
	--sign --sign-key=615D449FE6E6A235 --sign-args="--batch --pinentry-mode loopback"

When I update a downloaded cryptomator-1.6.12-x86_64.AppImage with AppImageUpdate, it states, that it did the update, but the AppImage is unchanged after the update. When I start it, it misses the features of 1.6.14.

ralph@fusion Downloads % ./AppImageUpdate-x86_64.AppImage cryptomator-1.6.12-x86_64.AppImage
AppImageUpdate version 1-alpha (commit 20a6450), build 93 built on 2022-01-24 18:35:10 UTC
Fetching latest release information from GitHub API
Updating from GitHub Releases via ZSync
Fetching latest release information from GitHub API
zsync2: Target file: /home/ralph/Downloads/cryptomator-1.6.14-x86_64.AppImage
zsync2: Reading seed file: /home/ralph/Downloads/cryptomator-1.6.12-x86_64.AppImage
zsync2: Usable data from seed files: 49,375158%
zsync2: Renaming temp file
zsync2: Fetching remaining blocks
zsync2: Downloading from https://objects.githubusercontent.com/github-production-release-asset-2e65be/16446099/366e220d-6e46-4376-a9bb-1d7781e93d4e?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220906%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220906T161459Z&X-Amz-Expires=300&X-Amz-Signature=596a4ba8fb5663cf4bdead51bec6c45b8d98abe4da3d27911d9ffe7faa396c5f&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=16446099&response-content-disposition=attachment%3B%20filename%3Dcryptomator-1.6.14-x86_64.AppImage&response-content-type=application%2Foctet-stream
zsync2: optimized ranges, old requests count 78, new requests count 7

zsync2: Verifying downloaded file
zsync2: checksum matches OK
zsync2: used 27916288 local, fetched 30223552
gpg: Schlüssel 615D449FE6E6A235: 2 Beglaubigungen wegen fehlender Schlüssel nicht geprüft
gpg: Schlüssel 615D449FE6E6A235: Öffentlicher Schlüssel "Cryptobot <releases@cryptomator.org>" importiert
gpg: Anzahl insgesamt bearbeiteter Schlüssel: 1
gpg:                              importiert: 1
gpg: keine ultimativ vertrauenswürdigen Schlüssel gefunden
gpg: Schlüssel 615D449FE6E6A235: 2 Beglaubigungen wegen fehlender Schlüssel nicht geprüft
gpg: Schlüssel 615D449FE6E6A235: "Cryptobot <releases@cryptomator.org>" nicht geändert
gpg: Anzahl insgesamt bearbeiteter Schlüssel: 1
gpg:                             unverändert: 1
Signature validation passed
ralph@fusion Downloads %

I am a little lost now. @infeo, @overheadhunter: do your have more experience with this, e.g. if this should work already or what I am missing here?

[overheadhunter]

AppImage is unchanged after the update

Ummm not sure how this is supposed to work... The log suggests that it does not update the original file, but rather creates a new one:

zsync2: Target file: /home/ralph/Downloads/cryptomator-1.6.14-x86_64.AppImage
zsync2: Reading seed file: /home/ralph/Downloads/cryptomator-1.6.12-x86_64.AppImage

which is admittedly a little pointless... I guess we have to look for answers in AppImage/AppImageUpdate

[purejava]

Ummm not sure how this is supposed to work... The log suggests that it does not update the original file, but rather creates a new one:

You are right.
I looked into the C++ source code. The moment it executes a zsync2::ZSyncClient was when I decided to ask the devs on their IRC channel, instead of finding out what it does myself. 😀

This works as designed. It creates the latest AppImage and pulls only that portion of data over the wire, that is not already contained in the referenced file / file that is forseen for the update.

So you end up with two AppImages: the old one and the latest released one.

From reading the AppImage docs, I was under the impression that "updating the Applmage" means, that the origin file itself is being updated, but that’s wrong.

which is admittedly a little pointless...

I agree. It’s less effort to download the latest AppImage right away.

[purejava]

I bet that means that we can rule out AppImageUpdate.

[overheadhunter]

I bet that means that we can rule out AppImageUpdate.

Maybe not, when combined with AppImageLauncher, which integrates the update process into the context menu of the application launcher. Did you give this a try?

[purejava]

No, I wasn't aware of AppImageLauncher, thanks for the hint.

Indeed, it is quite neat. You have to install AppImageLauncher. It creates an Applications directory and when you move an AppImage, that contains update information, into that directory (like Cryptomator does), you can manage the AppImage / Cryptomator seamlessly with your UI.

You see here updating a non latest AppImage and deleting the updated AppImage afterwards:

EDIT: changed screenshots to English

[purejava]

• Something else (maybe WinSparkle?) for Cryptomator for Windows

I created Java bindings for WinSparkle with jextract and added WinSparkle to a fork of our project for testing purposes.
The update mechanism works. You can give it a try with this MSI installer. The code is identical to Cryptomator 1.6.15, but initializes WinSparkle. Just start Cryptomator two times after each other to trigger WinSparkle.

The Java bindings do work fine. Unfortunately it's not possible right now to use the callbacks provided by the WinSparkle.dll. The JVM crashes on using them.

[purejava]

The Java bindings for WinSparkle have been released in two flavors: with and without the WinSparkle.dll included.

Would you consider a PR adding the WinSparkle functionality to this project?
If this question would be answered with yes, some things need to be discussed, namely:

• should the WinSparkle functionality be added as is on top or replace the "Updates" tab in the prefs?
• what's a good place to publish the appcast.xml? (WinSparkle loads this on start-up, so this is one more call to the internet Cryptomator makes)

[infeo]

If possible in a multi-platform manner

As far as i know, WinSparkle is Windows only. In general, system-specific functionality applicable to all systems should go to the system specific integrations-library and Cryptomator just make calls over https://github.com/cryptomator/integrations-api. If you already have a prototype, my suggested next step is to design a general API for auto updating, such that an autoupdate can be implemented against this API for every system.

Also, i'm asking myself: What is the benefit of using an external library on Windows?
The update mechanism in Windows is quite simple: Download msi or exe, quit Cryptomator and execute it. Theoretically, we could whip up a screen to to show update info, the model makes the regarding https calls and on update, exits the app and starts the msi or exe. With the benefits that we

1. do not have to deal with native code;
2. can choose the design and
3. do not depend on a third party.

@tobihagemann What do you think about WinSparkle/Sparkle ?

[overheadhunter]

I would love to see this (also on Mac). And yes, I would in this case fully remove the update checker via a system property that can be un/set depending on the package type (sometime updates are managed by third party tools anyway, eg when using a package manager).

Updating via (win)sparkle is significantly more comfortable than needing to download sth manually.

However we should check if this allows us to also keep third party tools up to date (WinFSP).

[purejava]

If possible in a multi-platform manner

As far as i know, WinSparkle is Windows only. In general, system-specific functionality applicable to all systems should go to the system specific integrations-library and Cryptomator just make calls over https://github.com/cryptomator/integrations-api. If you already have a prototype, my suggested next step is to design a general API for auto updating, such that an autoupdate can be implemented against this API for every system.

Yes, WinSparkle is Windows only. I agree, that this should be generalized with / over the integrations-api as far as possible.

Also, i'm asking myself: What is the benefit of using an external library on Windows? The update mechanism in Windows is quite simple: Download msi or exe, quit Cryptomator and execute it. Theoretically, we could whip up a screen to to show update info, the model makes the regarding https calls and on update, exits the app and starts the msi or exe. With the benefits that we

1. do not have to deal with native code;
2. can choose the design and
3. do not depend on a third party.

@tobihagemann What do you think about WinSparkle/Sparkle ?

I disagree on this one. Using WinSparkle is basically just a few lines of code to integrate it and WinSparkle does have it all: prompt the user for updates, show a changelog, postpone updates, ... Sparkle for Mac should be similar. So why re-inventing all this by ourselves?

[purejava]

However we should check if this allows us to also keep third party tools up to date (WinFSP).

AppImageUpdate, WinSparkle and Sparke all work the same. You provide an update information as a link to the latest package, like this for WinSparkle:

<enclosure sparkle:os="windows" url="https://github.com/purejava/cryptomator/releases/download/1.6.17/Cryptomator-1.6.17-x64.msi" sparkle:version="1.6.17" sparkle:dsaSignature="MEYCIQCtoQSh62IOJFeFeu1w30bZhuhsEAmoBMFQDrot/E04NAIhAKE9ukAlwHq7 sCRFZAaGY7RnxjuM7e4M28/04zuvJGyY" type="application/octet-stream"/>

I haven't checked it, but it should allow to reference the exe installer including WinFSP instead of the msi package.

[overheadhunter]

I haven't checked it, but it should allow to reference the exe installer including WinFSP instead of the msi package.

The best sparkle ux would be a "silent" installation though, progress only showing in the sparkle ui.

[infeo]

The best sparkle ux would be a "silent" installation though

It won't be in the same UI, but a quiet installation with the MSI or EXE is possible and supported by winsparkle: https://github.com/vslavik/winsparkle/wiki/Appcast-Feeds#installer-arguments

this allows us to also keep third party tools up to date (WinFSP).

Do you mean seperatly to Cryptomator? Might be possible, but i think this will make the whole process quite complicated. I would keep the scope small. If a user decides to use the MSI, the user has to update WinFSP.

[purejava]

I reconsidered all arguments and googled a bit.

Maybe adopting and integrating this would be a good solution for Windows and Mac:

• no additional library dependencies needed
• a sole code base for all platforms

And if we would bundle AppImageUpdate itself into our AppImage, even the Linux package might be updatable as well with the solution mentioned above.

[tobihagemann]

I'm more hesitant with UpdateFX. It was recently archived and hasn't been updated for many years now. From a UX perspective, I'm still a huge fan of Sparkle on macOS. It's still the de facto standard on macOS. Not sure about Windows but I guess WinSparkle is basically the same.

I don't know if this can be solved via the integrations-api. But having Sparkle/WinSparkle integrated would be a huge benefit for everyone. I'd love to see a PR on this. We would take care of hosting appcast.xml (that's what we already do with latestVersion.json).

[purejava]

Ok, thanks for the discussion. Let's go this way!

I found out how to bundle AppImageUpdate into our AppImage. It's as simple as copying it into the AppDir under e.g. usr/bin/ from where it can be triggered.

As the way AppImageUpdate works is that it checks, if a newer AppImage is available and downloads that right away next to the running AppImage, this lacks all the convenient features WinSparkle and Sparkle do provide, like displaying a change log, postponing an update, enabling or disabling updates at all and probably more.

So, the further route is as follows:

• add an AutoUpdateProvider to the integrations-api that has initAutoUpdate(), cleanUpAutoUpdate(), enableAutoUpdate(), disableAutoUpdate() ...
• remove the update checker via a system property that can be un/set depending on the package type
• decide for AppImages, if we should add some features that are available with the more powerful third party libs like enabling or disabling updates at all, displaying a change log, ending the running AppImage after AppImageUpdate has done its work and starting the newer AppImage instead ...

I haven't checked Sparkle so far. I think the minimum effort here is to create Java bindings for the Obj-C APIs and to find out about the programmatic setup of Sparkle, which looks a little more complicated than WinSparkle to me in the sense of not providing documentation on how this needs to be done.

I'd go ahead and do this. Suggestions, stop signs and feature requests are welcome!

[purejava]

The Java bindings do work fine. Unfortunately it's not possible right now to use the callbacks provided by the WinSparkle.dll. The JVM crashes on using them.

I opened a bug report upstream and Oracle looked into the report, found the root cause for the JVM crash and suggests a solution that works.

As soon as I'll release a new version of the Java bindings, it'll be possible to use the callbacks provided by the WinSparkle.dll. 😃

[purejava]

I don't know if this can be solved via the integrations-api. But having Sparkle/WinSparkle integrated would be a huge benefit for everyone. I'd love to see a PR on this. We would take care of hosting appcast.xml (that's what we already do with latestVersion.json).

The first part - a PR implementing WinSparkle - is ready for comment.

[purejava]

Let me give you a short status update on my progress here:

I did an not notable code improvement for the WinSparkle implementation and started to implement the AppImage auto update functionality. These changes can be found here, but are not merged into this PR yet, as there is a problem that I cannot get grip on right now.

There is a new class AppImageHandler that prompts the user to enable or disable auto update only for AppImages. This relies on a new setting "autoUpdateType".

AppImageUpdate got bundled inside the AppImage itself and gets triggered on application start when auto update is wanted.

This downloads the latest Cryptmator AppImage next to the running one. So far, so good, but it refuses to start the newly downloaded AppImage from within the running one. I know, that there is the IPC check for other running instances, but it not even executes the ProcessBuilder to run the AppImage. Right now, I do not know, what's missing.

[purejava]

I'd like to add some more notes on the Windows part.
After this comment it seems unlikely to me, that we'll add WinSparkle to this project, unless WinSparkle supports EdDSA signatures. Right now, there seems to be no progress with this.

I googled for alternatives to WinSparkle and App Installer from Microsoft seems quite promising for our needs.

A simple XML file, similar to the following, allows to distribute app updates by publishing an updated version of the XML file.

<?xml version="1.0" encoding="utf-8"?>
<AppInstaller
    xmlns="http://schemas.microsoft.com/appx/appinstaller/2017/2"
    Version="1.0.0.0"
    Uri="https://winsparkle-java.s3.eu-central-1.amazonaws.com/cryptomator.appinstaller" >

    <MainPackage
        Name="Cryptomator"
        Publisher="CN=Skymatic GmbH"
        Version="1.1.6.16"
        Uri="https://github.com/cryptomator/cryptomator/releases/download/1.6.16/Cryptomator-1.6.16-x64.msi"
        ProcessorArchitecture="x64" />

</AppInstaller>

This would require Cryptomator to be packaged as an msix app, which currently is neither available with jpackage nor as a GitHub workflow.

At least there are tools available to re-package a msi packaged to an msix app. Two I found are the Advanced Installer and Conveyor.
The MSIX Packaging Tool did not work for me, as it requires a VS project as an origin for re-packaging.

[purejava]

Regarding Linux:

This downloads the latest Cryptmator AppImage next to the running one. So far, so good, but it refuses to start the newly downloaded AppImage from within the running one. I know, that there is the IPC check for other running instances, but it not even executes the ProcessBuilder to run the AppImage. Right now, I do not know, what's missing.

I experimented a litte more on this one. The IPC check is not causing trouble. The code works when started from the IDE, but not when started as / from within an AppImage. The self-contained JRE makes the difference, I guess. To get this working, we'd need to find a way to launch the new AppImage, while the old one was shut down already.