Documentation

Mordechai Meisels edited this page Nov 16, 2018 · 56 revisions

Overview

Goal

The update4j project makes it very easy to push new versions of your software to all installed machines. All you need is a special file called the "configuration", which the framework checks against for possible updates. This file should be available on the web for download, and load its content into the framework.

Features

Whenever you wish to release an update, all you need is publish a new version of the configuration and the job will be done by itself. It will pull all dependencies from the locations declared in the configuration and download them into the file location declared in the configuration.

As an extra layer of security, the framework has the ability to sign dependency files and put the signature into the configuration. Upon download, it will check against your public key if the file is correctly signed and reject the download otherwise.

Signing is very simple, when creating a new configuration in code (using the Configuration class) you can provide your private key; it'll do the rest itself.


Note: Although signing is completely optional, we strongly recommend to do it, should your server ever be compromised.


When your download is complete (or even when you didn't do any updates at all) you can dynamically load the files listed in the configuration onto the Java Module System modulepath or the good old-fasioned classpath, (each file in the config individually expresses how to be loaded) and will be readily be available to the rest of the application.

As a side feature you can run your application in single-instance mode. Think of your favorite text editor; double clicking a file when the editor is already running will just hand over that file to running instance. Using SingleInstanceManager operates exactly the same; if invoked on the first instance it will set up listeners. When you try to start it again these listeners will be notified and its command-line arguments well be passed over.

Lifecycle

Before you start to wrap around how everything comes together, first start out with the default settings that doesn't require you to understand its internals. To use it, just start the framework as a jar file and follow the output instructions.

Read on if you want to leverage the flexibility of this framework. Otherwise, skip all the way to the example.

Creating a Bootstrap

While grasping how everything comes together might be complex at first, it is its powerful flexibility that really counts. Once you understand the lifecycle, it should be pretty straightforward.

Your application will consist of 2 parts, the bootstrap application and the business application. The bootstrap is where you generally do the update/launch logic. The business application is the regular application you want to distribute.

There are 2 ways to start the bootstrap:

  • Standard Mode: Run your own main method and use update4j as a regular dependency.

  • Delegate Mode: Run update4j's main method from org.update4j.Bootstrap and it will forward the process to a registered delegate. Continue use this framework as a regular dependency as in standard mode. This mechanism allows you to easily update your bootstrap by simply adding a newer delegate to the classpath or modulepath.

    You should generally not use delegate mode in the beginning, as it adds an extra layer you complexity to the development. Instead, migrate once everything runs smoothly on standard mode.

Once your bootstrap application is running (in either mode) you will normally do some setups of your choice and then load a configuration from a location of your choice. Once the configuration is loaded there are 3 paths to continue:

  • Update: Using the configuration, the framework will locate your local files and check if they are synchronized by comparing the checksum of the file with the one declared in the config. In the event the system detects that updates should be performed it will download only the files that requires an update. You can also provide a public key and all downloads will be verified against the signature declared in the config.

    You can optionally monitor the download progress by providing an UpdateHandler. By default all progress will be reported in the console.

  • Launch: Using your already running bootstrap application, dynamically load your business application onto the running VM by extending the classpath, or using Java 9's ModuleLayer for modular jars -- depending in the configuration. This is effectively the handover from the bootstrap to your regular application.

    When handing over the process it will call the run() method of the highest version registered Launcher, or an instance passed in the launch() method. By default it will use the DefaultLauncher that starts a main(String[]) according to a known main class.

  • Update to Temp Directory: While some applications may want to always run the newest version and may check for updates before the launch, some applications might make it completely user-feedback based. You may first launch your app before any update check, silently check if updates are available and tell the user that updates are available. The user may opt to update it now, but be aware that you cannot replace your old files as the JVM locks them upon launch. You might of-course create new files, but you will have to control the deletions of the old files yourself.

    The framework gives a nice alternative: Namely, download all new files to a predefined directory, restart the whole application and finalize the update in the bootstrap upon restart by calling Update::finalizeUpdate, then launch the newest version.

    Be aware, files that are required in the bootstrap will only be available after another restart after finalization. Otherwise, for business depenedencies, it should be all ready once finalized.

Example

Example Configuration

Here in an example of a perfectly valid configuration with dummy URLs and locations; read the inline comments. You might also check out the Javadoc on how to create configurations.

Example Configuration Usage

Loading a config into code is very easy:

Configuration config = null;
try(InputStreamReader in = new InputStreamReader(myUrl.openStream()) {
    config = Configuration.read(in);
}

On the other side, writing a config to a file is just the same:

try(Writer out = Files.newBufferedWriter(Paths.get("config.xml"))) {
    config.write(out);
}

Using config methods:

/*
 * Updates all outdated files
 */
config.update();


/*
 * Updates all outdated files and verifies against the given public key
 */
config.update(myPubKey);

/*
 * Updates all outdated files and saves them in a temporary location.
 * To finalize the update, call Update.finalizeUpdate()
 */
config.updateTemp(tempDir);

/*
 * Launches the business application
 */
config.launch();

/*
 * Launches the business application and pass the list as arguments
 */
config.launch(List.of("a","b","c"));


/*
 * The timestamp when this configuration was created
 */
config.getTimestamp();


/*
 * Get a list of all files listed in the config
 */
config.getFiles();


/*
 * Return the resolved value of the property, after replacing all placeholders
 */
config.getResolvedProperty("prop.name");

Creating new Configurations can be using one of the builder methods using fluent chaining:

using Java 10's var

PrivateKey key = ... // Load a private key        
  
var cb = Configuration.builder()

             // base URI from where to download, overridable in
             // each individual file setting 
             .baseUri("https://example.com/")  

             // base path where to save on client machine, overridable in
             // each individual file setting              
             .basePath("/home/myapp/")
             .signer(key) // used to sign the files

             // List this property
             .property("app.name", "MyApplication")

             // Automatically resolves system property
             .property("user.location", "${user.home}/myapp/");

             // List this file, uri and path are same as filename
             // Read metadata from real file on dev machine
             // Will be dynamically loaded on the modulepath
             .file(FileMetadata.readFrom("myFile.jar")
                               .modulepath()) 

             // Path will be same as uri
             // Will be dynamically loaded on the classpath
             .file(FileMetadata.readFrom("myFile2.jar")
                               .uri("file.zip")
                               .classpath())

             // Will not be loaded on any path, ideal for bootstrap dependencies
             // or non jar files
             // that are loaded on JVM startup
             .file(FileMetatdata.readFrom("my-resource.png")
                                .path("${user.location}/my-resource.png")
                                .uri("https://example.com/pix/my-picture.png"));

             // adds the whole directory and marks all jar files with 'classpath'
             // getSource() returns the path of the real file.
             .files(FileMetadata.streamDirectory("target")
                                .peek(r -> r.classpath(r.getSource().toString().endsWith(".jar"))));
   

// Once all settings are set, let's build it
Configuration config = cb.build();

Dealing with Providers

In order to make your project as updatable as possible we leveraged Java's ServiceLoader mechanism which has been greatly improved in Java 9. Once you want to leave the defaults universe, there are 3 services you can provide. All providers are part of the org.update4j.service package.

  1. Delegate — Easily lets you replace your bootstrap application when starting the framework in Delegate Mode. When you want to push a new version of the bootstrap, simply list it in the configuration. When the new version is located on the next bootstrap restart, the new version will be selected.

  2. Update Handler — Monitors the update life-cycle with callbacks for every state change.

  3. Launcher — The hand-over to the business application, like the main method.

Advertising Providers

In order to make your provider discoverable, they must be "advertised".

Providers Loaded in the Modulepath

Advertising is done by adding the provides directive in its module-info file:

module hello.world {
    // for delegates
    provides org.update4j.service.Delegate with hello.world.MyDelegate;

    // same with other providers
    provides org.update4j.service.UpdateHandler with hello.world.MyUpdateHandler;
}

Providers Loaded in the Classpath

Advertise the provider by adding a special "provider-configuration" file, located in META-INF/services/ resource directory in the jar.

The file should be named with the fully-qualified name of the service class, with no special extension. The contents of the file should be the fully-qualified name of the provider, each provider on a seperate line.

For instance:

META-INF/services/org.update4j.service.UpdateHandler should contain

com.example.MyUpdateHandler

Please refer to ServiceLoader for more details.

Provider Versioning

All services have a method version(). When there are more than one of a specific provider in the classpath or modulepath, the one with the highest version will be loaded. To override the version, the delegate should be loaded as follows:

$ java --module-path . --module org.update4j --delegate=com.example.MyDelegate

And for update handler and launcher you should list them in the configuration in the <provider> element. If you attempt to override, but the class could not be loaded, it falls back to the default behavior using the version. It will output a warning in the error stream:

a.b.MyLauncher not found between providers for org.update4j.service.Launcher, using a.b.MyOtherLauncher instead.

All class names should be written in Canonical Class Name, i.e. the string return when calling clazz.getCanonicalName().

There is no documented behaviour if 2 providers have the same version number, which of them will be loaded.

Provider Discovery Timing

Since the modulepath is static and the classpath not, they behave completely differently.

Providers in the classpath are discovered whenever the service is needed. If the provider jar file was added to a running instance it will be discovered when next looked for.

Providers in the modulepath are loaded when the application starts or when a new module layer is created:

  • Delegate & UpdateHandler — Are loaded in the bootstrap, so only providers that were available at bootstrap startup are seen. A restart is required to make newer visible.
  • Launcher — Is (read: should generally be) loaded in the new layer, therefore if a new version is available in the business application even if it wasn't there when the bootstrap started; they will be discovered.

Starting the Application

If you use your own bootstrap in standard mode, you can start the application in any way you wish, there is no special requirements. For the default bootstrap or delegate mode, you should start the framework with its own main method: org.update4j.Bootstrap.

It is strongly encouraged to reduce the visibility of the boot (i.e. JVM native) classpath and modulepath so it doesn't statically load business application dependencies that should instead be dynamically loaded by the framework itself. If you are using default bootstrap, always start the application by explicitly listing the framework's jar only in the boot classpath or modulepath -- don't use the * or . wildcards. If you define your own bootstrap, keep the boot libraries in a seperate directory from the rest of the (business) application and only point the classpath or modulepath to that directory.

To start the framework in the modulepath:

$ java -p update4j-<version>.jar -m org.update4j
$ java -p boot/ -m org.update4j

To start the framework in the classpath:

$ java -jar update4j-<version>.jar
$ java -cp boot/* org.update4j.Bootstrap

Or mix them both (assume classpath and modulepath are directories):

$ java -cp classpath/* -p modulepath/ -m org.update4j
$ java -p modulepath/ -jar update4j.jar

You can also start them in delegate mode by calling any method from the org.update4j.Bootstrap class.

Important: Files that are loaded dynamically, i.e. is marked either classpath="true" or modulepath="true" must not be loaded on the JVM boot classpath or modulepath. Errors such as locked files and NoClassDefFoundError will happen otherwise.

Properties

You can list properties in the config similar to how you would in a Maven POM file. You can then refer to them using the placeholder ${prop.key}.

Properties can be injected into all URIs, paths, handler class names, file comments, and into other properties' value itself. In the latter, it is valid unless it creates a cycle (prop1 referring to prop2 that refers to prop1).

If a property is not listed between the rest properties but still used with the placeholder, it will try to load the system property with that name. If it doesn't exist, it will try to load the system environment variable with that name. If this fails too, an Exception will be thrown.

Properties might be OS specific, where a certain property has a certain value depending on the underlying system. OS specific properties will have precedence over properties with the same name but without OS specificity. You may not list 2 properties with the same name and same OS (or no OS).

Files

All required files are listed as a files, with the corresponding object FileMetadata. Valid attributes are:

Attribute Meaning
uri The remote location of that file.
path Where the file should be downloaded on the client's machine.
checksum The Adler32 checksum of the file. Used to check if an existing file is outdated.
size The file size.
classpath Whether the file should be loaded on the business application's class path.
modulepath Whether the file should be loaded on the business application's module path. Non jar files must not declare this true
comment An optional string you may use to signal anything to an observer. E.g. it may say "restart" to signal that if this file was downloaded now, it requires a restart.
ignoreBootConflict Every jar file after download gets checked if its module name or package names already exists in the boot layer. In order to prevent accidental breakage among all clients it will reject the download (as once it's there you can't even start the JVM and no way to remotely fix this). If the new file was carefully placed so it won't get loaded onto the boot modulepath (i.e. only in the dynamic modulepath if marked with modulepath="true") you may override it with this flag.
It might also be useful if a file has the .jar file extension and is not a real archive.
signature A Base64 string of the signature. Generated when a private key is supplied to the framework.

Additionally there are 3 child elements. All of them are ignored if the file is not loaded in the modulepath (modulepath above is not true)

Element Meaning
addExports Export packages to another module in the new module layer, even if it doesn't export them in the module descriptor. Equivalent to --add-exports on the boot layer.
addOpens Open packages to another module in the new module layer, even if it doesn't open them in the module descriptor. Equivalent to --add-opens on the boot layer.
addReads Add reading another module new module layer, even if it doesn't require them in the module descriptor. Equivalent to --add-reads on the boot layer.

Classpath and Modulepath

Using classpath or modulepath or even mixing them both together is fully supported. You can specify whether a file should be loaded in the classpath or in the modulepath by setting the classpath and modulepath attributes respectively. If both attributes are set to false, or -- for this purpose -- not set at all, it will not be (dynamically) loaded when the business application is started. This is what you should usually do for non jar files (e.g. resources), or bootstrap files (that are loaded statically by the boot classpath or modulepath).

Important: Files that are loaded dynamically, i.e. is marked either classpath="true" or modulepath="true" must not be loaded on the JVM boot classpath or modulepath. Errors such as locked files and NoClassDefFoundError will happen otherwise.

Files that are run in the bootstrap application, such as delegates, update handlers and its dependencies must be loaded by the JVM boot classpath or modulepath. Since these files get locked upon running the application these files are not normally updatable. Services (as Delegate and UpdateHandler) are updated by simply putting a newer version (with higher version number) in the boot classpath or modulepath.

Launchers should generally be loaded dynamically, and should not be in the boot classpath, otherwise it will not have full access to dynamically loaded classes unless you reflect with the context class-loader.

Access Classpath in Modulepath

All automatic modules have out-of-the-box access to files in the classpath. Explicit modules can't express a dependency on the unnamed module, use an automatic module as a bridge between the two. It does have reflective access.

Access Modulepath in Classpath

The classpath cannot access the modulepath in the new module layer. The only way to gain access is with reflection using the LaunchContext::getClassLoader class loader:

Class<?> clazz = Class.forName("com.example.ClassName", true, ctx.getClassLoader());

The classpath does have access to the whole bootstrap application, even on the modulepath, even without reflection.

Access Classpath and Modulepath in Bootstrap

The bootstrap application does not have access to the classes added after a launch. You can access it via reflection in the same manner detailed above.

URI and Path

All files must be referenced by a valid absolute URI and absoltue path. The URI will be used to locate the remote file and download it when it updates. The path it used to point to the local location where the file should be downloaded to. If most of your files have the same base URI/path you may reference it in the <base> element. All relative locations will resolve against the base. You might still override the base by giving an absolute URI/path for the file.

If the base is specified, and the file has one of the two (only uri or path) specified, it will infer the other by either using the full path of the other - if it is relative to the base - or just the last part - if it is absolute.


Note: If an OS specific file's URI or Path uses an OS specific property in the configuration file, Configuration::read will not be able to resolve these values on other operating systems. The getUri() and getPath() methods in FileMetadata will always return null in such cases.

Security

For maximum security you can sign files in the config and also the config itself.

When you sign the files and pass a public key to the update() method it will verify newly downloaded files against the key. It will not check local files.

When the configuration itself is signed and you read it with the Configuration.read(Reader, PublicKey) method, it will verify the contents inside the <configuration> block has not changed. The timestamp and the signature fields and not verified. It will ony verify for significant changes, namely, anything that changes how the resulting config object will load. Whitespaces, attribute and element ordering are insignificant. List elements order (as <file> and <property>) are significant, as they change how properties are resolved and which file is downloaded first.

When using the default bootstrap, passing the --cert option will verify both of the about.

Signing

There are 2 ways to sign: Using the Builder API, or syncing.

The Builder API has the method signer(); passing a private key will sign both the files and the configuration itself.

Calling Configuration::sync with a private key will update the currently listed files' size, checksum, and signature. It will also re-sign the configuration.

Integration with JLink

The jlink command is a very handy tool to create smaller Java images for production. By default, jlink bakes the required modules and its dependencies into the image and cannot be updated by update4j. With a little work, we can hand modify the image and get the required results.

Create the image

To create an image, run this command:

jlink --add-modules java.se  --output image --compress=2

You will get a new directory image with the substructure:

image
  |- bin
  |- conf
  |- include
  |- legal
  |- lib
  |- release

Create directories to match this structure:

image
  |- app
      |- business
      |- classpath
      |- modulepath
      |- update4j	

Launcher script

Under the app directory create a shell script:

Windows

@echo off

set VM_OPTIONS= -p ../app/update4j/update4j-<version>.jar;../app/modulepath -cp ../app/classpath/*

set APP_ARGS=

"../bin/java" %VM_OPTIONS% -m org.update4j/org.update4j.Bootstrap %APP_ARGS% %*

UNIX

#!/bin/sh

VM_OPTIONS= -p ../app/update4j/update4j-<version>.jar:../app/modulepath -cp ../app/classpath/*

APP_ARGS=

../bin/java $VM_OPTIONS -m org.update4j/org.update4j.Bootstrap $APP_ARGS $@

Drop the update4j module into the update4j directory and replace the <version> in above scripts with the actual filename.

For applications running in standard mode change the -m org.update4j/org.update4j.Bootstrap with the appropriate module/mainClass


Note: Since we generally want to be able to push any type of update that may depend on system modules even those you do not currently use, we include the whole java.se module in the image.

If you really want to create smaller images and you know you will never depend on certain modules you may list those you want to include manually. In that case you cannot start update4j in the modulepath, since it expresses a dependency to java.se. You can modify the script to include ../app/update4j/update4j-<version>.jar under -cp instead of -p and remove -m org.update4j/.


The classpath and modulepath directories are reserved for bootstrap dependencies only. All business app dependencies should reside in the business directory. Do not add business to the script, they should be loaded dynamically.


Note: On Windows, you can create an exe file that wraps around a script. Check out Bat to Exe Converter.


Updating update4j itself

Updating the framework itself is very easy: Add the new update4j file to the config file and let it download to the update4j directory. Include the script itself in the config and simply change the <version> to the new version. Once both of these files are updated, next launch will only point to the new version.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.