Skip to content

Latest commit

 

History

History
343 lines (230 loc) · 20.3 KB

File metadata and controls

343 lines (230 loc) · 20.3 KB

WildFly Channels - Specification

WildFly Channels

Channel schema Version

2.1.0

Manifest schema Version

1.1.0

Blocklist schema Version

1.0.0

Summary

This document describes the overall architecture and concepts related to Channels to provision WildFly.

Concepts

Provisioning Concepts

WildFly uses Galleon to provision and update its installation. All concepts related to Galleon are defined in its documentation

Component Version

It is out of scope of this document to define how WildFly and the components it incorporates are versioned. However, channels are referencing Maven versions. A channel can specify a pattern of version and pick the “latest” version that matches that pattern.

This "latest" version is determined by comparing all the versions and returns the last one using the org.wildfly.channel.version.VersionMatcher.getLatestVersion(Set<String> versions) method.

Note

This provides a deterministic way to find the "latest" version. However, it is out of scope of this specification to capture all the different version patterns. Our recommendation is to specify a pattern in versionPattern that constrains as much as possible the set of versions to compare before determine the latest ones among them.

Channel Compatibility

Backwards compatibility is only guaranteed inside a single channel. If an installation subscribes to multiple channels, the backwards compatibility of the whole installation is not guaranteed.

Channel Model

Channel combines a Channel Manifest defining content of channel, one or more Repositories from which the content can be resolved and an optional Blocklist Manifest.

Channel definition

A channel is composed of several fields:

  • An optional name that is a human-readable one-line description of the channel (Channel for WildFly 27)

  • An optional description that provides human-readable description of the channel

  • An optional vendor that defines who provides the channel and their support level. This field is descriptive only and is not used to enforce any mechanism. This field is composed of:

    • A required name to identify the vendor (eg WildFly Community Project)

    • A required support type that accepts the following values:

      • supported - Components provided by this channel are supported by the vendor. Some features provided by this channel can still be considered as tech-preview.

      • tech-preview - Feature provided by this channel are Tech Preview by the vendor

      • community - Components provided by this channel are a community effort from the vendor

  • A collection of repositories that defines repositories associated with the channel. Only listed repositories are used to resolve channel’s components. Each repository is composed of:

    • A required id that defines the repository name used in Maven cache

    • A required url pointing to a default Maven repository

  • Optional manifest corresponding to the Channel Manifest artifact. Channel Manifest is used to define streams available in the channel.

    • One of the following, mutually exclusive fields, used to resolve the Channel Manifest:

      • maven corresponds to Maven coordinates of the Manifest. It’s composed of:

        • Mandatory groupId and artifactId elements that are the Maven coordinates of the manifest.

        • Optional version to stick to a given manifest version (instead of requiring the latest version of that manifest). In the absence of this version, the latest version of the manifest will be determined based on the Maven repository metadata (see Component Version).

      • url corresponding to a URL where the manifest file can be found.

      • signature-url corresponding to a URL where the signature of the manifest file can be found.

  • Optional blocklist corresponding to the Blocklist artifact. Blocklist is used to define versions of artifacts excluded from a channel.

    • One of the following, mutually exclusive fields, used to resolve the Blocklist:

      • maven corresponds to Maven coordinates of the Blocklist. It’s composed of:

        • Mandatory groupId and artifactId elements that are the Maven coordinates of the blocklist.

        • Optional version to stick to a given blocklist version (instead of requiring the latest version of that blocklist). In the absence of this version, the latest version of the blocklist will be determined based on the Maven repository metadata (see Component Version).

      • url corresponding to a URL where the blocklist file can be found.

  • An optional resolve-if-no-stream that defines strategy for version resolution if the artifact has not been found in the streams collection or if the channel didn’t declare a manifest. Allowed values are:

    • latest - use the latest version found in the Maven repositories (using mechanism described in Component Version)

    • maven-latest - a version marked as latest in the Maven metadata

    • maven-release - a version marked as release in the Maven metadata

    • none - do not attempt to resolve versions of artifact not listed in the streams collection. Default value if no strategy is provided.

  • Optional gpg-check boolean indicating that during artifact resolution, this channel will verify a signature of every artifact. If the artifact signature cannot be found, or cannot be validated, the artifact will not be resolved from the channel. The channel repositories must contain a detached GPG signature paired with the artifact as described below.

  • Optional gpg-urls a list of URLs that the public GPG keys used to validate artifact signatures are resolved from.

Manifest definition

A Channel Manifest is composed of following fields:

  • An optional name that is a human-readable one-line description of the channel (manifest for WildFly 27)

  • An optional id element that is used to identify channel.

  • An optional description that provides human-readable description of the channel

  • An optional logical-version that provides a human-readable short description of the version of the manifest. Note this may, but doesn’t need to correspond to the Maven version of the artifact (from Manifest schema 1.1.0).

  • A collection of requires. Each element of that list corresponds to another channel that is required to provision components from this channel. This field can be used for layered products to enforce their dependencies so that the installation only need to update the top level channel to get updates from all required channels. Each element is composed of:

    • Mandatory id element corresponding to the ID of required manifest.

    • Optional maven element representing Maven coordinates the required manifest is published at.

      • Mandatory groupId and artifactId elements.

      • Optional version to stick to a given manifest version (instead of requiring the latest version of that manifest). In the absence of this version, the latest version of the manifest will be determined based on the Maven repository metadata.

  • A collection of streams that defines all the components installable from this channel. Each stream is composed of:

    • A required groupId that corresponds to Maven GroupId to pull artifacts (it is not allowed to specify * for the groupId).

    • A required artifactId that corresponds to Maven ArtifactId to pull artifacts. Special syntax * can be used to match any artifactId.

    • One of the following fields (which are mutually exclusive) that provides rules to resolve the Maven artifacts to provision. At most one field must be present in the stream definition.

      • versionPattern corresponds to a Pattern through which the available versions are matched (e.g. 2\.2\..*)

      • version corresponds to a single version (e.g. 2.2.Final)

Blocklist definition

  • A collection of blocks that defines all the component versions excluded from version resolution. Each exclusion is composed of:

    • A required groupId that corresponds to Maven GroupId of the excluded artifacts (it is not allowed to specify * for the groupId).

    • A required artifactId that corresponds to Maven ArtifactId of the excluded artifacts. Special syntax * can be used to match any artifactId.

    • A required list of versions corresponding to excluded version (e.g. [2.2.Final, 2.2.1.Final])

Channel Schema

JSON Schemas are provided to validate that channels and manifests comply with the model described above.

Channel Representation

A channel is specified in the YAML language with a corresponding JSON schema to validate its structure.

A manifest is specified in the YAML language with a corresponding JSON schema to validate its structure.

A blocklist is specified in the YAML language with a corresponding JSON schema to validate its structure.

Channel Actions and Responsibilities

Create a channel

A minimal Channel definition is a single file that complies with the channel YAML representation. The Channel definition may reference two additional resources: a Manifest file and a Blocklist file.

Creating a channel corresponds to the creation of the above files.

Publish a channel

A channel may be “published” so that it can be read (or downloaded) by WildFly provisioning tooling. Channels are published as a Maven artifact with the classifier channel and extension/type yaml.

If the Channel uses maven coordinates to reference Manifest and/or Blocklist, those files must be "published" in one of the repositories defined in the Channel so that the Channel is able to resolve them. Otherwise, those files must be made available at the URLs defined by the Channel.

When manifests are published as Maven artifacts, they must use the classifier manifest and extension/type yaml.

When blocklists are published as Maven artifacts, they must use the classifier blocklist and extension/type yaml.

Update a channel

A channel definition and manifest can be updated to add or modify streams, change its requirements, etc. Each of the channel files can be updated independently of each other.

To update a channel file, it needs to be published with a new version.

Consume a channel

The main consumers of WildFly Channels are the provisioning tooling provided by the WildFly project.

They consume channels by pulling the channel artifact corresponding to the groupId/artifactId of a channel. If a version is specified, the channel corresponding to that version is pulled. Otherwise, the latest version of the channel is determined based on the Maven metadata from the repository that hosts the channel artifacts.

Resolving channel manifest

The manifest is resolved when a channel is initialized. The channel can omit the manifest definition in which case the resolve-if-no-stream strategy will be used to resolve artifacts.

If the channel defines a manifest using URL, the manifest will be read from that location. If instead maven coordinates are used, the specified version of manifest is resolved from the channel’s repositories. If only groupId and artifactId is provided, the latest available version of the channel manifest (as defined in Component Version) will be used.

If the chanel defines a manifest, but no manifest can be resolved (using either URL or GA[V]), an error will be thrown.

Resolving required channels

Required channels are identified by their manifests' ID. If multiple channels are used in the channel session, the required IDs are first resolved from the list of channels.

If an ID cannot be resolved in this way, and the requirement provides maven element, a resolution will be attempted using provided maven coordinates. The parent channel’s repositories will be used to resolve the new manifest and the created channel will inherit repository settings from the parent channel.

If the resolution cannot be achieved, an error will be thrown.

If the required channels form a cycle, an error will be thrown.

If the set of channels contains non-unique manifest IDs, an error will be thrown.

Maven Artifact resolution

A Maven artifact can be resolved through a channel. Such a resolution will use the Maven repositories defined within the channel. If a channel requires a different channel, the required channel will use the repositories from its own definition.

The channels will be searched for a stream that matches the groupId/artifactId of the artifact.

If a channel directly defines a stream that matches the groupId/artifactId of the artifact, the version will be resolved from this stream.

If channel does not directly define a stream, required channels will be searched. The latest version of the stream found in the required channels will be used.

If multiple channels are defined, the latest version from any channel that defines the stream (directly or through required channels) is used. Channel which manifests are required by another channel, are excluded from a direct search.

If no stream that matches the artifact have been found, version is resolved using fallback strategy defined in resolve-if-no-stream for the channel. An error is returned to the caller if:

  • the fallback strategy is none

  • the fallback strategy is latest, maven-latest or maven-release but underlying Maven repository contains no metadata for artifacts with required groupId and artifactId

If the stream defines a version, the artifact will be resolved based on this version. If that version of the artifact can not be pulled from the Maven repositories, an error is returned to the caller. If the stream defines a versionPattern, the version will be determined by querying the version of the artifacts from the Maven repositories and use the latest version that matches the pattern. If no version matches the pattern, an error is returned to the caller.

Maven repository proxies

A channel defines repositories using their id and a default url. When creating the channel from definition the provisioning tool can replace the provided url with URL of a proxy server or an alternative repository. The id of the repository must not be changed.

Authenticating URL-based metadata files

When channel definitions, manifests, or blocklists are hosted on remote servers that require authentication, the WildFly Channel library supports pre-emptive authentication using Bearer tokens.

Configuration

Authentication can be configured using either a system property or an environment variable:

  • System property: org.wildfly.channel.http.auth.token

  • Environment variable: WILDFLY_CHANNEL_HTTP_AUTH_TOKEN

The system property takes precedence over the environment variable. If the system property is not set (or is a blank value), the environment variable will be checked as a fallback.

When either configuration is set to a non-blank value, the library will automatically include an Authorization header with the Bearer token in all HTTP and HTTPS requests for channel metadata files.

Supported resources

Pre-emptive authentication is supported for the following resources when accessed via URL:

  • Channel definitions (when using url field in channel resolution)

  • Channel manifests (when using url field in manifest coordinate definition)

  • Blocklists (when using url field in blocklist coordinate definition)

  • Manifest signature files (when using signature-url field)

Protocol requirements

Authentication is only applied to HTTP and HTTPS URLs. Other protocols (such as file://) do not support authentication and will ignore the configured token.

Authorization header format

When the org.wildfly.channel.http.auth.token property is set to a non-blank value, the library sends the following HTTP header:

Authorization: Bearer <token>

Where <token> is the value of the system property.

Usage examples

To configure authentication for downloading channel metadata using a system property:

java -Dorg.wildfly.channel.http.auth.token=my-secret-token ...

Alternatively, using an environment variable:

export WILDFLY_CHANNEL_HTTP_AUTH_TOKEN=my-secret-token
java ...

Or set inline:

WILDFLY_CHANNEL_HTTP_AUTH_TOKEN=my-secret-token java ...

Any of these configurations will cause all HTTP/HTTPS requests for channel, manifest, and blocklist files to include the Bearer token in the Authorization header.

Note

The authentication token is sent with every HTTP/HTTPS request for metadata files. Ensure that:

  • The token is kept secure and not exposed in logs or version control

  • HTTPS is used when transmitting sensitive tokens to prevent interception

  • The token has appropriate permissions for accessing the required resources

Blocking artifact versions

When using an open channel, a situation may arise where certain artifacts are promoted to the channel and later discovered to be invalid. As it’s impossible to remove artifacts already deployed into a repository, those versions have to blocklisted.

Format

The blocklist file contains a list of stream Maven coordinates with multiple versions:

---
blocks:
- groupId: io.undertow
  artifactId: undertow-core
  versions:
  - 2.2.18.Final
  - 2.2.17.Final
…

The artifactId can use a wildcard syntax to refer to all the artifacts with the same groupId

---
blocks:
- groupId: io.undertow
  artifactId: “*”
  versions:
  - 2.2.18.Final
  - 2.2.17.Final

Resolving channel blocklist

The blocklist is resolved when a channel is initialized.

If the channel defines a blocklist using URL, the blocklist will be read from that location. If instead maven coordinates are used, the specified version of manifest is resolved from the channel’s repositories. If only groupId and artifactId is provided, the latest available version of the blocklist (as defined in Component Version) will be used.

If no blocklist artifact can be resolved with supplied Maven coordinates, the channel treats it as an empty blocklist.

A blocklist applies only to the channel that defined it, not its required channels.

Resolving artifact version

During artifact version resolution, a stream matching artifact GA is located in the channel. The blocklist is always checked for excluded versions, except when using resolveDirectMavenArtifact method. The excluded versions are removed from the set of available artifact versions before the latest remaining version matching the stream’s pattern is used to resolve the artifact. If the blocklist excludes all available artifact versions, UnresolvedMavenArtifactException is thrown.

Verifying artifact PGP signatures

If a channel sets value of gpg-check property to true, any artifact resolved from it (including the manifest itself) must have a valid GPG signature.

Verifying Maven artifact signatures

Maven artifacts are expected to have their detached GPG signatures available in the channel repositories. The detached artifacts must have the same GAV as the artifact, but append ".asc" suffix to the file extension. The signature file must contain an armoured GPG signature.

The signature file is resolved at the same time as the artifact. If the signature is invalid, the public key is not found, the public key is expired or revoked, the artifact must be rejected and a SignatureException must be thrown.

Verifying manifest signatures

If the manifest of a channel is defined as a Maven GA(V), it is treated as any other maven artifact. If it is defined as an URL, the signature file must be available at the same URL with ".asc" suffix. Alternatively, a signature-url element can be used to provide a location of the signature. If the signature cannot be resolved, the channel creation must fail.

Public keys

The public keys are identified by the a keyID in hexadecimal form. The public keys required by the channel can be defined using gpg-urls property. Each gpg-urls must be a URL to a downloadable key.

Implementations may provide additional means of providing keys - local stores, remote keyservers, etc.

Changelog

Version 2.1.0

  • Adding ability to verify artifact signatures

    • Adding gpg-check and gpg-urls fields to the Channel definition.

    • Adding signature-url field to the Channel’s manifest coordinate definition.

Version 2.0.0

  • Introduction of the Channel Manifest and Blocklist

Version 1.0.0

  • Initial release of the Channel specification