IEP |
9 |
Title |
Incremental builds Maven repository |
Author |
|
Status |
💬 In-process |
Type |
Architecture |
Created |
2018-03-19 |
Discussions-To |
In order to support more rapid continuous delivery models, such as that
described by
Jenkins Essentials,
Jenkins core and plugin builds must be deployed into a Maven repository much
more incrementally rather than waiting for a developer to manually deploy a
release to the existing releases
[1]
repository.
For a more incremental developer workflow, rather than attempting to make sense
of potentially thousands of SNAPSHOT
versions of artifacts in a repository.
this document proposes a specific and tightly controlled Maven repository for
such builds.
This would mean Artifactory would have an incrementals
repository just for
these kinds of pre-release builds. For example, rather than
git-client-2.4.0-SNAPSHOT.jar
the repository would contain
git-client-2.4.0-a3dbf.jar
, assuming a3dbf
is the Git short-commit for the
build.
Artifacts in the incrementals
repository must be garbage collected to the
most recent 5 artifacts or the most recent 30 days of artifacts. For
example, if a plugin (io.jenkins.plugins.retrocean
) has no new commits in a
30 day period, the io/jenkins/plugins/retrocean/
directory should have no
more than the five most recent artifacts stored. More active plugins should
have no more than 30 days worth of artifacts stored.
ℹ️
|
Plugin and core tooling should use a consistent format for defining incremental
built versions, such as: |
By adding this new, somewhat ephemeral, Maven repository, we can support newer development workflows in the Jenkins project without adversely affecting the existing "mainstream" development workflow of Jenkins plugins.
As mentioned briefly in Costs, overhead is sufficiently low to adding that adding a new Maven repository, even if it’s experimental and eventually abandoned, is worth the exploration.
A Maven repository hosted in Artifactory, alongside the releases
and a number
of other repositories, results in the simplest to deploy, and simplest to
consume "bucket of artifacts" we currently have at our disposal.
In order to avoid over-reliance on these "incremental builds", the artifacts themselves should be expected to be deleted after the specified "garbage collection" period.
One alternative approach which was briefly discussed in person with Carlos Sanchez was to drop these "incremental build" artifacts directly into an Azure storage container.
This approach was rejected as it would require additional non-native tooling to be used in the development workflow. By adopting a Maven repository, existing tools for grabbing Maven dependencies can be utilized by developers wishing to incorporate "incremental builds" into their build and test workflows.
Re-using the existing releases
Maven repository was explicitly not
considered as an alternative approach as the Jenkins infrastructure team has
had multiple performance issues
[2]
with that repository over the past couple of months. Most notably, the indexing
times in the repository have varied between 10 minutes and over 60 minutes due
to opaque server-side issues. The consequence of adding more load, and many
more artifacts, to the repository would severely affect the indexing time and
adversely affect the ability for Jenkins users to receive new plugin updates in
the Update Center [3].
As Artifactory is a hosted service provided by JFrog, it is not expected that any additional financial cost will be involved in this proposal.
Additionally, since this document proposes an additional repository, it is not
expected to incur any substantial runtime/performance cost on the existing
Update Center and releases
repository indexing process.
Since we have no staging implementation of Artifactory, there is no reference implementation, acceptance of this IEP document would result in a new repository being created on repo.jenkins-ci.org.