OpenSearch: Automate increment to next development iteration

[dblock]

Is your feature request related to a problem? Please describe

We begin incrementing versions after we decide that we want a release. This takes a day and makes it that nightly distribution builds do not include full bundle until T-14 days to release (#1140). Instead, prepare the project to make the next release after the previous release is done, so that we can save that day and have release candidates always available for the next development iteration.

After automatic tagging has happened, increment the version for the next development iteration. For example:

• when we release 1.2.1, increment version to 1.2.2 on the 1.2 branch
• when we release 1.3.0, increment version to 1.4.0 on 1.x branch and 1.3.1 on 1.3 branch
• when we release 2.0, increment version to 3.0 on main

x 15 plugins

Describe the solution you'd like

Every project should have a workflow that notices new tags, and increments versions accordingly and makes a PR.

• On OpenSearch this change is a bit involved because it needs custom work in bcwVersions.
• In plugins it's mostly build.gradle, but there are plugins with their own weird custom steps

Acceptance Criteria

• POC Concept/Solution to Automate the version increment to next development iteration
  POC: Concept/Solution to Automate the version increment to next development iteration. #2215
• Implement Automated workflow that will increment the version for plugin and take care of all other code changes required to pass the CI.
  Implement Automated workflow that will increment the version for plugin and take care of all other code changes #2223
  Pr: Workflow for version increment automation.  #2291
• PR is merged upon review and ready to use.

Additional Acceptance Criteria

Follow Core Branching Strategy (Ensure 1.x and 2.x branches).
opensearch-project/opensearch-plugins#142

Campaign Issues:

OpenSearch

Solution Proposed
Gradle project: Staging to add gradle tasks (setVersion and versionIncrement) that support version increment automation for supported versions and add/update gradle.properties file to the project.

• opensearch-plugins
  Pr's: Add Version increment automation task opensearch-plugin-template-java#32
• alerting
  Pr's:
  Staging for version increment automation alerting#489
• anomaly-detection
  Staging for version increment automation anomaly-detection#624
  Pr's:
  1.x: Staging for version increment automation anomaly-detection#607
  1.3: Staging for version increment automation anomaly-detection#603
  main: Staging for version increment automation anomaly-detection#608
• asynchronous-search
  Pr's: Staging for version increment automation asynchronous-search#157
  1.x: Staging for version increment automation asynchronous-search#161
• common-utils
  Pr's:
  1.3: Staging for version increment automation. common-utils#192
  1.x: Staging for version increment automation common-utils#199
  main: Staging for version increment automation common-utils#200
• dashboards-reports
  Pr's: Staging for version increment automation reporting#391
• index-management
  Pr's: Staging for version increment automation index-management#409
• job-scheduler
  Pr's:
  1.x: Staging for version increment automation.  job-scheduler#203
  1.3: Staging for version increment automation.  job-scheduler#196
  Main: Staging for version increment automation job-scheduler#204
• k-NN
  Pr's:
  1.3: Staging for version increment automation k-NN#432
  1.x: Staging for version increment automation k-NN#436
  main: Staging for version increment automation k-NN#437
  Staging for version increment automation k-NN#442
  Pr's: Staging for version increment automation k-NN#432
• notifications
  Pr's: Staging for version increment automation notifications#476
• performance-analyzer
  Pr's:
  1.x: Staging Version increment automation performance-analyzer#239
  main: Staging for version increment automation performance-analyzer#238
• performance-analyzer-rca
  Pr's:
  main: Staging for version increment automation performance-analyzer-rca#197
  2.1: Version increment automation: update task name performance-analyzer-rca#213
  2.2: Version increment automation: update task name performance-analyzer-rca#214
• security
  Pr's:
  main: Staging for version increment automation security#1932
  1.x: Staging for version increment automation security#1933
• sql
  Pr's: Staging for version increment automation sql#684
• observability
  Pr's: Staging for version increment automation observability#848
• cross-cluster-replication
  Pr's:
  main: Staging for version increment automation  cross-cluster-replication#449
  backport 2.1: [backport 2.1]Staging for version increment automation  cross-cluster-replication#465
  1.3: Staging for version increment automation cross-cluster-replication#466
• ml-commons
  Pr's:
  1.x: Staging for version increment automation ml-commons#363
  main: Staging for version increment automation ml-commons#362
  1.3: [backport 1.3]Staging for version increment automation ml-commons#375
• geospatial
  Pr's: Staging for version increment automation  geospatial#91

OpenSearch Dashboards

Proposal Pending

Remove the dependency with hardcoded zips for 1.x versions:

• index-management
  Example with hardcoded zips: Version Increment to 1.3.3 OpenSearch release index-management#374
• dashboards-reports
  Example with hardcoded zips: Version Increment to 1.3.3 OpenSearch release reporting#366
• sql
  Example with hardcoded zips: Incremented version to 1.3.2. sql#593

Describe alternatives you've considered

No response

Additional context

It takes a day for 1 person to increment the version everywhere, documented in opensearch-project/opensearch-plugins#119. Issues that would reduce that amount of work aside of what's proposed here.

• [BUG] Checked in versions of job-scheduler-*-SNAPSHOT.zip  anomaly-detection#17
• [BUG] Replace checked in reports-scheduler/src/test/resources/job-scheduler/opensearch-job-scheduler-1.2.3.0-*.zip with a dynamic dependency reporting#263
• [BUG] Project integration tests require same version of plugins to be checked in index-management#123
• Replace maven by gradle security#1533
• [BUG] Use snapshots from distribution/maven for dependencies  cross-cluster-replication#274

[peternied]

@dblock what do you think about updating the component release template with these as steps? Then we could create a campaign to automation this step in each repository, which a base case that hits 80% of them with a copypasta workflow

[ohltyler]

@dblock is there any further blocker for other component teams to set this up themselves? Should this be added as a campaign for 2.0.0?

[dblock]

It didn't work for common-utils: https://github.com/opensearch-project/common-utils/actions/runs/2000270385, needs to be debugged, opensearch-project/common-utils#138.

It didn't work for OpenSearch, https://github.com/opensearch-project/OpenSearch/actions/runs/2000274118, the PR GHA needed to be added to permissions (I did that).

It didn't quite work for job-scheduler, opensearch-project/job-scheduler#148, DCO was one problem.

Before adding more we need to fix these, but I didn't get to it. If someone wants to pick these up, go for it!

[VachaShah]

@dblock I added the secrets for common-utils. I will fix DCO for both job-scheduler and common-utils.

[dblock]

We also need this in https://github.com/opensearch-project/OpenSearch/blob/main/.github/workflows/version.yml#L56, and possibly in other places? Would you mind adding it too please?

[VachaShah]

We also need this in https://github.com/opensearch-project/OpenSearch/blob/main/.github/workflows/version.yml#L56, and possibly in other places? Would you mind adding it too please?

Yup added it in OpenSearch version.yml PR opensearch-project/OpenSearch#2572. I will see if there are any other pull request workflows that need the signoff option.

[prudhvigodithi]

Proposed Solution:

Automate this version increment process across all plugin components, that increment’s the version, modifies the required files and raise a PR that awaits for plugin team approval, this way an engineer need not spend hours of time to increment the version for all plugin components.

Solution Details:

The following details uses sql component, version increment process, to demonstrate the solution workflow.

• Each plugin will have a gradle property opensearch.version stored in gradle.properties file.
  Example opensearch.version=2.0.0-SNAPSHOT https://github.com/prudhvigodithi/sql/blob/main/gradle.properties

  With current/existing setup depending on OpenSearch version, the plugins version is inferred, this property is injected using gradle system properties.
  Moving forward this version will be inferred from plugin specific gradle property and rest of the flow just depend upon this opensearch.version gradle property.

  Note: This can be debated if opensearch.version should be passed from CI during distribution runtime or straight away use from plugin own gradle properties.
  Example:
  Existing flow -Dopensearch.version=$VERSION from build script is inferred from manifest
  Proposed Flow, to directly use project properties.

• Each Plugin, gradle build system will have two additional tasks which are responsible to increment the version and modify the required files.

  1. setVersion : This task will update the gradle property opensearch.version and updates the value in gradle.properties file.

  2. versionIncrement : This task is configurable and depends on the plugin requirement to modify any files where version is present and that needs to incremented, this task uses ant.replaceregexp to parse the required files that are added in include() section of the task. Example for sql

  For JSON files to increment the version, plugin teams can even include gradle node processes inside versionIncrement task, instead of RegEx parsing. However in the sql example i have used RegEx parsing to increment the version.
  https://ant.apache.org/manual/Tasks/replaceregexp.html

• The task setVersion is dependent on versionIncrement task, so when called ./gradlew setVersion -PnewVersion=2.1.0-SNAPSHOT (this task will be called using GitHub workflow explained below) the gradle property in gradle.propperties file gets updated to opensearch.version=2.1.0-SNAPSHOT and all include() section files will be replaced from 2.0.0 to 2.1.0 or 2.1.0.0 depending upon the value in the file.

• Using GitHub workflows the plugin specific branch is compared against the OpenSearch specific branch to check for OpenSearch version, if it’s different, i.e if OpenSearch is incremented and not the plugin, the workflow will auto raise a PR.
  Example:
  plugin 1.3 branch will look for OpenSearch repo 1.3 branch, checks if it has same version.
  plugin 2.x branch will look for OpenSearch repo 2.x branch, checks if it has same version.
  If the OpenSearch version is different (condition check done using github workflow) from what the plugin has the workflow will execute ./gradlew setVersion -PnewVersion=<OpenSearch_Version> which performs the actual version increment process and raises a PR.
  Sample PR’s: [AUTO] Increment version to 3.0.0-SNAPSHOT prudhvigodithi/sql#3 [AUTO] Version Increment to 3.0.0-SNAPSHOT prudhvigodithi/sql#2

  Note: This workflow can be scheduled using cron, on after release tag cut. This workflow can be part of each plugin .github/workflows/ or can be researched to be part of central build repo and target to create PR’s across all plugin components.

[prudhvigodithi]

Hey please add your thoughts for the above added solution.
@dblock, @bbarani, @peternied, @VachaShah, @ohltyler

[dblock]

I like it! Make it work.

[prudhvigodithi]

For non-Gradle projects:

Since Gradle support seamless integration of tasks for build automation, it’s preferable to add a gradle project to existing repos, for this integration a repo needs build.gradle, and gradle installation files. Then the implementation flow would be same as specified above to add setVersion and versionIncrement task.

The following details uses alerting-dashboards-plugin component, version increment process, to demonstrate the solution workflow.
Existing Sample version increment PR for alerting-dashboards-plugin
opensearch-project/alerting-dashboards-plugin#277

Sample build.gradle file

/*
 * Copyright OpenSearch Contributors
 * SPDX-License-Identifier: Apache-2.0
 */

buildscript {
    ext {
        opensearch_version = project.property('opensearch.version')
    }
}

task versionIncrement {
    final String workingDir = project.buildDir.toString() + "/../"
    if(project.hasProperty('newVersion')) {
         println 'Set Project to new Version '+newVersion.tokenize('-')[0]
        ant.replaceregexp(match: opensearch_version.tokenize('-')[0], replace: newVersion.tokenize('-')[0], flags:'g', byline:true) {
            fileset(dir: workingDir) {
                include(name: ".github/workflows/cypress-workflow.yml")
            }
        }
        ant.replaceregexp(match:'"version": "\\d+.\\d+.\\d+.\\d+', replace:'"version": ' + '"' + newVersion.tokenize('-')[0] + '.0', flags:'g', byline:true) {
            fileset(dir: workingDir) {
                include(name: "package.json")
                include(name: "opensearch_dashboards.json")
            }
        }
        ant.replaceregexp(file:'opensearch_dashboards.json', match:'"opensearchDashboardsVersion": "\\d+.\\d+.\\d+', replace:'"opensearchDashboardsVersion": ' + '"' + newVersion.tokenize('-')[0], flags:'g', byline:true)
    }
}
task setVersion(dependsOn: versionIncrement) {
    if(project.hasProperty('newVersion')) {
        ant.propertyfile(
            file: "gradle.properties") {
            entry( key: "opensearch.version", value: "${newVersion}")
        }
    }
}

Existing version:

yarn version

info Current version: *2.0.1.0*

Executing the gradle task:
./gradlew setVersion -PnewVersion=2.1.0-SNAPSHOT

Version Increment:

yarn version

Current version: *2.1.0.0*

The alternative approach is to use npm-version, but for each repo the version is present in more than one file, this requires additional linux parsing logic apart from regular npm commands.
Going with gradle way ensures consistent solution which is in par with other gradle projects.

thoughts @dblock @bbarani @peterzhuamazon ?

[dblock]

Since most operations are ant.relaceregex, and I think all should be basically identical or at least very similar, I think you can go one step further and implement an extension in which a plugin only has to list the files which contain a version number, and move all the logic into the shared plugin implementation, e.g.:

incrementVersion {
   fileset(dir: workingDir) {
       include(name: ".github/workflows/cypress-workflow.yml")
       include(name: 'build.gradle')
   }
}

[prudhvigodithi]

This META can be closed, the version increment for OpenSearch plugins is now automated with a single click using the GH workflow (Sample Run: https://github.com/opensearch-project/opensearch-build/actions/runs/2967596360)
(The workflow only fails if the release branch does not exists from the plugin end, then it has to be re-triggered once the branch is created by the plugin team.)

The only pending issue open from this META are:
The branch sync, since this is an ongoing process in discussion with plugins, the version increment workflow will continue to compare against Core to raise an auto version increment PR.
@dblock @bbarani

[dblock]

But so it's not automated until it is, let's reopen it :) Which plugins don't follow the branching strategy that keep breaking this? It's a long list in opensearch-project/opensearch-plugins#142?

[prudhvigodithi]

The automation to add the branch entry to the version increment workflow is now done using the existing manifest workflow
Related Issue: #2601
Related PR: #2602

[prudhvigodithi]

The version increment workflow, will be now triggered daily with cron expression cron: 0 0 * * * added to the github workflow
Related PR: #2623

[prudhvigodithi]

Hey @dblock the automation is now in place, the branch entry is added via the manifest workflow, sample PR: https://github.com/opensearch-project/opensearch-build/pull/2628/files, and the workflow is now triggered using cron: 0 0 * * *, if you have any thoughts please let me know, else I will proceed to close this.
Thank you

[prudhvigodithi]

Thanks, closing this issue, please feel free to re-open if required.

[peternied]

@prudhvigodithi With Also handle BWC tests as part of version increment automation #2373 this process isn't fully automated. Are we tracking that issue separately?

[prudhvigodithi]

Hey @peternied as mentioned, the version increment process is generic and is not related to any BWC tests, each plugin right now follow their own way to add the BWC test version change and also the version change files are not common, this has to be handled from plugin team and this change is specific with a major version release.
Thank you
@bbarani @zelinh

[prudhvigodithi]

@peternied this can be extended within the plugin own updateVersion (example for security) gradle task, so when called using version-increment workflow, the BWC test version will also be incremented.