[8.19] [ML] Automate patch version bump in CI pipeline (#3030)#3067
Merged
Conversation
…ngelog unittest (elastic#3052) - Add dev-tools pytest step (python:3.11-bookworm) with test-requirements.txt including PyYAML/jsonschema so changelog helpers import during collection. - Add run_dev_tools_tests.sh, pytest.ini, and dev_tools_pytest Buildkite step. - Restrict test-changelog-tools.sh unittest discover to test_changelog_tools.py so pytest-only modules are not loaded without pytest installed. Co-authored-by: Cursor <cursoragent@cursor.com>
Replace the ml-cpp-version-bump stub with a two-phase Buildkite flow. Phase 1 (job-version-bump.json.py): validate_version_bump_params.sh on Wolfi (patch-only rules; optional git fetch re-check when enabled), then version_bump_upload_phase2.sh uploads phase 2 only when work remains (step conditionals cannot rely on build meta-data). Phase 2 (job-version-bump-phase2.json.py): bump_version.sh bumps elasticsearchVersion on a ci/ml-cpp-version-bump-* topic branch, pushes, opens a PR via GitHub CLI (ensure_github_cli.sh, create_github_pull_request.sh), defaults to gh pr merge --auto --squash, optional immediate merge and VERSION_BUMP_MERGE_ADMIN; records meta-data for no-op and PR URL. Slack step (send_slack_version_bump_notification.sh) uses Wolfi and step notify. DRA wait (wait_version_bump_dra.py) skips on DRY_RUN or when validation/bump report no change. Adds dev-tools/unittest for validation and pipeline JSON. Does not modify .backportrc.json (reserved for future release automation). Minor version bumps stay a follow-up change. Co-authored-by: Cursor <cursoragent@cursor.com>
edsavage
added
>build
>non-issue
:ml
backport
auto-backport
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
There was a problem hiding this comment.
Pull request overview
Backports the automated patch elasticsearchVersion bump workflow to the 8.19 branch by adding a two-phase Buildkite pipeline (validate → upload phase 2) plus the supporting dev-tools scripts/tests, and bundles the prerequisite dev-tools pytest CI wiring.
Changes:
- Add Buildkite phase 1/phase 2 pipeline generators for the
ml-cpp-version-bumpworkflow, including Slack notification and a DRA “wait for artifacts” step. - Add dev-tools scripts for parameter validation, PR creation/merge via
gh, and DRA polling; add unit tests for the new validation and pipeline JSON generators. - Add dev-tools pytest CI step + test runner/dependency list (
test-requirements.txt,pytest.ini, Buildkite step script).
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
dev-tools/wait_version_bump_dra.py |
Polls DRA staging/snapshot JSON until versions match; gated by Buildkite meta-data. |
dev-tools/version_bump_validation.py |
Centralized patch-bump validation rules + CLI. |
dev-tools/version_bump_upload_phase2.sh |
Uploads phase-2 pipeline only when validate indicates a bump is needed. |
dev-tools/validate_version_bump_params.sh |
Validates NEW_VERSION/BRANCH against origin branch gradle.properties and sets no-op meta-data. |
dev-tools/unittest/test_wait_version_bump_dra.py |
Tests DRA wait polling/logging behavior. |
dev-tools/unittest/test_version_bump_validation.py |
Tests validation rules + CLI, with opt-in integration tests. |
dev-tools/unittest/test_job_version_bump_pipeline.py |
Tests JSON shape/env wiring for both Buildkite pipeline generators. |
dev-tools/test-requirements.txt |
Pytest + import-time deps for dev-tools tests. |
dev-tools/run_dev_tools_tests.sh |
Runs dev-tools unit tests via pytest. |
dev-tools/pytest.ini |
Pytest configuration (testpaths/pythonpath + integration marker). |
dev-tools/ensure_github_cli.sh |
Ensures gh is installed (apk or Linux tarball fallback). |
dev-tools/create_github_pull_request.sh |
Creates (and optionally merges/auto-merges) PRs via gh. |
dev-tools/bump_version.sh |
Performs the version bump on a topic branch and opens/merges a PR; sets Buildkite meta-data for downstream steps. |
.buildkite/scripts/steps/dev_tools_pytest.sh |
Buildkite step script: pip install dev-tools deps then run pytest. |
.buildkite/pipelines/send_version_bump_notification.sh |
Legacy version bump Slack notification template adjustment. |
.buildkite/pipelines/send_slack_version_bump_notification.sh |
Step-level Slack notification after PR creation (includes PR link when available). |
.buildkite/pipelines/format_and_validation.yml.sh |
Adds a “dev-tools pytest” PR check step. |
.buildkite/job-version-bump.json.py |
Phase 1 pipeline generator (validate + schedule phase 2). |
.buildkite/job-version-bump-phase2.json.py |
Phase 2 pipeline generator (bump + Slack + DRA wait). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Validate NEW_VERSION/BRANCH before git fetch, use git fetch -- for safety, fail fast when phase-2 meta-data read errors, and use Slack mrkdwn bold. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Backport of #3030 to the 8.19 branch, with #3052 (dev-tools pytest CI) bundled as a prerequisite.
Automates patch
elasticsearchVersionbumps for ml-cpp via the ml-cpp-version-bump Buildkite pipeline: validate parameters, optionally run the bump (topic branch + GitHub PR + merge), notify Slack, then wait for DRA artifacts.Commits
run_dev_tools_tests.sh,test-requirements.txt, Buildkite step). Changelog validation steps from main are omitted on 8.19 (no per-PR changelog feature on this branch).Conflict resolution
Same as the 9.3 backport (#3062):
job-version-bump.json.pyandsend_version_bump_notification.shdid not exist on 8.19; added from main.Test plan
BRANCH=8.19,WORKFLOW=patch)Made with Cursor