[FR] Add Versioning Processes to DR

.github/PULL_REQUEST_GUIDELINES/bug_guidelines.md

@@ -31,4 +31,5 @@ These guidelines serve as a reminder set of considerations when addressing a bug
 - [ ] Ensure that the bug fix does not break existing functionality.
 - [ ] Review the bug fix with a peer or team member for additional insights.
 - [ ] Verify that the bug fix works across all relevant environments (e.g., different OS versions).
-- [ ] Confirm that all dependencies are up-to-date and compatible with the changes.
+- [ ] Confirm that all dependencies are up-to-date and compatible with the changes.
+- [ ] Confirm that the proper version label is applied to the PR `patch`, `minor`, `major`.

.github/PULL_REQUEST_GUIDELINES/enhancement_guidelines.md

@@ -32,3 +32,4 @@ These guidelines serve as a reminder set of considerations when addressing addin
 - [ ] Review the enhancement with a peer or team member for additional insights.
 - [ ] Verify that the enhancement works across all relevant environments (e.g., different OS versions).
 - [ ] Confirm that all dependencies are up-to-date and compatible with the changes.
+- [ ] Confirm that the proper version label is applied to the PR `patch`, `minor`, `major`.

.github/PULL_REQUEST_GUIDELINES/hunt_tuning_guidelines.md

@@ -27,4 +27,4 @@ These guidelines serve as a reminder set of considerations when tuning an existi
 - [ ] Evidence of testing and valid query usage.
 - [ ] Markdown Generated: Run `python -m hunting generate-markdown` with specific parameters to ensure a markdown version of the hunting TOML files is created.
 - [ ] Index Refreshed: Run `python -m hunting refresh-index` to refresh indexes.
-- [ ] Run Unit Tests: Run `pytest tests/test_hunt_data.py` to run unit tests.
+- [ ] Run Unit Tests: Run `pytest tests/test_hunt_data.py` to run unit tests.

.github/PULL_REQUEST_GUIDELINES/schema_enhancement_guidelines.md

@@ -43,4 +43,5 @@ These guidelines serve as a reminder set of considerations when addressing addin
 - [ ] Implemented requisite downgrade functionality
 - [ ] Cross-referenced the feature with product documentation for consistency
 - [ ] Incorporated a comprehensive test rule in unit tests for full schema coverage
-- [ ] Conducted system testing, including fleet, import, and create APIs (e.g., run `make test-remote-cli`)
+- [ ] Conducted system testing, including fleet, import, and create APIs (e.g., run `make test-remote-cli`)
+- [ ] Confirm that the proper version label is applied to the PR `patch`, `minor`, `major`.

.github/PULL_REQUEST_TEMPLATE.md

@@ -36,7 +36,7 @@ from your submission, but they are here to help bring them to your attention.
 
 <!-- Delete any items that are not applicable to this PR. -->
 
-- [ ] Added a label for the type of pr: `bug`, `enhancement`, `schema`, `Rule: New`, `Rule: Deprecation`, `Rule: Tuning`, `Hunt: New`, or `Hunt: Tuning` so guidelines can be generated
+- [ ] Added a label for the type of pr: `bug`, `enhancement`, `schema`, `maintenance`, `Rule: New`, `Rule: Deprecation`, `Rule: Tuning`, `Hunt: New`, or `Hunt: Tuning` so guidelines can be generated
 - [ ] Added the `meta:rapid-merge` label if planning to merge within 24 hours
 - [ ] Secret and sensitive material has been managed correctly
 - [ ] Automated testing was updated or added to match the most common scenarios

.github/release-drafter.yml

@@ -0,0 +1,32 @@
+name-template: 'v$RESOLVED_VERSION'
+tag-template: 'v$RESOLVED_VERSION'
+categories:
+  - title: 🚀 Features
+    label: 'enhancement'
+  - title: 🐛 Bug Fixes
+    label: 'bug'
+  - title: 🛠 Internal Changes
+    labels:
+      - 'maintenance'
+      - 'schema'
+      - 'documentation'
+      - 'python'
+  - title: 🔍 Hunting Updates
+    label: 'Hunting'
+change-template: '- $TITLE (#$NUMBER) @$AUTHOR'
+exclude-labels:
+  - 'skip-changelog'
+version-resolver:
+  major:
+    labels:
+      - 'major'
+  minor:
+    labels:
+      - 'minor'
+  patch:
+    labels:
+      - 'patch'
+  default: patch
+template: |
+  ## Changes
+  $CHANGES

.github/workflows/version-code-and-release.yml

@@ -0,0 +1,86 @@
+name: Version Code Check and Draft Release
+
+on:
+  pull_request:
+    paths:
+      - 'lib/**'
+      - 'hunting/**/*.py'
+      - 'pyproject.toml'
+      - 'Makefile'
+      - 'docs/**'
+      - 'detection_rules/**'
+      - 'tests/**'
+      - '**/*.md'
+    types: [opened, reopened, synchronize]
+  push:
+    branches:
+      - main
+    paths:
+      - 'pyproject.toml'
+
+permissions:
+  contents: read
+  pull-requests: read
+
+jobs:
+  label_check:
+    if: github.event_name == 'pull_request'
+    runs-on: ubuntu-latest
+    steps:
+      - name: Ensure PR has Version Bump Label
+        uses: actions/github-script@v6
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          script: |
+            const labels = ['major', 'minor', 'patch'];
+            const prLabels = context.payload.pull_request.labels.map(label => label.name);
+            const hasVersionLabel = labels.some(label => prLabels.includes(label));
+            if (!hasVersionLabel) {
+              throw new Error("PR must have one of the following labels: major, minor, or patch.");
+            }
+
+  version_check:
+    if: github.event_name == 'pull_request'
+    needs: label_check
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Check if core pyproject.toml was updated
+        run: |
+          BASE_COMMIT="${{ github.event.pull_request.base.sha }}"
+
+          if ! git diff --name-only "$BASE_COMMIT" "$GITHUB_SHA" | grep '^pyproject.toml$'; then
+            echo "Code changes detected in core, but pyproject.toml was not updated."
+            exit 1
+          fi
+
+      - name: Check if lib pyproject.toml files were updated
+        run: |
+          BASE_COMMIT="${{ github.event.pull_request.base.sha }}"
+
+          if git diff --name-only "$BASE_COMMIT" "$GITHUB_SHA" | grep -E 'lib/kql/|lib/kibana/'; then
+            if ! git diff --name-only "$BASE_COMMIT" "$GITHUB_SHA" | grep -E 'lib/kql/pyproject.toml|lib/kibana/pyproject.toml'; then
+              echo "Changes detected in kql or kibana library, but respective pyproject.toml was not updated."
+              exit 1
+            fi
+          fi
+
+  release_drafter:
+    if: github.ref == 'refs/heads/main'
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Run Release Drafter
+        uses: release-drafter/release-drafter@v6
+        with:
+          config-name: release-drafter.yml
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

docs/versioning.md

@@ -1,4 +1,4 @@
-# Supported Versions and Releases
+# Rule Supported Versions and Releases
 
 This document provides detailed information about the different versions that are supported and released for prebuilt detection rules.
 
@@ -24,3 +24,138 @@ The following version(s) are maintained along with the current version.
 ## End of Life Policy
 
 Our policy is to support and provide public releases for `Current`, `Current-1`, `Current-2`, `Current-3` versions. We maintain and do not release `Current-4` and `Current-5` versions.
+
+
+# Code Supported Versions and Releases
+
+This outlines the versioning strategy and release process for the [detection-rules](https://github.com/elastic/detection-rules) repository, covering the core code, `kql` and `kibana` libraries, configuration files, and the `hunting` folder. The strategy follows semantic versioning to ensure clear communication of changes to users and compatibility with different Elastic Stack versions.
+
+> [!IMPORTANT]
+> This versioning process **excludes** the detection rules themselves. Detection rules are released separately and are not tied to the following process.
+
+---
+
+## Versioning Strategy
+
+### Components Covered by Versioning:
+- **Core Detection-Rules Code**: Handles logic for rule management, CLI, etc.
+- **Libraries**:
+  - **`kql`**: Manages Kibana Query Language parsing and operations.
+  - **`kibana`**: Handles integrations and API interactions with Kibana.
+- **Configuration Files**: Under the `etc/` folder that impact schema and DAC.
+- **Hunting Logic**: The `hunting/` folder, which manages hunting rules.
+
+
+### Semantic Versioning Approach:
+We will use **Semantic Versioning** with the format `MAJOR.MINOR.PATCH`:
+- **MAJOR version (`X.0.0`)**: For backward-incompatible changes.
+- **MINOR version (`0.Y.0`)**: For backward-compatible new features.
+- **PATCH version (`0.0.Z`)**: For backward-compatible bug fixes or small improvements.
+
+> [!NOTE]
+> The GitHub labels `patch`, `minor`, or `major` will be used in PRs to indicate the type of change being made.
+
+---
+
+## Versioning Guidelines
+
+### Patch Version (`0.0.Z`):
+Increment the patch version when making bug fixes, performance improvements, or small enhancements that do not break backward compatibility. Open a PR to ensure the proper `pyproject.toml` files and any other `version` related files are bumped.
+
+<details><summary>Expand for Examples</summary>
+<p>
+
+**Examples**:
+- **Kibana Library**:
+  - Minor fixes to API calls to ensure correct data retrieval.
+  - Updates to the `kibana` lib without adding new features.
+- **KQL Library**:
+  - Small bug fixes in the query parsing logic.
+  - Optimizations that don't alter functionality.
+- **Core Detection-Rules Code**:
+  - Fixes for CLI bugs or performance tweaks.
+  - Minor enhancements to rule management that don’t require users to change workflows.
+- **Hunting Folder**:
+  - Bug fixes in hunting rules logic.
+  - Small performance tweaks for the hunting rule management.
+- **Docs Folder**:
+  - Updates to documentation.
+
+</p>
+</details>
+
+---
+
+### Minor Version (`0.Y.0`):
+Increment the minor version when adding backward-compatible new features, enhancements, or functionality.
+
+<details><summary>Expand for Examples</summary>
+<p>
+
+**Examples**:
+- **Kibana Library**:
+  - Adding a new API endpoint to interact with Elastic Kibana X.Y while maintaining backward compatibility with older versions.
+- **KQL Library**:
+  - Adding new query parsing functionality that is backward-compatible with previous Elastic Stack versions.
+- **Core Detection-Rules Code**:
+  - New CLI commands or functionality for managing detection rules.
+  - New optional fields in rule schemas that have minimum compatibility requirements. (e.g adding `alert_suppression` with `min_compat=8.14`).
+- **Hunting Folder**:
+  - Adding new hunting rule management features that are optional and backward-compatible.
+  - Enhancements in generating hunting rule markdown or CLI features.
+
+</p>
+</details>
+
+> [!NOTE]
+> When bumping this version, the patch version should be reset to `0` and the major version should remain the same.
+
+---
+
+### Major Version (`X.0.0`):
+Increment the major version when introducing backward-incompatible changes that require users to update workflows, Elastic Stack versions, or rule management strategies.
+
+<details><summary>Expand for Examples</summary>
+<p>
+
+**Examples**:
+- **Kibana Library**:
+  - Replacing or removing an existing API endpoint that forces users to upgrade to Elastic X.Y
+- **KQL Library**:
+  - Structural changes to query parsing logic that break compatibility with previous Elastic Stack versions.
+- **Core Detection-Rules Code**:
+  - Breaking changes to rule schema definitions or CLI workflows that require user updates.
+  - Forcing users to migrate to a newer Elastic Stack version due to changes in core code or schema compatibility.
+- **Hunting Folder**:
+  - Major refactors of the hunting logic that break existing workflows.
+  - Changes to how hunting rules are defined or managed, requiring users to adjust configurations.
+
+</p>
+</details>
+
+> [!NOTE]
+> When bumping this version, the minor version and patch version should be reset to `0`.
+
+---
+
+## Tagging Process
+
+Each release will be tagged using the following format:
+- **Tag Format**: `vX.Y.Z` (e.g., `v1.2.0`).
+- **Single Tag for Combined Releases**: If there are changes to the core detection-rules code or libraries (`kql`, `kibana`), they will be tagged together as a single release with the core detection-rules versioning.
+- **Hunting Folder**: Changes to the hunting logic will be included in the combined release.
+
+> [!CAUTION]
+> When a version is bumped in a lib, we need to also bump the core `pyproject.toml` file *(e.g A version bump in `kql` will also require a similar version bump in the core detection-rules versioning)*.
+---
+
+## When to Trigger a GitHub Release
+
+A draft release will be triggered in the following cases:
+- **New Feature or Bug Fix**: Once a feature or bug fix is merged into `main`, a version bump is made according to the semantic versioning rules.
+- **Version Bump**: After the version bump, a GitHub release will be created using **release-drafter** CI workflow to automate draft release generation.
+
+As pull requests are merged, a draft release is kept up-to-date listing the changes, ready to publish quarterly.
+
+> [!IMPORTANT]
+> Proper PR labels need to be added for this to properly be labeled and added to the draft.

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "detection_rules"
-version = "0.1.0"
+version = "0.2.0"
 description = "Detection Rules is the home for rules used by Elastic Security. This repository is used for the development, maintenance, testing, validation, and release of rules for Elastic Security’s Detection Engine."
 readme = "README.md"
 requires-python = ">=3.12"