Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Automates changing Terraform supported versions in provider documentation #2058

Merged
merged 11 commits into from
Mar 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/terraform-compatibility-matrix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
output=$(./scripts/get-terraform-supported-versions.sh)
output=$(./scripts/get-terraform-supported-versions.sh "false")
quoted_output=$(echo "${output}" | jq -c .)
echo "supported_versions=${quoted_output}" >> "${GITHUB_OUTPUT}"
outputs:
Expand Down
30 changes: 30 additions & 0 deletions .github/workflows/update_tf_compatibility_matrix.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Update Terraform Compatibility Matrix documentation
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as we're already in the TF repo, I would remove it from the file and title

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't fully agree, just using "compatibility matrix" could refer to terraform, terraform plugin framework, terraform provider, etc and cause confusion.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @oarbusi I don't think it's redundant


# Checks if any changes are required to be made to our documentation for supported Terraform versions. Runs daily and can be triggered manually.
on:
schedule:
- cron: "0 7 * * *" # Everyday at 7:00 AM
workflow_dispatch:

jobs:
update-tf-compatibility-matrix:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@9bb56186c3b09b4f86b1c65136769dd318469633
- name: Update files
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: make update-tf-compatibility-matrix
- name: Verify Changed files
uses: tj-actions/verify-changed-files@843c0b95f87cd81a2efe729380c6d1f11fb3ea12
id: verify-changed-files
- name: Create PR
uses: peter-evans/create-pull-request@70a41aba780001da0a30141984ae2a0c95d8704e
if: steps.verify-changed-files.outputs.files_changed == 'true'
with:
title: "doc: Updates Terraform Compatibility Matrix documentation"
commit-message: "doc: Updates Terraform Compatibility Matrix documentation"
delete-branch: true
branch: terraform-compatibility-matrix-update
body: Automatic updates for Terraform Compatibility Matrix documentation
4 changes: 4 additions & 0 deletions GNUmakefile
Original file line number Diff line number Diff line change
Expand Up @@ -134,3 +134,7 @@ scaffold-schemas:
.PHONY: generate-doc
generate-doc: ## Generate the resource documentation via tfplugindocs
./scripts/generate-doc.sh ${resource_name}

.PHONY: update-tf-compatibility-matrix
update-tf-compatibility-matrix: ## Update Terraform Compatibility Matrix documentation
./scripts/update-tf-compatibility-matrix.sh
114 changes: 72 additions & 42 deletions scripts/get-terraform-supported-versions.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,26 +14,35 @@
# See the License for the specific language governing permissions and
# limitations under the License.
set -euo pipefail

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same, I would remove TF from filename

usage() {
echo "Usage: $0 [true|false]"
echo " true: Returns details of supported Terraform versions"
echo " false: Returns only supported Terraform version numbers"
exit 1
}

get_last_day_of_month() {
last_day_of_month=0
case $1 in
01|03|05|07|08|10|12)
last_day_of_month=31 ;;
04|06|09|11)
last_day_of_month=30 ;;
02)
# February: check if it's a leap year
if (( year % 4 == 0 && ( year % 100 != 0 || year % 400 == 0 ) )); then
last_day_of_month=29
else
last_day_of_month=28
fi
;;
esac
01 | 03 | 05 | 07 | 08 | 10 | 12)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

formatted

last_day_of_month=31
;;
04 | 06 | 09 | 11)
last_day_of_month=30
;;
02)
# February: check if it's a leap year
if ((year % 4 == 0 && (year % 100 != 0 || year % 400 == 0))); then
last_day_of_month=29
else
last_day_of_month=28
fi
;;
esac
echo $last_day_of_month
}


add_end_support_date() {
new_json_list=$(echo "$1" | jq -c '.[]' | while IFS= read -r obj; do
input_date=$(echo "$obj" | jq -r '.published_at')
Expand All @@ -55,37 +64,58 @@ add_end_support_date() {
echo "$new_json_list"
}

page=1
api_version="2022-11-28"

while true; do
response=$(curl -s \
--request GET \
--url "https://api.github.com/repos/hashicorp/terraform/releases?per_page=100&page=$page" \
--header "Authorization: Bearer $GITHUB_TOKEN" \
--header "X-GitHub-Api-Version: $api_version")
if [[ "$(printf '%s\n' "$response" | jq -e 'length == 0')" == "true" ]]; then
break
else
versions=$(echo "$response" | jq -r '.[] | {version: .tag_name, published_at: .published_at}')
filtered_versions_json=$(printf '%s\n' "${versions}" | jq -s '.')
updated_date_versions=$(add_end_support_date "$filtered_versions_json")
filtered_out_deprecated_versions=$(echo "$updated_date_versions" | jq 'map(select((.version | test("alpha|beta|rc"; "i") | not) and ((.end_support_date | fromdate) >= now))) | map(select(.version | endswith(".0")))')

if [ -z ${json_array+x} ]; then
# If json_array is empty, assign filtered_versions directly
json_array=$(jq -n --argjson filtered_versions "${filtered_out_deprecated_versions}" '$filtered_versions')
get_terraform_supported_versions_details() {
page=1
api_version="2022-11-28"

while true; do
response=$(curl -s \
--request GET \
--url "https://api.github.com/repos/hashicorp/terraform/releases?per_page=100&page=$page" \
--header "Authorization: Bearer $GITHUB_TOKEN" \
--header "X-GitHub-Api-Version: $api_version")
if [[ "$(printf '%s\n' "$response" | jq -e 'length == 0')" == "true" ]]; then
break
else
# If json_array is not empty, append filtered_versions to it
json_array=$(echo "$json_array" | jq --argjson filtered_versions "${filtered_out_deprecated_versions}" '. + $filtered_versions')
versions=$(echo "$response" | jq -r '.[] | {version: .tag_name, published_at: .published_at}')
filtered_versions_json=$(printf '%s\n' "${versions}" | jq -s '.')
updated_date_versions=$(add_end_support_date "$filtered_versions_json")
filtered_out_deprecated_versions=$(echo "$updated_date_versions" | jq 'map(select((.version | test("alpha|beta|rc"; "i") | not) and ((.end_support_date | fromdate) >= now))) | map(select(.version | endswith(".0")))')
if [ -z ${json_array+x} ]; then
# If json_array is empty, assign filtered_versions directly
json_array=$(jq -n --argjson filtered_versions "${filtered_out_deprecated_versions}" '$filtered_versions')
else
# If json_array is not empty, append filtered_versions to it
json_array=$(echo "$json_array" | jq --argjson filtered_versions "${filtered_out_deprecated_versions}" '. + $filtered_versions')
fi

((page++))
fi
done

echo "$json_array"
}

get_terraform_supported_versions() {
json_array=$(get_terraform_supported_versions_details)

((page++))
fi
done
versions_array=$(printf '%s\n' "${json_array}" | jq -r '.[] | .version')

versions_array=$(printf '%s\n' "${json_array}" | jq -r '.[] | .version')
formatted_output=$(echo "$versions_array" | awk 'BEGIN { ORS = "" } {gsub(/^v/,"",$1); gsub(/\.0$/,".x",$1); printf("%s\"%s\"", (NR==1?"":", "), $1)}' | sed 's/,"/," /g')

echo "[$formatted_output]" | jq -c .
}

formatted_output=$(echo "$versions_array" | awk 'BEGIN { ORS = "" } {gsub(/^v/,"",$1); gsub(/\.0$/,".x",$1); printf("%s\"%s\"", (NR==1?"":", "), $1)}' | sed 's/,"/," /g')
if [ $# -ne 1 ]; then
usage
fi

echo "[$formatted_output]" | jq -c .
get_details=$1
if [ "$get_details" = "true" ]; then
get_terraform_supported_versions_details
elif [ "$get_details" = "false" ]; then
get_terraform_supported_versions
else
echo "Invalid parameter."
usage
fi
102 changes: 102 additions & 0 deletions scripts/update-tf-compatibility-matrix.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
#!/usr/bin/env bash
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another big shell script 😅 Did you consider using python?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I essentially re-used the existing script

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or Go as @AgustinBettati did for the one in CFN listing CFN versions in regions ;-)
Go templates are nice.

a great topic you may want to bring to next tech meeting


# Copyright 2024 MongoDB Inc
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -euo pipefail

input_array=$(./scripts/get-terraform-supported-versions.sh "true")

indexFile="website/docs/index.html.markdown"

transform_array() {
local arr="$1"
local updated_arr="["
local isFirstElement=true

for ((i = 0; i < $(jq length <<<"$arr"); i++)); do
version=$(jq -r ".[$i].version" <<<"$arr" | sed 's/^v//;s/\.0/.x/')
published_at=$(jq -r ".[$i].published_at" <<<"$arr" | cut -dT -f1)
end_support_date=$(jq -r ".[$i].end_support_date" <<<"$arr" | cut -dT -f1)

if [ "$isFirstElement" = false ]; then
updated_arr+=","
fi
updated_arr+="{\"version\": \"$version\", \"published_at\": \"$published_at\", \"end_support_date\": \"$end_support_date\"}"
isFirstElement=false
done

updated_arr+="]"

echo "$updated_arr"
}

generate_matrix_markup() {
local output_array="$1"

table="| HashiCorp Terraform Release | HashiCorp Terraform Release Date | HashiCorp Terraform Full Support End Date | MongoDB Atlas Support End Date |\n"
table+="|:-------:|:------------:|:------------:|:------------:|\n"

for ((i = 0; i < $(jq length <<<"$output_array"); i++)); do
version=$(jq -r ".[$i].version" <<<"$output_array")
published_at=$(jq -r ".[$i].published_at" <<<"$output_array")
end_support_date=$(jq -r ".[$i].end_support_date" <<<"$output_array")

table+="| $version | $published_at | $end_support_date | $end_support_date |\n"
done

echo -e "$table"
}

update_index_markdown_file() {
local markup="$1"
local tempFile="$indexFile.tmp"
local placeholderStart="<!-- MATRIX_PLACEHOLDER_START -->"
local placeholderEnd="<!-- MATRIX_PLACEHOLDER_END -->"
local inPlaceholder=0

# Ensure the temp file is empty or does not exist
: > "$tempFile"

while IFS= read -r line || [[ -n "$line" ]]; do
if [[ "$line" == "$placeholderStart" ]]; then
inPlaceholder=1
echo "$line" >>"$tempFile"
echo "$markup" >>"$tempFile"
continue
fi

if [[ "$line" == "$placeholderEnd" ]]; then
inPlaceholder=0
echo "$line" >>"$tempFile"
continue
fi

if [[ $inPlaceholder -eq 0 ]]; then
echo "$line" >>"$tempFile"
fi
done <"$indexFile"

mv "$tempFile" "$indexFile"

echo "Updated Terraform version compatibility matrix in $indexFile"
}

# Transform the array data
updated_array=$(transform_array "$input_array")

# Generate markup for the compatibility matrix
markup=$(generate_matrix_markup "$updated_array")

# Update the Markdown file with the generated markup
update_index_markdown_file "$markup"
6 changes: 4 additions & 2 deletions website/docs/index.html.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,9 @@ In addition to [generic `provider` arguments](https://www.terraform.io/docs/conf
For more information on configuring and managing programmatic API Keys see the [MongoDB Atlas Documentation](https://docs.atlas.mongodb.com/tutorial/manage-programmatic-access/index.html).

## [HashiCorp Terraform Version](https://www.terraform.io/downloads.html) Compatibility Matrix


<!-- DO NOT remove below placeholder comments as this table is auto-generated -->
<!-- MATRIX_PLACEHOLDER_START -->
| HashiCorp Terraform Release | HashiCorp Terraform Release Date | HashiCorp Terraform Full Support End Date | MongoDB Atlas Support End Date |
|:-------:|:------------:|:------------:|:------------:|
| 1.7.x | 2024-01-17 | 2026-01-31 | 2026-01-31 |
Expand All @@ -189,7 +191,7 @@ For more information on configuring and managing programmatic API Keys see the [
| 1.4.x | 2023-03-08 | 2025-03-31 | 2025-03-31 |
| 1.3.x | 2022-09-21 | 2024-09-30 | 2024-09-30 |
| 1.2.x | 2022-05-18 | 2024-05-31 | 2024-05-31 |

<!-- MATRIX_PLACEHOLDER_END -->
Copy link
Member

@lantoli lantoli Mar 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a nice test would be to have this empty in the PR and see how tomorrow the PR is generated to create the table

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried running the job manually in the fork and PR was created maastha#1
WDYT?

For the safety of our users, we require only consuming versions of HashiCorp Terraform that are currently receiving Security / Maintenance Updates. For more details see [Support Period and End-of-Life (EOL) Policy](https://support.hashicorp.com/hc/en-us/articles/360021185113-Support-Period-and-End-of-Life-EOL-Policy).

HashiCorp Terraform versions that are not listed on this table are no longer supported by MongoDB Atlas. For latest HashiCorp Terraform versions see [here](https://endoflife.date/terraform ).
Expand Down