Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[terraform/upgrade-modules] Update all module sources to latest version #95

Merged
merged 6 commits into from
Aug 24, 2018
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
61 changes: 61 additions & 0 deletions bin/upgrade_terraform_modules.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
#!/usr/bin/env bash

function github_latest_release() {
local org=$1
local repo=$2
local ref=$(curl -sSL https://api.github.com/repos/$org/$repo/releases/latest | jq .tag_name -r)
Copy link
Sponsor Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitHub rate-limits unauthenticated calls to their api endpoint by source IP.

https://developer.github.com/changes/2012-10-14-rate-limit-changes/

I've found this kind of query works fine locally, but fails sporadically from CI. The typical workaround seems to be to create a bot user with zero access to anything, generate an access token, and then use basic auth in the curl command....

curl -sSL https://api.github.com/repos/$org/$repo/releases/latest?access_token=<the access token>

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right. I was just being lazy =)

I've added support for the tokens. Decided to use a header since that is less likely to show up in access logs. https://gist.github.com/caspyin/2288960#oauth

if [ $? -eq 0 ]; then
echo $ref
fi
}

function upgrade_modules() {
local file=$1
echo "Processing $file..."
for source in $(json2hcl -reverse < $file | jq -r '.module | .[][] | first | .source' 2>/dev/null); do
if [[ $source =~ github.com/ ]]; then
echo "[GITHUB]: $source"
if [[ $source =~ github.com/(.*?)/(.*?)\.git ]]; then
org="${BASH_REMATCH[1]}"
repo="${BASH_REMATCH[2]}"
fi
if [[ $source =~ \?ref=([0-9.]+) ]]; then
ref="${BASH_REMATCH[1]}"
fi
if [[ $source =~ \?ref=tags/([0-9.]+) ]]; then
ref="${BASH_REMATCH[1]}"
fi

if [ -z "$org" ] || [ -z "$repo" ] || [ -z "$ref" ]; then
echo " - Failed to parse module source (org: $org, repo: $repo, ref: $ref)"
else
latest_ref=$(github_latest_release "$org" "$repo")
latest_source="git::https://github.com/$org/$repo.git?ref=tags/$latest_ref"
if [ "$latest_source" == "$source" ]; then
echo " - Current: $ref"
else
echo " - Latest: $ref -> ${latest_ref}"
echo " - Source: $latest_source"
sed -i"" "s,$source,$latest_source,g" "$file"
fi
fi
else
echo "[SKIPPED]: $source"
fi
done
}

files=""
if [ $# -eq 0 ]; then
echo "Usage: $0 [all|file1.tf...fileN.tf]"
exit 1
elif [ $1 == "all" ]; then
files=$(find . -type f -name '*.tf')
else
files="$*"
fi

for file in $files; do
upgrade_modules $file
done

4 changes: 4 additions & 0 deletions modules/terraform/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,7 @@ terraform/validate:
terraform/lint:
@FAIL=`$(TERRAFORM) fmt -write=false | xargs --no-run-if-empty -n 1 printf '\t- %s\n'`; \
[ -z "$$FAIL" ] || (echo "Terraform configuration needs linting. Run '$(TERRAFORM) fmt'"; echo $$FAIL; exit 1)

## Upgrade all terraform module sources
terraform/upgrade-modules:
@$(BUILD_HARNESS_PATH)/bin/upgrade_terraform_modules.sh all