Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Build links to Docker images for Chart.yaml #6963

Merged
merged 18 commits into from
Feb 9, 2023
Merged

fix: Build links to Docker images for Chart.yaml #6963

Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
f802616
style: Split long lines, follow .editorconfig
opello Feb 3, 2023
a6b1669
fix: Clarify why some sources are being excluded
opello Feb 3, 2023
0a78810
fix: Comment the image-to-URL code
opello Feb 4, 2023
02a9013
refactor: Use case instead of if-ladder
opello Feb 4, 2023
d0c025f
fix: Remove dead code
opello Feb 4, 2023
52a4597
fix: tccr.io image links
opello Feb 4, 2023
96a5d53
fix: lscr.io image links
opello Feb 4, 2023
9b3835e
fix: gcr.io image links
opello Feb 4, 2023
17482ce
feat: Do not add sources if no prefix is created
opello Feb 4, 2023
a09e27f
fix: Disable azurecr.io image links
opello Feb 4, 2023
b65e97b
feat: Disable mcr.microsoft.com image links
opello Feb 4, 2023
f887477
fix: public.ecr.aws image links
opello Feb 4, 2023
b083be2
fix: Disable ocir.io image links
opello Feb 4, 2023
3401173
refactor: Add Docker Hub hosted image links
opello Feb 4, 2023
761b572
feat: ghcr.io image links
opello Feb 4, 2023
f8ef3ff
feat: quay.io image links
opello Feb 4, 2023
4676277
feat: Do not generate likely-bad links
opello Feb 4, 2023
2226985
refactor: Sort cases
opello Feb 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
190 changes: 124 additions & 66 deletions .github/workflows/daily.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,74 +68,132 @@ jobs:
# Designed to ensure the appversion in Chart.yaml is in sync with the primary Chart tag if found
# Also makes sure that home link is pointing to the correct url
sync_tag() {
local chart="$1"
local chartname="$2"
local train="$3"
echo "Attempting to sync primary tag with appversion for: ${chartname}"
local tag="$(cat ${chart}/values.yaml | grep '^ tag: ' | awk -F" " '{ print $2 }' | head -1)"
tag="${tag%%@*}"
tag="${tag:-auto}"
tag=$(echo $tag | sed "s/release-//g")
tag=$(echo $tag | sed "s/release_//g")
tag=$(echo $tag | sed "s/version-//g")
tag=$(echo $tag | sed "s/version_//g")
tag="${tag#*V.}"
tag="${tag#*v-}"
tag="${tag#*v}"
tag="${tag%-*}"
tag="${tag:0:10}"
tag="${tag%-}"
tag="${tag%_}"
tag="${tag%.}"
echo "Updating tag of ${chartname} to ${tag}..."
sed -i -e "s|appVersion: .*|appVersion: \"${tag}\"|" "${chart}/Chart.yaml"
echo "Updating icon of ${chartname}..."
sed -i -e "s|icon: .*|icon: https:\/\/truecharts.org\/img\/hotlink-ok\/chart-icons\/${chartname}.png|" "${chart}/Chart.yaml"
echo "Updating home of ${chartname}..."
sed -i -e "s|home: .*|home: https:\/\/truecharts.org\/charts\/${train}\/${chartname}|" "${chart}/Chart.yaml"
echo "Attempting to update sources of ${chartname}..."
echo "Using go-yq verion: <$(go-yq -V)>"
# Get all sources (except truecharts)
curr_sources=$(go-yq '.sources[] | select(. != "https://github.com/truecharts*" and . != "https://ghcr*" and . != "docker.io*" and . != "https://docker.io*" and . != "https://hub.docker*" and . != "https://fleet.*" and . != "https://github.com/truecharts/containers/tree/master/mirror/*" and . != "https://public.ecr.aws*" and . != "https://ocir.io*" and . != "https://gcr*" and . != "https://azurecr*" and . != "https://quay*" and . != "https://lscr*" and . != "https://github.com/truecharts/containers*" and . == "http*")' "${chart}/Chart.yaml")
# Empty sources list in-place
go-yq -i 'del(.sources.[])' "${chart}/Chart.yaml"
# Add truechart source
tcsource="https://github.com/truecharts/charts/tree/master/charts/$train/$chartname" go-yq -i '.sources += env(tcsource)' "${chart}/Chart.yaml"
container=$(cat website/docs/charts/description_list.md | grep "\[${chartname}\]" | cut -f3 -d '|' | grep -v 'Not Found' || echo "")
if [ ! -z "$container" ]; then
prefix="https://hub.docker.com/"
container=${container#https://}
if echo ${container} | grep 'tccr'; then
container=${container#tccr.io}
container="https://github.com/truecharts/containers/tree/master/mirror"
prefix=""
elif echo ${container} | grep 'lscr'; then
container="https://fleet.linuxserver.io/"
prefix=""
elif echo ${container} | grep 'gcr'; then
prefix="https://"
elif echo ${container} | grep 'azurecr'; then
prefix="https://"
elif echo ${container} | grep 'public.ecr.aws'; then
prefix="https://"
elif echo ${container} | grep 'ocir.io'; then
prefix="https://"
elif echo ${container} | grep 'hub.docker'; then
prefix="https://"
elif echo ${container} | grep 'docker.io'; then
container=${container#docker.io}
prefix="https://hub.docker.com"
elif echo ${container} | grep 'hub.docker'; then
prefix="https://"
fi
local chart="$1"
local chartname="$2"
local train="$3"
echo "Attempting to sync primary tag with appversion for: ${chartname}"
local tag="$(cat ${chart}/values.yaml | grep '^ tag: ' | awk -F" " '{ print $2 }' | head -1)"
tag="${tag%%@*}"
tag="${tag:-auto}"
tag=$(echo $tag | sed "s/release-//g")
tag=$(echo $tag | sed "s/release_//g")
tag=$(echo $tag | sed "s/version-//g")
tag=$(echo $tag | sed "s/version_//g")
tag="${tag#*V.}"
tag="${tag#*v-}"
tag="${tag#*v}"
tag="${tag%-*}"
tag="${tag:0:10}"
tag="${tag%-}"
tag="${tag%_}"
tag="${tag%.}"
echo "Updating tag of ${chartname} to ${tag}..."
sed -i -e "s|appVersion: .*|appVersion: \"${tag}\"|" "${chart}/Chart.yaml"
echo "Updating icon of ${chartname}..."
sed -i -e "s|icon: .*|icon: https:\/\/truecharts.org\/img\/hotlink-ok\/chart-icons\/${chartname}.png|" "${chart}/Chart.yaml"
echo "Updating home of ${chartname}..."
sed -i -e "s|home: .*|home: https:\/\/truecharts.org\/charts\/${train}\/${chartname}|" "${chart}/Chart.yaml"
echo "Attempting to update sources of ${chartname}..."
echo "Using go-yq verion: <$(go-yq -V)>"
# Get current sources, exluding those that may have been added automatically.
curr_sources=$(
go-yq '
.sources[] |
select(
. != "https://github.com/truecharts*" and
. != "https://ghcr*" and
. != "docker.io*" and
. != "https://docker.io*" and
. != "https://hub.docker*" and
. != "https://fleet.*" and
. != "https://github.com/truecharts/containers/tree/master/mirror/*" and
. != "https://public.ecr.aws*" and
. != "https://ocir.io*" and
. != "https://gcr*" and
. != "https://azurecr*" and
. != "https://quay*" and
. != "https://lscr*" and
. != "https://github.com/truecharts/containers*" and
. == "http*"
)
' \
"${chart}/Chart.yaml"
)
Comment on lines +99 to +121
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may be controversial ...

# Empty sources list in-place
go-yq -i 'del(.sources.[])' "${chart}/Chart.yaml"
# Add truechart source
tcsource="https://github.com/truecharts/charts/tree/master/charts/$train/$chartname" go-yq -i '.sources += env(tcsource)' "${chart}/Chart.yaml"
# Get the container image name that was parsed out of the Dockerfile for the website.
container=$(cat website/docs/charts/description_list.md | grep "\[${chartname}\]" | cut -f3 -d '|' | grep -v 'Not Found' || echo "")
# Convert the container image name to a URL.
if [ ! -z "$container" ]; then
prefix=""
case "$container" in
lscr.io/linuxserver/*)
prefix="https://fleet.linuxserver.io/image?name="
container=${container#lscr.io/}
;;
tccr.io/truecharts/*)
prefix="https://github.com/truecharts/containers/tree/master/mirror"
container=${container#tccr.io/truecharts/}
;;
mcr.microsoft.com/*)
prefix=""
;;
public.ecr.aws/*)
prefix="https://gallery.ecr.aws/"
container=${container#public.ecr.aws/}
;;
ghcr.io/*)
prefix="https://"
;;
quay.io/*)
prefix="https://"
;;
gcr.io/*)
prefix="https://"
;;
*.azurecr.io/*)
prefix=""
;;
*.ocir.io/*)
prefix=""
;;
# There have been a number of domains used for the Docker Hub registry over the years.
# NOTE: This is also the default case!
docker.io/*|index.docker.io/*|registry-1.docker.io/*|registry.hub.docker.com/*|*)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default case is a little sneakily hidden here. I don't think it's awesome but it minimizes repetition without adding a new function.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm okey with that tbh.

prefix="https://hub.docker.com/r/"
container=${container#docker.io/}
container=${container#index.docker.io/}
container=${container#registry-1.docker.io/}
container=${container#registry.hub.docker.com/}
# If the image name does not contain a slash it is a Docker Official Image.
if [ "$container" == "${container////}" ]; then
prefix="https://hub.docker.com/_/"
# If the user name is library it is a Docker Official Image.
elif [ "${container%%/*}" == "library" ]; then
prefix="https://hub.docker.com/_/"
container=${container#library/}
fi
# Avoid creating a bad link since an unsupported registry may have been used.
slashes=${container//[^\/]/}
# Bail out if the image name has more than 1 slash.
if [ ${#slashes} -gt 1 ]; then
prefix=""
echo "WARNING: Not assuming '$container' is a Docker Hub image"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I introduced printing this warning but didn't redirect it to stderr as I wasn't sure of how such things should be handled, but it seemed appropriate feedback to generate. But maybe it belongs somewhere else, like where the Dockerfile gets added to the containers repository instead?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine to warn about this here :)

fi
;;
esac
if [ -n "${prefix}" ]; then
container="${prefix}${container}" go-yq -i '.sources += env(container) | .sources |= unique' "${chart}/Chart.yaml"
fi
# Add the rest of the sources
while IFS= read -r line; do
src="$line" go-yq -i '.sources += env(src)' "${chart}/Chart.yaml" || echo "src set error"
done <<< "$curr_sources"
echo "Sources of ${chartname} updated!"
}
fi
# Add the rest of the sources
while IFS= read -r line; do
src="$line" go-yq -i '.sources += env(src)' "${chart}/Chart.yaml" || echo "src set error"
done <<< "$curr_sources"
echo "Sources of ${chartname} updated!"
}
export -f sync_tag

for train in enterprise stable incubator dependency; do
Expand Down