Skip to content
This repository has been archived by the owner on Apr 7, 2024. It is now read-only.

Add org-wide maintenance workflow #12

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

ferferga
Copy link
Contributor

@ferferga ferferga commented Sep 8, 2021

Adds a workflow that performs the following actions:

Job 1

  • Remove all the workflow runs from the specified repos except those that are successful or failed (conclusion of workflows that will be removed: cancelled, skipped, startup_failure... See all the possible conclusions that the API has here)

  • After that, remove all the failed or successful workflows that doesn't have logs or artifacts anymore.

I added all the repos on our org that use GitHub Actions as of today (08/09/2021), as this is unlikely to impact the workflow of somebody. However, any maintainer/team can opt out by commenting on this Pull Request before it's merged (or by removing the repo from the matrix used in the workflow at any time)

Job 2

  • Remove dangling images from GHCR: Contrary to DockerHub, GHCR doesn't remove untagged images, so they start to pile up. There's no need for keeping those images and, while GitHub doesn't impose any limits in the storage space we can use, this is neither responsible or environmentally friendly ☺.

See how many dangling images the jellyfin-vue container has (more than 5000), and it has been using GHCR for a relatively short period of time.

Needed tokens

This PR needs that the following org-wide secrets are made available in this repo:

  • JF_BOT_TOKEN: For obvious reasons. Needs workflow, write/read:packages, delete:packages, read:org

Copy link

@h1dden-da3m0n h1dden-da3m0n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments but otherwise LGTM once cleared

.github/dependabot.yml Outdated Show resolved Hide resolved
.github/workflows/maintenance.yml Show resolved Hide resolved
while read -r line; do
id="$line"
artifact_count=$(gh api /repos/${repo}/actions/runs/${id}/artifacts | jq -r '.total_count')
if [ "${artifact_count}" = "0" ]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume the artifacts count is > 0 if there are logs but 0 once they where cleaned up? (wasn't able to confirm that with the docs)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, artifacts != logs. Artifacts are the binaries produced by workflows and uploaded using this action. More info here.

Right now we don't use artifacts anywhere but in JMP. See this workflow as an example. Artifacts are located at the bottom. Logs are the output of the job

Artifacts also expire, but might have a different expiration date than logs. That's why we need to set a preference order on what's more important, logs or artifacts.

For my script's logic, I choose that artifacts > logs. I assume that maintainers/teams that have configured the artifact retention period to be larger than the retention for logs have their reasons to do so, so I'm not allowed to remove workflows runs that might have valuable data.

How the script works:

As I mentioned, for each workflow, the script checks if there are artifacts:

  • IF TRUE (total_count != 0): We continue the loop to the next one and ignore if the workflow run has logs or not because artifacts are more relevant.
  • IF FALSE (total_count == 0): We call the API endpoint to download the logs (there is no API endpoint for querying for its existence, so I must perform the download straightaway). If the logs doesn't exist, GitHub will reply with HTTP 401 Gone, so we can remove the workflow run without further ado.

gh exits with code 1 when it receives the 401, so we can use the standard || in bash to run the deletion command. This happens in this line:

gh api --silent /repos/${repo}/actions/runs/${id}/logs || \ 
				echo -n | gh api --method DELETE /repos/${repo}/actions/runs/${id} --input - && \
				echo "Workflow run without logs and artifacts with ID $id deleted successfully!"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow I must have been blind not seeing that gh api --silent /repos/${repo}/actions/runs/${id}/logs, all good then xD

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants