-
Notifications
You must be signed in to change notification settings - Fork 697
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to remove uploaded artifact? #5
Comments
ping... |
@madhurig for artifact questions |
Currently there is no experience to remove uploaded artifacts. Is there a scenario you have that you would need this feature for or are you looking to remove an artifact accidentally uploaded? |
We expect to use GitHub Action and artifacts to provide 'nightly builds' without having to deal with GitHub Releases. I.e., we currently build a tarball and run tests with it, as a regular CI job. We expect to reuse the same tarball, instead of triggering all the release procedure (which is meant for tagged commits/releases). In the situations when a tarball passes the tests but it is not valid because of any other reason, we want to remove the artifacts. This would allow us to tell users: you can pick the latest successful (green) run which contains artifacts, and ensure that the first file they download will work. BTW, can you please have a look at #3? |
@madhurig I was using artifacts to pass docker images and some gcloud configs between jobs so that I don't have to re-auth with the gcloud sdk in different jobs. It might not be the most optimised way but I was exploring that idea a few weeks ago. It will be good if we can remove these artifacts at the end of the jobs to clean up. |
@williamli - Thanks for elaborating. We will evaluate this feature. |
We need this feature because there is a 1GB limits on artifacts and we do not need the older ones. |
Is there some API to remove uploaded artifacts instead? |
As far as I can tell there’s not only no API, but no manual removal route either. This is because, as noted above, they’d not been evaluated the needed user experience. For instance, while they could release an api, would an expiration policy work better? Or maybe something else? Perhaps a totally different tool is needed, eg a cleaner way to pass messages or files from one job to another. ATM my use case is that of inter job communications. Later I might use artifacts to hold build or test results for review. |
Thanx @kf6kjg I mean, not having a way to remove artifacts and at the same time billing for over-filled storage is a contradiction to me. Btw my usecase: we integrated cypress with videos stored as artifacts - there is no need to have them for longer than few days. |
Hi all, we're working on an API for deleting an artifact. Thanks for sharing your use cases ❤️. I don't have a timeline yet that I can share on when it will be available. |
There is a workaround, you could create an empty directory and upload a new artifact of the same name with path of the empty directory.
Example: https://github.com/rtoken-project/rdai/blob/master/.github/workflows/cd.yml Update: it seems github changed some logic, the workround is no longer working!! |
Unless there's special server code for that case, a zip file of an empty directory still occupies ~80kB of space. Certainly less than the zip it replaced, but continual growth is still growth even if you manage to reduce the rate. |
I could not manage to make @hellwolf 's workaround to work: After uploading the empty folder with the same name as an artifact I could download the original artifact from action's UI, and the size displayed matched with the original file size as if nothing changed. When I added an empty file to Even though I could not access the artifact in the last case, I'm not sure it's actually replaced with the empty file behind the scenes. |
@dodie yes, it seems github changed some logic, the workround is no longer working!! |
Because of this perhaps? https://github.blog/changelog/2020-01-14-github-actions-changes-to-artifact-download-experience/ |
@gimenete not sure if this is intentional or not but there's no confirmation dialog on mobile like there is on desktop. |
@dodie @hellwolf @dentarg The workaround should no longer work, we recently made some backend changes to our artifacts service. The recent changes to the download experience were part of a broader backend update but not directly responsible for the workaroud no longer working. Recommend to use the new UI now 😀 That will completely delete the artifact and there won't be any empty directories. I've updated the |
is it possible to remove the artifacts from an action / api? |
|
And there was joyful singing! Now to get that incorporated into the actions that need it! :D |
Any news on this? Would love to see |
That's a joke, right? The scenario is that 30 well paid developers are suddenly and completely blocked because artifacts are unnecessarily persisted beyond the workflow run. |
per actions/upload-artifact#5 there is no way to automatically remove them after this job completes
It's almost 12 hours since I deleted artifacts from all previous workflow run and I still get this error: "Create Artifact Container failed: Artifact storage quota has been hit. Unable to upload any new artifacts". This is frustrating our agile process. And I have removed spending limit for about an hour too, still can't run my workflow successfully... :( |
@princemokut How did you solve your problem? I'm hitting it now, 6 months later and can't figure out how to increase my quota (and I deleted my old artifacts). I'm blocked! Edit it looks like its a caching issue and eventually it "fixes itself". Ugh.. |
There does not seem to be an easy way to delete artifacts of the existing run (see details below). Intsead, this patch truncates them so at least we do not waste space. I tried the following: - name: Delete artifacts uses: actions/github-script@v3 with: github-token: ${{ github.token }} script: | const {GITHUB_RUN_ID, GITHUB_REPOSITORY} = process.env const artifactsURL = `/repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/artifacts` console.log(`Deleting artifacs from ${artifactsURL}`) const response = await github.request(`GET ${artifactsURL}`) console.log(response.data) for (artifact of response.data.artifacts) { console.log(`Deleting artifact with name=${artifact.name}, id=${artifact.id}`) const deleteResponse = await github.request(`DELETE /repos/${repo}/actions/artifacts/${artifact.id}`) } But it did not work, because the GITHUB_RUN_ID does not seem to be available over the REST API until after the action is finished. The logs from my action printed: Deleting artifacs from /repos/cilium/tetragon/actions/runs/2817658181/artifacts ... { total_count: 0, artifacts: [] } But if I curl the above URL after the action, I get: { "total_count": 1, "artifacts": [ { "id": 322874447, "node_id": "MDg6QXJ0aWZhY3QzMjI4NzQ0NDc=", "name": "tetragon-build", "size_in_bytes": 6, "url": "https://api.github.com/repos/cilium/tetragon/actions/artifacts/322874447", "archive_download_url": "https://api.github.com/repos/cilium/tetragon/actions/artifacts/322874447/zip", "expired": false, "created_at": "2022-08-08T11:50:54Z", "updated_at": "2022-08-08T11:50:54Z", "expires_at": "2022-08-13T11:49:30Z", "workflow_run": { "id": 2817658181, "repository_id": 473137633, "head_repository_id": 473137633, "head_branch": "pr/kkourt/lvhtest", "head_sha": "ea22990af2283c38aac92f7e6febacabf3786280" } } ] } For (even) more context, see: actions/upload-artifact#5 Signed-off-by: Kornilios Kourtis <kornilios@isovalent.com>
There does not seem to be an easy way to delete artifacts of the existing run (see details below). Intsead, this patch truncates them so at least we do not waste space. I tried the following: - name: Delete artifacts uses: actions/github-script@v3 with: github-token: ${{ github.token }} script: | const {GITHUB_RUN_ID, GITHUB_REPOSITORY} = process.env const artifactsURL = `/repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/artifacts` console.log(`Deleting artifacs from ${artifactsURL}`) const response = await github.request(`GET ${artifactsURL}`) console.log(response.data) for (artifact of response.data.artifacts) { console.log(`Deleting artifact with name=${artifact.name}, id=${artifact.id}`) const deleteResponse = await github.request(`DELETE /repos/${repo}/actions/artifacts/${artifact.id}`) } But it did not work, because the GITHUB_RUN_ID does not seem to be available over the REST API until after the action is finished. The logs from my action printed: Deleting artifacs from /repos/cilium/tetragon/actions/runs/2817658181/artifacts ... { total_count: 0, artifacts: [] } But if I curl the above URL after the action, I get: { "total_count": 1, "artifacts": [ { "id": 322874447, "node_id": "MDg6QXJ0aWZhY3QzMjI4NzQ0NDc=", "name": "tetragon-build", "size_in_bytes": 6, "url": "https://api.github.com/repos/cilium/tetragon/actions/artifacts/322874447", "archive_download_url": "https://api.github.com/repos/cilium/tetragon/actions/artifacts/322874447/zip", "expired": false, "created_at": "2022-08-08T11:50:54Z", "updated_at": "2022-08-08T11:50:54Z", "expires_at": "2022-08-13T11:49:30Z", "workflow_run": { "id": 2817658181, "repository_id": 473137633, "head_repository_id": 473137633, "head_branch": "pr/kkourt/lvhtest", "head_sha": "ea22990af2283c38aac92f7e6febacabf3786280" } } ] } For (even) more context, see: actions/upload-artifact#5 Signed-off-by: Kornilios Kourtis <kornilios@isovalent.com>
There does not seem to be an easy way to delete artifacts of the existing run (see details below). Instead, this patch truncates the build artifact so at least we do not waste space. I tried the following: - name: Delete artifacts uses: actions/github-script@v3 with: github-token: ${{ github.token }} script: | const {GITHUB_RUN_ID, GITHUB_REPOSITORY} = process.env const artifactsURL = `/repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/artifacts` console.log(`Deleting artifacs from ${artifactsURL}`) const response = await github.request(`GET ${artifactsURL}`) console.log(response.data) for (artifact of response.data.artifacts) { console.log(`Deleting artifact with name=${artifact.name}, id=${artifact.id}`) const deleteResponse = await github.request(`DELETE /repos/${repo}/actions/artifacts/${artifact.id}`) } But it did not work, because the GITHUB_RUN_ID does not seem to be available over the REST API until after the action is finished. The logs from my action printed: Deleting artifacs from /repos/cilium/tetragon/actions/runs/2817658181/artifacts ... { total_count: 0, artifacts: [] } But if I curl the above URL after the action, I get: { "total_count": 1, "artifacts": [ { "id": 322874447, "node_id": "MDg6QXJ0aWZhY3QzMjI4NzQ0NDc=", "name": "tetragon-build", "size_in_bytes": 6, "url": "https://api.github.com/repos/cilium/tetragon/actions/artifacts/322874447", "archive_download_url": "https://api.github.com/repos/cilium/tetragon/actions/artifacts/322874447/zip", "expired": false, "created_at": "2022-08-08T11:50:54Z", "updated_at": "2022-08-08T11:50:54Z", "expires_at": "2022-08-13T11:49:30Z", "workflow_run": { "id": 2817658181, "repository_id": 473137633, "head_repository_id": 473137633, "head_branch": "pr/kkourt/lvhtest", "head_sha": "ea22990af2283c38aac92f7e6febacabf3786280" } } ] } For (even) more context, see: actions/upload-artifact#5 Signed-off-by: Kornilios Kourtis <kornilios@isovalent.com>
There does not seem to be an easy way to delete artifacts of the existing run (see details below). Instead, this patch truncates the build artifact so at least we do not waste space. I tried the following: - name: Delete artifacts uses: actions/github-script@v3 with: github-token: ${{ github.token }} script: | const {GITHUB_RUN_ID, GITHUB_REPOSITORY} = process.env const artifactsURL = `/repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/artifacts` console.log(`Deleting artifacs from ${artifactsURL}`) const response = await github.request(`GET ${artifactsURL}`) console.log(response.data) for (artifact of response.data.artifacts) { console.log(`Deleting artifact with name=${artifact.name}, id=${artifact.id}`) const deleteResponse = await github.request(`DELETE /repos/${repo}/actions/artifacts/${artifact.id}`) } But it did not work, because the GITHUB_RUN_ID does not seem to be available over the REST API until after the action is finished. The logs from my action printed: Deleting artifacs from /repos/cilium/tetragon/actions/runs/2817658181/artifacts ... { total_count: 0, artifacts: [] } But if I curl the above URL after the action, I get: { "total_count": 1, "artifacts": [ { "id": 322874447, "node_id": "MDg6QXJ0aWZhY3QzMjI4NzQ0NDc=", "name": "tetragon-build", "size_in_bytes": 6, "url": "https://api.github.com/repos/cilium/tetragon/actions/artifacts/322874447", "archive_download_url": "https://api.github.com/repos/cilium/tetragon/actions/artifacts/322874447/zip", "expired": false, "created_at": "2022-08-08T11:50:54Z", "updated_at": "2022-08-08T11:50:54Z", "expires_at": "2022-08-13T11:49:30Z", "workflow_run": { "id": 2817658181, "repository_id": 473137633, "head_repository_id": 473137633, "head_branch": "pr/kkourt/lvhtest", "head_sha": "ea22990af2283c38aac92f7e6febacabf3786280" } } ] } For (even) more context, see: actions/upload-artifact#5 Signed-off-by: Kornilios Kourtis <kornilios@isovalent.com>
There does not seem to be an easy way to delete artifacts of the existing run (see details below). Instead, this patch truncates the build artifact so at least we do not waste space. I tried the following: - name: Delete artifacts uses: actions/github-script@v3 with: github-token: ${{ github.token }} script: | const {GITHUB_RUN_ID, GITHUB_REPOSITORY} = process.env const artifactsURL = `/repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/artifacts` console.log(`Deleting artifacs from ${artifactsURL}`) const response = await github.request(`GET ${artifactsURL}`) console.log(response.data) for (artifact of response.data.artifacts) { console.log(`Deleting artifact with name=${artifact.name}, id=${artifact.id}`) const deleteResponse = await github.request(`DELETE /repos/${repo}/actions/artifacts/${artifact.id}`) } But it did not work, because the GITHUB_RUN_ID does not seem to be available over the REST API until after the action is finished. The logs from my action printed: Deleting artifacs from /repos/cilium/tetragon/actions/runs/2817658181/artifacts ... { total_count: 0, artifacts: [] } But if I curl the above URL after the action, I get: { "total_count": 1, "artifacts": [ { "id": 322874447, "node_id": "MDg6QXJ0aWZhY3QzMjI4NzQ0NDc=", "name": "tetragon-build", "size_in_bytes": 6, "url": "https://api.github.com/repos/cilium/tetragon/actions/artifacts/322874447", "archive_download_url": "https://api.github.com/repos/cilium/tetragon/actions/artifacts/322874447/zip", "expired": false, "created_at": "2022-08-08T11:50:54Z", "updated_at": "2022-08-08T11:50:54Z", "expires_at": "2022-08-13T11:49:30Z", "workflow_run": { "id": 2817658181, "repository_id": 473137633, "head_repository_id": 473137633, "head_branch": "pr/kkourt/lvhtest", "head_sha": "ea22990af2283c38aac92f7e6febacabf3786280" } } ] } For (even) more context, see: actions/upload-artifact#5 Signed-off-by: Kornilios Kourtis <kornilios@isovalent.com>
There does not seem to be an easy way to delete artifacts of the existing run (see details below). Instead, this patch truncates the build artifact so at least we do not waste space. I tried the following: - name: Delete artifacts uses: actions/github-script@v3 with: github-token: ${{ github.token }} script: | const {GITHUB_RUN_ID, GITHUB_REPOSITORY} = process.env const artifactsURL = `/repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/artifacts` console.log(`Deleting artifacs from ${artifactsURL}`) const response = await github.request(`GET ${artifactsURL}`) console.log(response.data) for (artifact of response.data.artifacts) { console.log(`Deleting artifact with name=${artifact.name}, id=${artifact.id}`) const deleteResponse = await github.request(`DELETE /repos/${repo}/actions/artifacts/${artifact.id}`) } But it did not work, because the GITHUB_RUN_ID does not seem to be available over the REST API until after the action is finished. The logs from my action printed: Deleting artifacs from /repos/cilium/tetragon/actions/runs/2817658181/artifacts ... { total_count: 0, artifacts: [] } But if I curl the above URL after the action, I get: { "total_count": 1, "artifacts": [ { "id": 322874447, "node_id": "MDg6QXJ0aWZhY3QzMjI4NzQ0NDc=", "name": "tetragon-build", "size_in_bytes": 6, "url": "https://api.github.com/repos/cilium/tetragon/actions/artifacts/322874447", "archive_download_url": "https://api.github.com/repos/cilium/tetragon/actions/artifacts/322874447/zip", "expired": false, "created_at": "2022-08-08T11:50:54Z", "updated_at": "2022-08-08T11:50:54Z", "expires_at": "2022-08-13T11:49:30Z", "workflow_run": { "id": 2817658181, "repository_id": 473137633, "head_repository_id": 473137633, "head_branch": "pr/kkourt/lvhtest", "head_sha": "ea22990af2283c38aac92f7e6febacabf3786280" } } ] } For (even) more context, see: actions/upload-artifact#5 Signed-off-by: Kornilios Kourtis <kornilios@isovalent.com>
There does not seem to be an easy way to delete artifacts of the existing run (see details below). Instead, this patch truncates the build artifact so at least we do not waste space. I tried the following: - name: Delete artifacts uses: actions/github-script@v3 with: github-token: ${{ github.token }} script: | const {GITHUB_RUN_ID, GITHUB_REPOSITORY} = process.env const artifactsURL = `/repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/artifacts` console.log(`Deleting artifacs from ${artifactsURL}`) const response = await github.request(`GET ${artifactsURL}`) console.log(response.data) for (artifact of response.data.artifacts) { console.log(`Deleting artifact with name=${artifact.name}, id=${artifact.id}`) const deleteResponse = await github.request(`DELETE /repos/${repo}/actions/artifacts/${artifact.id}`) } But it did not work, because the GITHUB_RUN_ID does not seem to be available over the REST API until after the action is finished. The logs from my action printed: Deleting artifacs from /repos/cilium/tetragon/actions/runs/2817658181/artifacts ... { total_count: 0, artifacts: [] } But if I curl the above URL after the action, I get: { "total_count": 1, "artifacts": [ { "id": 322874447, "node_id": "MDg6QXJ0aWZhY3QzMjI4NzQ0NDc=", "name": "tetragon-build", "size_in_bytes": 6, "url": "https://api.github.com/repos/cilium/tetragon/actions/artifacts/322874447", "archive_download_url": "https://api.github.com/repos/cilium/tetragon/actions/artifacts/322874447/zip", "expired": false, "created_at": "2022-08-08T11:50:54Z", "updated_at": "2022-08-08T11:50:54Z", "expires_at": "2022-08-13T11:49:30Z", "workflow_run": { "id": 2817658181, "repository_id": 473137633, "head_repository_id": 473137633, "head_branch": "pr/kkourt/lvhtest", "head_sha": "ea22990af2283c38aac92f7e6febacabf3786280" } } ] } For (even) more context, see: actions/upload-artifact#5 Signed-off-by: Kornilios Kourtis <kornilios@isovalent.com>
The But I found geekyeggo/delete-artifact@v2 which does remove artifacts created in the same workflow before finishing the workflow. |
I managed to stop this error by adding a new step on my deploy script:
Place it right above the upload step and it will delete all previous artifacts of the repo. |
I am starting to think that having to
is feature not a bug and with my tinfoil hat on I would guess theres number of such people glowing somewhere in GH HQ. for people landing from google here have this (if solutions from this issue are not working) # https://github.com/actions/upload-artifact/issues/290#issuecomment-1374207010
name: Test
on:
workflow_run:
branches: [main]
workflows: [nonprod,prod,test]
types:
- completed
jobs:
on-success:
runs-on: ubuntu-latest
name: Delete artifact
if: github.event.workflow_run.head_branch == 'main' # feel free to take this out if you don't need it
steps:
- id: get-id
run: | # grab the id of the artifact we just created
echo "ARTIFACT_ID=$(gh api -H 'Accept: application/vnd.github+json' ${{github.event.workflow_run.artifacts_url}} --jq .artifacts[].id)" >> $GITHUB_ENV
env:
GITHUB_TOKEN: ${{ secrets.GH_REPO_ACCESS_PAT }}
- run: |
echo "artifact id is ${{ env.ARTIFACT_ID}}"
echo "run url is ${{github.event.workflow_run.artifacts_url}}"
- id: delete-artifact
run: | # https://docs.github.com/en/rest/actions/artifacts?apiVersion=2022-11-28#delete-an-artifact
gh api --method DELETE -H 'Accept: application/vnd.github+json' /repos/${{github.repository}}/actions/artifacts/${{env.ARTIFACT_ID}}
env:
GITHUB_TOKEN: ${{ secrets.GH_REPO_ACCESS_PAT }}
on-failure:
runs-on: ubuntu-latest
if: ${{ github.event.workflow_run.conclusion == 'failure' }}
steps:
- run: echo 'The triggering workflow failed' |
hey this is my workflow :) I have a stand alone script on CLI that will delete artifacts based on creation date: https://gist.github.com/DPatrickBoyd/afb54165df0f51903be3f0edea77f9cb |
Why is this issue closed? There still doesnt seem to be a particularly good way to remove artifacts? |
The issue is not resolved. Continuously closing it is not solving any problems and you've not addressed what people are raising. This has completely stopped my workflow and I need it to resolve to deploy something into prod. |
+1 for an official |
Do we have any workarounds on how to delete not all artifacts, but those associated with certain branches? Something using ${{ github.head_ref }} ? |
Not sure if this is completely fixed.
but the error doesn't go away. Not sure why its keep complaining even if there are no artifacts. |
This seems to work: |
It is possible to remove an uploaded artifact?
The text was updated successfully, but these errors were encountered: