Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"You are running out of disk space" Error. How can I fix this? #2840

Closed
1 of 7 tasks
jessehui opened this issue Mar 4, 2021 · 11 comments
Closed
1 of 7 tasks

"You are running out of disk space" Error. How can I fix this? #2840

jessehui opened this issue Mar 4, 2021 · 11 comments
Assignees
Labels
OS: Ubuntu question Further information is requested

Comments

@jessehui
Copy link

jessehui commented Mar 4, 2021

Description
The previous execution of this action never gives an error like this.
截屏2021-03-04 下午5 48 22

I think the reason is that the virtual environment is changed. But I am not sure. My question is whether there is a fix for this? I don't want to change the yaml file. Thank you. The action can be found here: https://github.com/occlum/occlum/actions/runs/619537350

Virtual environments affected

  • Ubuntu 16.04
  • Ubuntu 18.04
  • Ubuntu 20.04
  • macOS 10.15
  • macOS 11.0
  • Windows Server 2016 R2
  • Windows Server 2019

Image version

Image version where you are experiencing the issue.
Environment: ubuntu-18.04
Version: 20210219.1

@al-cheb al-cheb added OS: Ubuntu question Further information is requested and removed needs triage labels Mar 4, 2021
@al-cheb
Copy link
Contributor

al-cheb commented Mar 4, 2021

Hello, @jessehui
Based on documentation, we provide at least 14 GB of free space (https://docs.github.com/en/actions/reference/specifications-for-github-hosted-runners#supported-runners-and-hardware-resources).

As a workaround:

sudo rm -rf /usr/share/dotnet
sudo rm -rf /opt/ghc
sudo rm -rf "/usr/local/share/boost"
sudo rm -rf "$AGENT_TOOLSDIRECTORY"

@jessehui
Copy link
Author

jessehui commented Mar 5, 2021

@al-cheb Thank you for your reply. I would give it a try. I am just curious whether this limitation is added recently? Because we never encounter this error running the same workflow before.

@arroyc
Copy link
Contributor

arroyc commented Mar 5, 2021

@al-cheb is there any way that you can provide more disk space to the customer? we do multistage docker build and 14 gb is too small for us, we are using 'ubuntu-latest' ado / azure pipelines .. not github actions

Error processing tar file(exit status 1): write /opt/dotnet/3.1.405/packs/Microsoft.NETCore.App.Ref/3.1.0/ref/netcoreapp3.1/System.Resources.ResourceManager.dll: no space left on device

@maxim-lobanov
Copy link
Contributor

@jessehui , this limitation has always existed but previously, we have provided a bit more free space. Sometimes, image can have ~15-20 GB of free space and this volume can be changed without notice. We only guarantee that images contain at least 14 GB of free space.

@arroyc , unfortunately, no. Only workarounds that Aleks has shared above

@arroyc
Copy link
Contributor

arroyc commented Mar 5, 2021

@maxim-lobanov thanks for responding ... i just found following information in one of our builds ... its not even 14G ... its less than 10GB .. can you please confirm that? I'm assuming we are only using /dev/sda1

2021-03-05T17:45:05.3501527Z Filesystem Size Used Avail Use% Mounted on
2021-03-05T17:45:05.3506978Z /dev/root 84G 34G 50G 40% /
2021-03-05T17:45:05.3508722Z devtmpfs 3.4G 0 3.4G 0% /dev
2021-03-05T17:45:05.3510122Z tmpfs 3.4G 12K 3.4G 1% /dev/shm
2021-03-05T17:45:05.3511897Z tmpfs 696M 1.1M 695M 1% /run
2021-03-05T17:45:05.3513269Z tmpfs 5.0M 0 5.0M 0% /run/lock
2021-03-05T17:45:05.3515129Z tmpfs 3.4G 0 3.4G 0% /sys/fs/cgroup
2021-03-05T17:45:05.3516728Z /dev/loop0 139M 139M 0 100% /snap/chromium/1506
2021-03-05T17:45:05.3517833Z /dev/loop1 138M 138M 0 100% /snap/chromium/1497
2021-03-05T17:45:05.3518877Z /dev/loop2 56M 56M 0 100% /snap/core18/1988
2021-03-05T17:45:05.3520991Z /dev/loop3 163M 163M 0 100% /snap/gnome-3-28-1804/145
2021-03-05T17:45:05.3522240Z /dev/loop4 32M 32M 0 100% /snap/snapd/11036
2021-03-05T17:45:05.3523353Z /dev/sdb15 105M 7.8M 97M 8% /boot/efi
2021-03-05T17:45:05.3524425Z /dev/loop5 70M 70M 0 100% /snap/lxd/19188
2021-03-05T17:45:05.3525988Z /dev/loop6 65M 65M 0 100% /snap/gtk-common-themes/1514
2021-03-05T17:45:05.3526844Z /dev/loop7 33M 33M 0 100% /snap/snapd/11107
2021-03-05T17:45:05.3527413Z /dev/sda1 14G 4.1G 9.0G 32% /mnt

@jessehui
Copy link
Author

jessehui commented Mar 8, 2021

@al-cheb @maxim-lobanov Thank you for your help. We've managed to work this out. You may close this issue.

@al-cheb al-cheb closed this as completed Mar 9, 2021
behlendorf added a commit to behlendorf/zfs that referenced this issue Apr 1, 2021
Recently we've been running out of free space in the ubuntu 20.04
environment resulting in test failures.  This appears to be caused
by a change in the default available free space and not because of
any change in OpenZFS. Try and avoid this failure by applying a
suggested workaround which removes some unnecessary files.

  actions/runner-images#2840

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
behlendorf added a commit to openzfs/zfs that referenced this issue Apr 1, 2021
Recently we've been running out of free space in the ubuntu 20.04
environment resulting in test failures.  This appears to be caused
by a change in the default available free space and not because of
any change in OpenZFS. Try and avoid this failure by applying a
suggested workaround which removes some unnecessary files.

actions/runner-images#2840

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #11826
behlendorf added a commit to openzfs/zfs that referenced this issue Apr 7, 2021
Recently we've been running out of free space in the ubuntu 20.04
environment resulting in test failures.  This appears to be caused
by a change in the default available free space and not because of
any change in OpenZFS. Try and avoid this failure by applying a
suggested workaround which removes some unnecessary files.

actions/runner-images#2840

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #11826
adamdmoss pushed a commit to adamdmoss/zfs that referenced this issue Apr 10, 2021
Recently we've been running out of free space in the ubuntu 20.04
environment resulting in test failures.  This appears to be caused
by a change in the default available free space and not because of
any change in OpenZFS. Try and avoid this failure by applying a
suggested workaround which removes some unnecessary files.

actions/runner-images#2840

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#11826
ghost pushed a commit to truenas/zfs that referenced this issue May 6, 2021
Recently we've been running out of free space in the ubuntu 20.04
environment resulting in test failures.  This appears to be caused
by a change in the default available free space and not because of
any change in OpenZFS. Try and avoid this failure by applying a
suggested workaround which removes some unnecessary files.

actions/runner-images#2840

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#11826
ghost pushed a commit to truenas/zfs that referenced this issue May 6, 2021
Recently we've been running out of free space in the ubuntu 20.04
environment resulting in test failures.  This appears to be caused
by a change in the default available free space and not because of
any change in OpenZFS. Try and avoid this failure by applying a
suggested workaround which removes some unnecessary files.

actions/runner-images#2840

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#11826
ghost pushed a commit to truenas/zfs that referenced this issue May 6, 2021
Recently we've been running out of free space in the ubuntu 20.04
environment resulting in test failures.  This appears to be caused
by a change in the default available free space and not because of
any change in OpenZFS. Try and avoid this failure by applying a
suggested workaround which removes some unnecessary files.

actions/runner-images#2840

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#11826
ghost pushed a commit to truenas/zfs that referenced this issue May 7, 2021
Recently we've been running out of free space in the ubuntu 20.04
environment resulting in test failures.  This appears to be caused
by a change in the default available free space and not because of
any change in OpenZFS. Try and avoid this failure by applying a
suggested workaround which removes some unnecessary files.

actions/runner-images#2840

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#11826
ghost pushed a commit to truenas/zfs that referenced this issue May 10, 2021
Recently we've been running out of free space in the ubuntu 20.04
environment resulting in test failures.  This appears to be caused
by a change in the default available free space and not because of
any change in OpenZFS. Try and avoid this failure by applying a
suggested workaround which removes some unnecessary files.

actions/runner-images#2840

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#11826
ghost pushed a commit to truenas/zfs that referenced this issue May 10, 2021
Recently we've been running out of free space in the ubuntu 20.04
environment resulting in test failures.  This appears to be caused
by a change in the default available free space and not because of
any change in OpenZFS. Try and avoid this failure by applying a
suggested workaround which removes some unnecessary files.

actions/runner-images#2840

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#11826
ghost pushed a commit to truenas/zfs that referenced this issue May 10, 2021
Recently we've been running out of free space in the ubuntu 20.04
environment resulting in test failures.  This appears to be caused
by a change in the default available free space and not because of
any change in OpenZFS. Try and avoid this failure by applying a
suggested workaround which removes some unnecessary files.

actions/runner-images#2840

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#11826
ghost pushed a commit to truenas/zfs that referenced this issue May 13, 2021
Recently we've been running out of free space in the ubuntu 20.04
environment resulting in test failures.  This appears to be caused
by a change in the default available free space and not because of
any change in OpenZFS. Try and avoid this failure by applying a
suggested workaround which removes some unnecessary files.

actions/runner-images#2840

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#11826
behlendorf added a commit to openzfs/zfs that referenced this issue May 20, 2021
Recently we've been running out of free space in the ubuntu 20.04
environment resulting in test failures.  This appears to be caused
by a change in the default available free space and not because of
any change in OpenZFS. Try and avoid this failure by applying a
suggested workaround which removes some unnecessary files.

actions/runner-images#2840

Reviewed-by: George Melikov <mail@gmelikov.ru>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #11826
kingsd041 added a commit to kingsd041/ksd-update-build that referenced this issue Mar 4, 2024
# 清理 action 镜像磁盘空间
# actions/runner-images#2840
sudo rm -rf /usr/share/dotnet
sudo rm -rf /opt/ghc
sudo rm -rf "/usr/local/share/boost"
sudo rm -rf "$AGENT_TOOLSDIRECTORY"
vadz added a commit to wxWidgets/wxWidgets that referenced this issue Mar 16, 2024
Our CI jobs constantly fail due to the lack of disk space, so free up
some space by removing unnecessary directories following suggestions at
actions/runner-images#2840
rsanzante added a commit to Metadrop/drupal-boilerplate that referenced this issue Mar 25, 2024
rsanzante added a commit to Metadrop/drupal-boilerplate that referenced this issue Mar 25, 2024
jimmidyson pushed a commit to nutanix-cloud-native/cluster-api-runtime-extensions-nutanix that referenced this issue May 9, 2024
**What problem does this PR solve?**:
Noticed `no space left on device` [error
messages](https://github.com/nutanix-cloud-native/cluster-api-runtime-extensions-nutanix/actions/runs/9008250141/job/24773942020?pr=638#step:5:276)
in e2e tests which cases the tests to fail.

This PR runs a step to clean up the disk, replicated from upstream issue
actions/runner-images#2840

**Which issue(s) this PR fixes**:
Fixes #

**How Has This Been Tested?**:
<!--
Please describe the tests that you ran to verify your changes.
Provide output from the tests and any manual steps needed to replicate
the tests.
-->

**Special notes for your reviewer**:
<!--
Use this to provide any additional information to the reviewers.
This may include:
- Best way to review the PR.
- Where the author wants the most review attention on.
- etc.
-->
snissn added a commit to filecoin-project/filecoin-solidity that referenced this issue May 29, 2024
@neilmayhew
Copy link

sudo rm -rf /usr/share/dotnet
sudo rm -rf /opt/ghc
sudo rm -rf "/usr/local/share/boost"
sudo rm -rf "$AGENT_TOOLSDIRECTORY"

For the benefit of those who come across this several years later, as I did, here's a more up-to-date list of big things that you could remove at the start of a build:

# Remove software and language runtimes we're not using
sudo rm -rf \
  "$AGENT_TOOLSDIRECTORY" \
  /opt/google/chrome \
  /opt/microsoft/msedge \
  /opt/microsoft/powershell \
  /opt/pipx \
  /usr/lib/mono \
  /usr/local/julia* \
  /usr/local/lib/android \
  /usr/local/lib/node_modules \
  /usr/local/share/chromium \
  /usr/local/share/powershell \
  /usr/share/dotnet \
  /usr/share/swift
df -h /

sayboras added a commit to sayboras/cilium that referenced this issue Aug 8, 2024
We are having the below failure due to no disk space, it seems like we
can remove pre-installed software and language runtimes, which are not
use in Cilium, to reclaim more disk space.

Alternative option is to bump the runner, but it might not be the best
resource and cost utilization.

Relates: https://github.com/cilium/cilium/actions/runs/10300396788
Relates: actions/runner-images#2840 (comment)
Signed-off-by: Tam Mach <tam.mach@cilium.io>
sayboras added a commit to cilium/cilium that referenced this issue Aug 8, 2024
We are having the below failure due to no disk space, it seems like we
can remove pre-installed software and language runtimes, which are not
use in Cilium, to reclaim more disk space.

Alternative option is to bump the runner, but it might not be the best
resource and cost utilization.

Relates: https://github.com/cilium/cilium/actions/runs/10300396788
Relates: actions/runner-images#2840 (comment)
Signed-off-by: Tam Mach <tam.mach@cilium.io>
sayboras added a commit to cilium/cilium that referenced this issue Aug 8, 2024
We are having the below failure due to no disk space, it seems like we
can remove pre-installed software and language runtimes, which are not
use in Cilium, to reclaim more disk space.

Alternative option is to bump the runner, but it might not be the best
resource and cost utilization.

Relates: https://github.com/cilium/cilium/actions/runs/10300396788
Relates: actions/runner-images#2840 (comment)
Signed-off-by: Tam Mach <tam.mach@cilium.io>
sayboras added a commit to cilium/cilium that referenced this issue Aug 9, 2024
We are having the below failure due to no disk space, it seems like we
can remove pre-installed software and language runtimes, which are not
use in Cilium, to reclaim more disk space.

Alternative option is to bump the runner, but it might not be the best
resource and cost utilization.

Relates: https://github.com/cilium/cilium/actions/runs/10300396788
Relates: actions/runner-images#2840 (comment)
Signed-off-by: Tam Mach <tam.mach@cilium.io>
sayboras added a commit to cilium/cilium that referenced this issue Aug 9, 2024
We are having the below failure due to no disk space, it seems like we
can remove pre-installed software and language runtimes, which are not
use in Cilium, to reclaim more disk space.

Alternative option is to bump the runner, but it might not be the best
resource and cost utilization.

Relates: https://github.com/cilium/cilium/actions/runs/10300396788
Relates: actions/runner-images#2840 (comment)
Signed-off-by: Tam Mach <tam.mach@cilium.io>
sayboras added a commit to cilium/cilium that referenced this issue Aug 9, 2024
We are having the below failure due to no disk space, it seems like we
can remove pre-installed software and language runtimes, which are not
use in Cilium, to reclaim more disk space.

Alternative option is to bump the runner, but it might not be the best
resource and cost utilization.

Relates: https://github.com/cilium/cilium/actions/runs/10300396788
Relates: actions/runner-images#2840 (comment)
Signed-off-by: Tam Mach <tam.mach@cilium.io>
sayboras added a commit to cilium/cilium that referenced this issue Aug 9, 2024
We are having the below failure due to no disk space, it seems like we
can remove pre-installed software and language runtimes, which are not
use in Cilium, to reclaim more disk space.

Alternative option is to bump the runner, but it might not be the best
resource and cost utilization.

Relates: https://github.com/cilium/cilium/actions/runs/10300396788
Relates: actions/runner-images#2840 (comment)
Signed-off-by: Tam Mach <tam.mach@cilium.io>
joestringer pushed a commit to cilium/cilium that referenced this issue Aug 9, 2024
We are having the below failure due to no disk space, it seems like we
can remove pre-installed software and language runtimes, which are not
use in Cilium, to reclaim more disk space.

Alternative option is to bump the runner, but it might not be the best
resource and cost utilization.

Relates: https://github.com/cilium/cilium/actions/runs/10300396788
Relates: actions/runner-images#2840 (comment)
Signed-off-by: Tam Mach <tam.mach@cilium.io>
jschwinger233 pushed a commit to cilium/cilium that referenced this issue Aug 12, 2024
[ upstream commit e553bd2 ]

We are having the below failure due to no disk space, it seems like we
can remove pre-installed software and language runtimes, which are not
use in Cilium, to reclaim more disk space.

Alternative option is to bump the runner, but it might not be the best
resource and cost utilization.

Relates: https://github.com/cilium/cilium/actions/runs/10300396788
Relates: actions/runner-images#2840 (comment)
Signed-off-by: Tam Mach <tam.mach@cilium.io>
Signed-off-by: gray <greyschwinger@gmail.com>
jschwinger233 pushed a commit to cilium/cilium that referenced this issue Aug 12, 2024
[ upstream commit e553bd2 ]

We are having the below failure due to no disk space, it seems like we
can remove pre-installed software and language runtimes, which are not
use in Cilium, to reclaim more disk space.

Alternative option is to bump the runner, but it might not be the best
resource and cost utilization.

Relates: https://github.com/cilium/cilium/actions/runs/10300396788
Relates: actions/runner-images#2840 (comment)
Signed-off-by: Tam Mach <tam.mach@cilium.io>
Signed-off-by: gray <greyschwinger@gmail.com>
jschwinger233 pushed a commit to cilium/cilium that referenced this issue Aug 13, 2024
[ upstream commit e553bd2 ]

We are having the below failure due to no disk space, it seems like we
can remove pre-installed software and language runtimes, which are not
use in Cilium, to reclaim more disk space.

Alternative option is to bump the runner, but it might not be the best
resource and cost utilization.

Relates: https://github.com/cilium/cilium/actions/runs/10300396788
Relates: actions/runner-images#2840 (comment)
Signed-off-by: Tam Mach <tam.mach@cilium.io>
Signed-off-by: gray <greyschwinger@gmail.com>
github-merge-queue bot pushed a commit to cilium/cilium that referenced this issue Aug 13, 2024
[ upstream commit e553bd2 ]

We are having the below failure due to no disk space, it seems like we
can remove pre-installed software and language runtimes, which are not
use in Cilium, to reclaim more disk space.

Alternative option is to bump the runner, but it might not be the best
resource and cost utilization.

Relates: https://github.com/cilium/cilium/actions/runs/10300396788
Relates: actions/runner-images#2840 (comment)
Signed-off-by: Tam Mach <tam.mach@cilium.io>
Signed-off-by: gray <greyschwinger@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OS: Ubuntu question Further information is requested
Projects
None yet
Development

No branches or pull requests

9 participants