Renovate CE keeps running out of space

[shenke-adsk]

Ever since we switched to CE, we have constantly experienced disk full issue:

2024-02-23T04:46:21.106-05:00 | ERROR: Error deleting job
-- | --
  | 2024-02-23T04:46:21.106-05:00 | "driver": "sqlite",
  | 2024-02-23T04:46:21.106-05:00 | "err": {
  | 2024-02-23T04:46:21.106-05:00 | "errno": 13,
  | 2024-02-23T04:46:21.106-05:00 | "code": "SQLITE_FULL",
  | 2024-02-23T04:46:21.106-05:00 | "message": "SQLITE_FULL: database or disk is full",
  | 2024-02-23T04:46:21.106-05:00 | "stack": "Error: SQLITE_FULL: database or disk is full"
  | 2024-02-23T04:46:21.106-05:00 | }

Our env variables:

LOG_LEVEL | value | debug
MEND_RNV_ACCEPT_TOS | value | y
MEND_RNV_CRON_JOB_SCHEDULER | value | 0 */4 * * *
MEND_RNV_GITHUB_APP_ID | valueFrom | arn:aws:ssm:xxxxx:xxxxxx:parameter
MEND_RNV_GITHUB_APP_KEY | valueFrom | arn:aws:ssm:xxxxx:xxxxxx:parameter
MEND_RNV_LICENSE_KEY | valueFrom | arn:aws:ssm:xxxxxxxx:xxxxxxx:parameter
MEND_RNV_PLATFORM | value | github
MEND_RNV_WEBHOOK_SECRET | valueFrom | arn:aws:ssm:xxxxxxxx:xxxxxxx:parameter
MEND_RNV_WORKER_EXECUTION_TIMEOUT | value | 90
NODE_OPTIONS | value | --max_old_space_size=8192
RENOVATE_EXECUTION_TIMEOUT | value | 60
RENOVATE_GIT_AUTHOR | value | xxxxxxxxxxx
RENOVATE_LOG_FILE | value | /var/log/renovate/renovate_runtime.log
RENOVATE_PLATFORM_COMMIT | value | true
RENOVATE_REPOSITORY_CACHE | value | enabled

[rarkins]

Where on the file system is the disk space being used?

[shenke-adsk]

It is an AWS ECS fargate task. Currently we are not using any extra storage like EFS, because we had not experienced such issue when we were using Renovate On-Prem. If we have to, we can add EFS storage but I need to understand what file(s) are eating up space fast. Unfortunately it is not easy to connect to a running ECS fargate container and get a shell to run diagnostic commands. I think with fargate the default space for the container is 8GB.

[rarkins]

I recommend 25GB+. Tools like npm and Go will be keeping a cache

[ramidij]

I suspect it's the sqlite in memory db which is eating up the disk space quickly. Is there a way we can configure something like AWS Redis?

[rarkins]

I suspect it's not the sqlite, but Postgresql is also supported