Best practice to clean up disk space on Dagster Agent machine

[mudravrik]

Hi everyone!

We use Dagster Hybrid with agent running in Docker on GCP Compute Engine machine.
In our pipelines mostly default FilesystemIOManager is utilised, so we have a lot of intermediate products built up somewhere on the disk, which is obviously limited, and we want to manage the space there somehow.

So first of all, maybe Dagster have some built-in mechanism to clean up this stuff?

If not, what is a proper way to deal with it?
We do not have additional volumes set up for Agents aside docs-recommended config passing, so probably the easiest way will be just re-deploy Agent containers all together, since there no persistence of this data on host machine.
However is this really the best way, is there chances for breaking stuff on actual orchestration side, when some of its data magically disappears?
Another idea was to remove data from inside Dagster containers more precisely, but I was not able to actually locate it unfortunately.

[garethbrickman]

I think this duplicates or overlaps with #18536

[mudravrik]

Oh, I actually didn't realise. I should remove used containers above anything else, it should help, thank you so much!