Skip to content

v0.1.1 — relay-disk leak + ADS API request shape

Choose a tag to compare

@elkimek elkimek released this 05 May 05:15
· 23 commits to main since this release

Post-incident release. Both fixes shipped together because they were diagnosed in the same session — disk leak + stale atmosphere data from the broken request.

Fixed

Staging dir leak (#11)

_pull_cams_blocking previously created tempfile.TemporaryDirectory() with no dir= argument — defaulting to /tmp inside the container. On graceful exit the with block cleaned up; on SIGKILL/OOM mid-retrieve, TemporaryDirectory's finalizer never fired, leaving ~470 MB of unzipped netCDF orphaned in the writable layer. Over weeks of dev-cycle restarts on a production deploy, this filled the host disk to 100% and wedged the colocated evolu-relay's SQLite writes with SQLITE_FULL / ENOSPC: no space left on device.

Staging now lives under cache_dir (the persistent /data volume) with a stable cams-stage-* prefix. CamsCache.__init__ sweeps any orphan cams-stage-* dirs from prior runs at startup. Worst-case future leak class can't take down a colocated service — the volume is monitored independently of the host disk.

CAMS API request shape (#12)

Background pulls were failing with HTTP 400 — invalid combination of values. The legacy format: 'netcdf_zip' key passed through ecmwf-datastores-client's compatibility shim, which silently rewrote it to data_format: 'netcdf' AND injected grid: [0.4, 0.4]. CAMS Atmospheric Composition Forecasts has a fixed native grid; an explicit grid is rejected.

Request now uses the ADS-native data_format key directly. time and type wrapped as lists to match the shape the ADS web form emits. Shim no longer touches our payload.

Production rollout

ssh root@VPS 'cd /opt/getbased-uvdata && git pull && docker compose up -d --build uvdata'
docker logs getbased-uvdata-uvdata-1 -f  # expect: "CAMS pull OK: N timesteps, ..."

Watch df -h / to confirm the writable layer stays bounded after the next dev-cycle restart.