Summary
Per-job log files (${JOB_LOGS_DIR:-logs/jobs}/<jobId>.log, one per job run, written by JobRunner via FileSink) are never pruned. The JobScheduler fires jobs indefinitely, so the directory grows without bound — one small file per run, unbounded file count over time.
There is no rotation, retention, or cleanup anywhere in ccas.server.jobs. Contrast the API cache, which already has retention: ApiResponseCache.deleteBefore(now - retention) runs on every startup (Tables.ensureTables), windowed by the DB-owned cache_retention_days setting (AppSetting.CacheRetentionDays, compiled default 7 days). Job logs have no equivalent.
Why now
Surfaced via #130 (membership server jobs were writing empty logs). Mitigation already applied: JOB_LOGS_DIR now points outside the repo by default in local dev, and .gitignore covers logs/. That fixes repo pollution but not the unbounded growth of the log directory itself.
Proposed fix
Add a startup sweep deleting ${JOB_LOGS_DIR}/*.log older than a retention window, mirroring the cache-retention pattern:
- Run it where
JobRunner.live (or ServerTables.ensureTables) already does one-shot startup work — it already creates logDir.
- Retention value: reuse the
app_setting registry — add a job_log_retention_days AppSetting.Key[Int] with a compiled default (suggest 14–30 days; job logs are smaller and more diagnostic-useful than cache rows, so a longer default than the cache's 7d is reasonable). DB-owned, runtime-tunable, no schema churn — same shape as CacheRetentionDays.
- Delete by file mtime (or parse the ULID jobId's timestamp prefix — ULIDs are time-sortable, so
jobId < ulidFor(cutoff) is a cheap filter without stat-ing every file). Log how many were swept.
Scope notes
- Server-only concern (CLI builds no
JobRunner).
- Orthogonal to the DB
job_run row retention — this is just the on-disk log files. If job_run rows are also unbounded, consider a sibling sweep, but that's a separate decision.
- Keep it a startup sweep (like the cache) rather than a background timer — simplest, and matches the existing convention.
References
src/main/scala/ccas/server/jobs/FileSink.scala — per-job file writer (${logDir}/<jobId>.log).src/main/scala/ccas/server/jobs/JobRunner.scala — creates logDir once at startup; the natural sweep site.ApiResponseCache.deleteBefore + AppSetting.CacheRetentionDays — the retention pattern to mirror.application.conf job-logs.directory / JOB_LOGS_DIR — the configurable target.
Summary
Per-job log files (
${JOB_LOGS_DIR:-logs/jobs}/<jobId>.log, one per job run, written byJobRunnerviaFileSink) are never pruned. TheJobSchedulerfires jobs indefinitely, so the directory grows without bound — one small file per run, unbounded file count over time.There is no rotation, retention, or cleanup anywhere in
ccas.server.jobs. Contrast the API cache, which already has retention:ApiResponseCache.deleteBefore(now - retention)runs on every startup (Tables.ensureTables), windowed by the DB-ownedcache_retention_dayssetting (AppSetting.CacheRetentionDays, compiled default 7 days). Job logs have no equivalent.Why now
Surfaced via #130 (membership server jobs were writing empty logs). Mitigation already applied:
JOB_LOGS_DIRnow points outside the repo by default in local dev, and.gitignorecoverslogs/. That fixes repo pollution but not the unbounded growth of the log directory itself.Proposed fix
Add a startup sweep deleting
${JOB_LOGS_DIR}/*.logolder than a retention window, mirroring the cache-retention pattern:JobRunner.live(orServerTables.ensureTables) already does one-shot startup work — it already createslogDir.app_settingregistry — add ajob_log_retention_daysAppSetting.Key[Int]with a compiled default (suggest 14–30 days; job logs are smaller and more diagnostic-useful than cache rows, so a longer default than the cache's 7d is reasonable). DB-owned, runtime-tunable, no schema churn — same shape asCacheRetentionDays.jobId < ulidFor(cutoff)is a cheap filter without stat-ing every file). Log how many were swept.Scope notes
JobRunner).job_runrow retention — this is just the on-disk log files. Ifjob_runrows are also unbounded, consider a sibling sweep, but that's a separate decision.References
src/main/scala/ccas/server/jobs/FileSink.scala— per-job file writer (${logDir}/<jobId>.log).src/main/scala/ccas/server/jobs/JobRunner.scala— createslogDironce at startup; the natural sweep site.ApiResponseCache.deleteBefore+AppSetting.CacheRetentionDays— the retention pattern to mirror.application.confjob-logs.directory/JOB_LOGS_DIR— the configurable target.