How to monitor weblate backup status

[oldboys92]

We host weblate via helm chart on a K8S cluster and configured weblate to backup via borg backup on a PVC/pod local file system. After a while the backups started to fail and we found this only by mistake (random checking the Admin weblate UI). We don't use any of the weblate monitor integrations (neither Sentry nor Rollbar), but for the K8S cluster we ingest all pod logs to Datadog.

After intensive investigations of weblate logs in Datadog, we can't identify a suitable trigger or log output for the failing backup jobs.

Are there any options to monitor if backups jobs complete/fail without using the documented integrations (Sentry / Rollbar)?

[nijel]

Backup failure triggers a configuration error (what you've seen in the UI) and the count of these is exposed in /api/metrics/. So that would be the easiest approach to see if something is wrong.

There don't seem to be logging of these to the log right now:

weblate/weblate/wladmin/models.py

Lines 268 to 273 in 095d01a

	def backup(self):
	try:
	log = backup(self.repository, self.passphrase)
	self.backuplog_set.create(event="backup", log=log)
	except BackupError as error:
	self.backuplog_set.create(event="error", log=str(error))