[MDS-5300] Added health check probe to docman celery

[simensma-fresh]

Objective

MDS-5300

Added health check probe to Docman Worker (celery). This is an attempt to resolve the high cpu usage + restarts experienced with Celery in dev/test/prod using celery inspect ping for health checks.

It's a known issue with the current health checks that it can cause high CPU usage according to celery/celery#4079, and the solution is heavily inspired by the proposed solutions there.

Solution

1. Have a "liveness file" regularly updated by the celery worker by using celery signals
2. Update the OC liveness checks to see when the liveness file was latest updated, if it hasn't been updated in a while, the worker is not functioning properly, and the worker can be considered unhealthy

Gitops repo PR with updated health check: https://github.com/bcgov-c/tenant-gitops-4c2ba9/pull/33

[matbusby]

Good thinking!

[asinn134]

nice!