Do we still need to define *-job services in docker-compose?

[regisb]

It seems to me that we could simply docker-compose run service sh -e -c "command". Why do we have to rely on a convoluted job architecture?

• Jobs don't add anything in terms of environment variables, volume mounting or dependencies.
• If a service must be present in jobs, but not started as a long-running service (e.g: mysql when RUN_MYSQL=false) then we can add it to docker-compose.jobs.yml, but that should not be the case for most services.
• Services that are run bypass the restart-policy, which is good for jobs (we don't want to restart jobs).
• We stopped doing k8s exec (here) because this caused us to bypass environment variables set by the Docker entrypoint. But since we no longer define env variables in the entrypoint, can we do exec again?

Such a change would considerably simplify the project architecture.

[regisb]

I had a closer look at this issue today. Here's my braindump. Tl;dr: there really is nothing we can do about this issue, so let's close it.

replacing jobs by exec: ❌

It appears that we cannot substitute jobs by exec. The reason is that exec requires the container to be up -- which might not be the case if it was not properly initialised. For instance, the lms container will crash if the mysql database was not created. In such a case, we will not be able to exec into the lms container because it will be stuck in a crash-fail loop.

rename job resources: ❌

So, we need to preserve Kubernetes job resources. We could definitely rename them and get rid of the "-job" suffix. But we would not be able to do the same for docker-compose job services. That's because such jobs would then have the same name as app services. They would then override other service declarations, and that would be very confusing.

get rid of job services in docker-compose only: ❌

We could just get rid of "*-job" services in docker-compose. Jobs would be executed as "run" commands with existing service declarations. But that would introduce a (potentially important) difference with Kubernetes. So that's probably not a good idea.

rename jobs for clarification: ❌

It's a little weird that jobs are added to the CLI_DO_INIT_TASKS filter without the "-job" suffix. On the other hand, if it was optional to add the "-job" suffix, then users would be allowed to create job services without the suffix. This would be confusing (see above).

We could be explicit about it, and force the "-job" suffix if it is not present. But this would not resolve any actual issue, and it would be even more confusing to explain to users.