Run Tasks in Background

[tiloio]

Be able to run stuff (like database) in background (with waiting mechanisms, e.g. wait for log output or URL returns status code x).

When running tests or starting an application I depend on other processes which have to run in the background. For example if I have integration tests which use a MongoDB the MongoDB has to be started first and be ready to receive requests. Another example is when I want to start my app which is using this MongoDB, a backend and a frontend I have to start everything by hand in separate terminals.

It would be great if we have something to release single task, but where we will still receive the logs.

For example when I run my application with task start the task could look like this:

version: '3'

output: prefixed

tasks:
  start:
    deps: [start-backend, start-frontend]

  start-backend:
    dir: backend
    deps: [start-database]
    cmd:
      - npm start
    release-on:
      stdout: 'Running on localhost:8080'
      timeout: 60

  start-database:
    cmd:
      - docker run --name some-mongo -p 27017:27017 mongo:latest
    release-on:
      cmd: 
        - url --connect-timeout 10 --silent --show-error localhost:27017
      tries: 10
      wait: 100

  start-frontend:
    dir: frontend
    cmd:
      - npm start
    release-on:
      http:
        url: http://localhost:4200
        status: 200

First the backend and frontend will start in parallel.
The backend needs the database, so the database is started with docker and we wait max ten times with 100 ms pauses until the database is reachable via the curl command.
After that the backend is started with npm start. There we wait until the stdout contains Running on localhost:8080 with a timeout of 60 seconds.
In between the frontend was also started with npm start. There we wait until http://localhost:4200 returns HTTP 200 OK with the task default timeout.

The log output may look like that:

[start-database] {"t":{"$date":"2022-06-08T19:01:16.963+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"-","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
[start-database] {"t":{"$date":"2022-06-08T19:01:16.973+00:00"},"s":"I",  "c":"NETWORK",  "id":4915701, "ctx":"-","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":13},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":13},"outgoing":{"minWireVersion":0,"maxWireVersion":13},"isInternalClient":true}}}
[start-frontend] Building frontend...
[start-database] Released; successful command
[start-backend] Building backend...
[start-frontend] Released; waited for 'http://localhost:4200' returning HTTP 200
[start-backend] Running on localhost:8080
[start-backend] Released; waited for stdout 'Running on localhost:8080'
[start] All processes released; logs are printed; use control+c to stop

And if some of the released tasks are logging, you will still see the log in the terminal.
E.g. if I hit /api on the backend, the log looks like:

[start-database] {"t":{"$date":"2022-06-08T19:01:16.963+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"-","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
[start-database] {"t":{"$date":"2022-06-08T19:01:16.973+00:00"},"s":"I",  "c":"NETWORK",  "id":4915701, "ctx":"-","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":13},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":13},"outgoing":{"minWireVersion":0,"maxWireVersion":13},"isInternalClient":true}}}
[start-frontend] Building frontend...
[start-database] Released; successful command
[start-backend] Building backend...
[start-frontend] Released; waited for 'http://localhost:4200' returning HTTP 200
[start-backend] Running on localhost:8080
[start-backend] Released; waited for stdout 'Running on localhost:8080'
[start] All processes released; logs are printed; use control+c to stop
[start-backend] /api - 200 OK 

In the end we started our whole infrastructure and application with just one task command. And we receive all logs of all services and are aware if something went wrong.

What do you think of this approach? Can we build something like this into task?

[ghostsquad]

This sounds like docker-compose would do very well for this use case. Have you tried using that?

[tiloio]

@ghostsquad docker-compose is not an option

1. its slower -> waste of valuable development time
2. can't handle watch tasks efficiently
3. less posibilities compared to run the commands directly on your machine (can use already installed software etc.)

[ghostsquad]

Ok, so for clarification, what you need is essentially that of the github.com/oklog/run.Group described below?

Create a zero-value run.Group, and then add actors to it.
Actors are defined as a pair of functions: an execute function, which should run synchronously; and an interrupt function, which, when invoked, should cause the execute function to return.

Finally, invoke Run, which concurrently runs all of the actors, waits until the first actor exits, invokes the interrupt functions, and finally returns control to the caller only once all actors have returned.

This general-purpose API allows callers to model pretty much any runnable task, and achieve well-defined lifecycle semantics for the group.

[tiloio]

That describes how we can run things in parallel, but we maybe have multiple layers of parallelism.

For example:

1. layer                       database                     runs in background & get all stdout and stderr
                              /        \                    some sort of waiting mechanism
2. layer           microservice a      microservice b       runs in background & get all stdout and stderr
                              \        /                    some sort of waiting mechanisms
3. layer                       frontend                     runs in background & get all stdout and stderr
                                   |                        some sort of waiting mechanisms
4. layer                       e2e-tests                    wait for execution finishes or interrupt

The run.Group looks like it can make a group for one layer and a group for all groups of the layers.

[andreynering]

Hi @tiloio, thanks for opening this issue.

This feature seems to be too specific to be built into Task IMO. Few people would use it.

As mentioned, Docker compose is a tool you could consider to achieve that. Or if you want to run these dependencies in our own machine you can either use something like systemd or run them (via Task or not) manually in different shell sessions.

[ghostsquad]

@tiloio I somewhat agree with @andreynering on this, it does kind of feel like what docker-compose is already good at. I've written compose files that do exactly what you are asking for:

Spin up multiple dependencies, finally spin up a client/test container that runs end2end tests. The exit-code of the client/test container is what is passed thru and becomes the exit code of the compose command, allowing me to use this method in CI and other automation.

Though, I am working on another project that might be able to handle this use case. If you would like, I can share my compose file and Taskfile.

[agilgur5]

docker-compose

Compose is very specific for containers as OP mentioned above.

This is more what foreman does (and its various ports like goreman), which does not use containers.

It is something that task runners like make and task are missing though, so I do think it could be good to have something like that built-in for an "all-in-one" experience.
In particular, similar but a bit different from the opening example, if your start-* tasks have multiple dependencies. foreman does not run dependent tasks nor handle sources, env, etc. For example, if start-server also depended on build-server (if you were compiling a binary, say). A task start-all than runs foreman would need to copy the dependencies list of all related tasks such as build-server.

kit is an example of a more recent project that handles both foreground tasks and their dependencies as well as and background tasks and their dependencies.
I would love for task to be able to handle this!

[titpetric]

The following example seems to work to put a docker logs invocation into a background task. The output is still coming from it to stdout which is the point. I didn't fully investigate if anything breaks yet.

cmds:
  - defer: docker compose down --remove-orphans
  - docker compose pull --ignore-pull-failures
  - docker compose up -d --remove-orphans
  - docker compose logs -f &
  - task: benchmark

https://stackoverflow.com/questions/3096561/fork-and-exec-in-bash - it kind of looks like it works, but don't know enough if this is advisable or not, would make a nice little docs section next to defer maybe.

[agilgur5]

docker compose logs -f &

This is the standard POSIX way of doing background processes, but then you have manually manage this. As in, you can't Ctrl+C to kill it, you'll have to manually kill the process when you're done (e.g. kill <process-num>). And in a CI system, backgrounding like this can cause various issues too.

If you're running multiple background services / tasks, it quickly becomes very unwieldy and as a result is something you'd want a task runner to help orchestrate.

The output is still coming from it to stdout which is the point.

IIRC it loses the prefix and colors (and is randomly spliced in your terminal as background processes are), so this becomes a bit hard to follow, especially when you have multiple.

[titpetric]

The color for the logs seem to remain and the process exists when task does.

[agilgur5]

the process exists [sic] when task does.

👀 I wonder if this is because docker compose is no longer running containers after and so logs -f exits on its own? So the process's specific usage matters.
That or the way you called it within task matters?

Whenever I ran background processes, I would complete the task and they would still run in my terminal until I manually killed them

[titpetric]

Yeah, maybe process exists due to the defer task cleanup (e.g. stopping/removing containers). I suppose the process would/could be orphaned otherwise.