Improve docker-compose logging

[Felixoid]

Changelog category (leave one):

• Build/Testing/Packaging Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Do not collect the docker-compose logs at the end of running since the script is often killed. Instead, collect them in the background.

Documentation entry for user-facing changes

• Documentation is written (mandatory for new features)

[clickhouse-gh]

Workflow [PR], commit [e91ec72]

[Felixoid]

Here's the output for pgrep on a runner at the parallel tests launch:

$ pgrep -af docker-compose | tail -4
3588986 docker compose --env-file /ClickHouse/tests/integration/test_backup_restore_new/_instances-0-gw1/.env --project-name roottestbackuprestorenew-gw1 --file /ClickHouse/tests/integration/test_backup_restore_new/_instances-0-gw1/instance/docker-compose.yml logs --follow
3589140 /usr/libexec/docker/cli-plugins/docker-compose compose --env-file /ClickHouse/tests/integration/test_backup_restore_new/_instances-0-gw1/.env --project-name roottestbackuprestorenew-gw1 --file /ClickHouse/tests/integration/test_backup_restore_new/_instances-0-gw1/instance/docker-compose.yml logs --follow
3619643 docker compose --env-file /ClickHouse/tests/integration/test_dictionaries_all_layouts_separate_sources/_instances-mongo_uri-0-gw4/.env --project-name roottestdictionariesalllayoutsseparatesourcesmongouri-gw4 --file /ClickHouse/tests/integration/test_dictionaries_all_layouts_separate_sources/_instances-mongo_uri-0-gw4/uri_node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_mongo.yml logs --follow
3619671 /usr/libexec/docker/cli-plugins/docker-compose compose --env-file /ClickHouse/tests/integration/test_dictionaries_all_layouts_separate_sources/_instances-mongo_uri-0-gw4/.env --project-name roottestdictionariesalllayoutsseparatesourcesmongouri-gw4 --file /ClickHouse/tests/integration/test_dictionaries_all_layouts_separate_sources/_instances-mongo_uri-0-gw4/uri_node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_mongo.yml logs --follow

Now, we'll have all the docker compose logs, regardless of whether the runner process was killed or not.

[azat]

Killer feature!

P.S. why does it have changes for Kafka here?

[azat]

Should we kill this process properly as well?

[Felixoid]

No, the docker compose logs --follow dies with exit code 0 when the compose services finish:

> docker compose up -d
[+] Running 5/5
 ✔ Container graphite-clickhouse-tldr-carbon-clickhouse-1    Started                                                                               0.5s
 ✔ Container graphite-clickhouse-tldr-carbonapi-1            Started                                                                               0.4s
 ✔ Container graphite-clickhouse-tldr-clickhouse-1           Started                                                                               0.5s
 ✔ Container graphite-clickhouse-tldr-grafana-1              Started                                                                               0.4s
 ✔ Container graphite-clickhouse-tldr-graphite-clickhouse-1  Started                                                                               0.5s
> docker compose logs -f &> docker.logs &
[1] 2323019
> docker compose stop
[+] Stopping 5/5
 ✔ Container graphite-clickhouse-tldr-grafana-1              Stopped                                                                               0.4s
 ✔ Container graphite-clickhouse-tldr-clickhouse-1           Stopped                                                                               2.5s
 ✔ Container graphite-clickhouse-tldr-carbonapi-1            Stopped                                                                               0.3s
 ✔ Container graphite-clickhouse-tldr-graphite-clickhouse-1  Stopped                                                                               0.4s
 ✔ Container graphite-clickhouse-tldr-carbon-clickhouse-1    Stopped                                                                               0.5s
[1]  + done       docker compose logs -f &> docker.logs
> tail -5 docker.logs
carbonapi-1 exited with code 0
grafana-1 exited with code 0
graphite-clickhouse-1 exited with code 0
carbon-clickhouse-1 exited with code 1
clickhouse-1 exited with code 0

[azat]

I know, but still looks odd, not to wait the process properly

[Felixoid]

We shouldn't wait for it, that's the case.

It's a background process writing the logs in realtime.

And it's killed on it's own, when the tests' docker compose is finished

[azat]

We shouldn't wait for it, that's the case.

This is minor, in terms of the behavior, but leaving processes without waiting is bad

For instance it could be the case that docker-compose has some buffering for logs and without properly waiting for it we cannot be sure that everything is dumped

[Felixoid]

I see. Yeah, we can wait for it in shutdown

[Felixoid]

P.S. why does it have changes for Kafka here?

I've prepared something for fixing #69452 and tests in a follow-up. It's just reordering for now to have the logs fixed.

[Felixoid]

You know, I'd really move splitting kafka tests into the different PR #76208