Add support for logging to stdout/stderr

[derekperkins]

We are running ClickHouse inside Kubernetes, which prefers all logs be written to stdout and stderr. It would be great to have that as a supported logger setting that would just write out logs and not handle any log rotation.

[proller]

Remove the following elements from your config:

<log>/var/log/clickhouse-server/clickhouse-server.log</log>              
<errorlog>/var/log/clickhouse-server/clickhouse-server.err.log</errorlog>

And add (to section)

<console>1</console>

[proller]

And simpler redefining logger config keys:
89b67dd

[derekperkins]

That helped some but didn't fully work for me. After applying these logger settings:

<logger>
    <level>trace</level>
    <console>1</console>
</logger>

I saw this in my error log:

2018.04.19 18:20:59.196532 [ 28 ] <Warning> Application: Logging trace to console

I started to see some error logs in the console output, but it also continued to write some logs (but not all) to /var/log/clickhouse-server/clickhouse-server.err.log. These are the only two logs I saw still being output to the error log after making the config change, but it's possible that there are more.

2018.04.19 18:30:37.794871 [ 3583 ] <Warning> TCPHandler: Client has not sent any data.
2018.04.19 18:30:40.782907 [ 80 ] <Error> MySQLDictionarySource: Poco::Exception. Code: 1000, e.code() = 0, e.displayText() = Exception: Connections to all replicas failed: clickhouse@mysql:3306 as user clickhouse, e.what() = Exception

It also continued to write to /var/log/clickhouse-server/clickhouse-server.log with no obvious difference. Do I need to restart the server to fully apply these logging changes?

[proller]

Yes, better to restart server.

[derekperkins]

After restarting, logs appear to be only routing to stdout. Thanks for your help!

[megastef]

Thank you for the workaround above. I developed a CH log parser with multi line parsing of CH logs, which works also for containers.

I wonder why these logger settings are not enabled by default in the docker specific configuration: https://github.com/yandex/ClickHouse/blob/master/docker/server/docker_related_config.xml

Is there a plan to enable console logs by default - many logging tools collect logs from container console output.

[corpix]

Corresponding to https://clickhouse.tech/docs/en/operations/server-configuration-parameters/settings/ this is not working anymore

UPD

Hacky workaround:

<log>/proc/self/fd/1</log>
<errorlog>/proc/self/fd/2</errorlog>

But it would be better if logging to stdout&stderr will be simpler than this.

[scraton]

The following worked for me with ClickHouse v21.7:

<logger>
    <console>true</console>
    <log remove="remove"/>
    <errorlog remove="remove"/>
</logger>

Source: https://github.com/ClickHouse/ClickHouse/blob/b89da7b420ada3eae6659b556c6cc3e2158a69a8/tests/integration/test_config_xml_full/configs/config.d/log_to_console.xml