Upload build time-trace data to CI database

[alexey-milovidov]

Changelog category (leave one):

• Build/Testing/Packaging Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

The compiler's profile data (-ftime-trace) is uploaded to ClickHouse Cloud.

[robot-ch-test-poll4]

This is an automated comment for commit b397d11 with description of existing statuses. It's updated for the latest CI running
The full report is available here
The overall status of the commit is 🔴 failure

Check name	Description	Status
AST fuzzer	Runs randomly generated queries to catch program errors. The build type is optionally given in parenthesis. If it fails, ask a maintainer for help	🔴 failure
CI running	A meta-check that indicates the running CI. Normally, it's in success or pending state. The failed status indicates some problems with the PR	🟡 pending
Fast test	Normally this is the first check that is ran for a PR. It builds ClickHouse and runs most of stateless functional tests, omitting some. If it fails, further checks are not started until it is fixed. Look at the report to see which tests fail, then reproduce the failure locally as described here	🟢 success
Flaky tests	Checks if new added or modified tests are flaky by running them repeatedly, in parallel, with more randomization. Functional tests are run 100 times with address sanitizer, and additional randomization of thread scheduling. Integrational tests are run up to 10 times. If at least once a new test has failed, or was too long, this check will be red. We don't allow flaky tests, read the doc	🟢 success
Mergeable Check	Checks if all other necessary checks are successful	🔴 failure
Performance Comparison	Measure changes in query performance. The performance test report is described in detail here. In square brackets are the optional part/total tests	🟢 success
Push to Dockerhub	The check for building and pushing the CI related docker images to docker hub	🟢 success
Stateful tests	Runs stateful functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc	🟢 success
Style Check	Runs a set of checks to keep the code style clean. If some of tests failed, see the related log from the report	🟢 success
Unit tests	Runs the unit tests for different release types	🟢 success
Upgrade check	Runs stress tests on server version from last release and then tries to upgrade it to the version from the PR. It checks if the new server can successfully startup without any errors, crashes or sanitizer asserts	🔴 failure

[Felixoid]

it's unnecessary change, we don't use additional stubs

[alexey-milovidov]

But mypy told me about it in clear text.

[Felixoid]

I think, the goal is to use the same DB as for checks, so https://github.com/ClickHouse/ClickHouse/blob/master/tests/ci/clickhouse_helper.py#L18 would be extended

[alexey-milovidov]

No, we will use a separate DB because the DB with checks is capped at about 3.6 TB, while ClickHouse Cloud is unlimited.

[Felixoid]

We usually use the SSM for this purpose, and this function https://github.com/ClickHouse/ClickHouse/blob/master/tests/ci/get_robot_token.py#L18

[alexey-milovidov]

CREATE TABLE build_time_trace
(
    pull_request_number UInt32,
    commit_sha String,
    check_start_time DateTime,
    check_name LowCardinality(String),
    instance_type LowCardinality(String),
    file String,
    library LowCardinality(String),
    date Date DEFAULT toDate(time),
    time DateTime64(6),
    
    pid UInt32,
    tid UInt32,
    ph Enum8('B', 'E', 'X', 'i', 'I', 'C', 'b', 'n', 'e', 'S', 'T', 'p', 'F', 's', 't', 'f', 'P', 'N', 'O', 'D', 'M'),
    ts UInt64,
    dur UInt64,
    cat LowCardinality(String),
    name LowCardinality(String),
    detail String,
    count UInt64,
    avgMs UInt64,
    args_name LowCardinality(String),
    is_total Bool DEFAULT name LIKE 'Total %'
)
ENGINE = MergeTree ORDER BY (date, file, name, args_name);

[alexey-milovidov]

Ok. It works, and the amount of changes is minimal.
Next step: change it to use SSM agent, but without breaking it.

[Felixoid]

I don't get the purpose of the review if it's just ignored completely and merged.

A new image has broken everything again