Add optional backend for ClickHouse storage

[danipozo]

Adds an optional flag to enable storage of traces in a ClickHouse server. This enables better scalability for users who wish to ingest and query data from many nodes, as the current FrostDB storage struggles with heavy use cases.

This introduces a new module implementing the same public interface as the parcacol module, based on FrostDB, and initializes the selected backend depending on the flag. Traces are inserted into and queried from a MergeTree table, the basic engine in ClickHouse. This could be modified to be a ReplicatedMergeTree to enable more complex setups. For the labels, it uses a column with the JSON type, which stores most frequent labels in separate columns for query efficiency (this was a design objective of FrostDB AFAIK).

Caveats

1. The ingester assumes that every location from the profile contains only one line, as I've only seen that case and I'm not sure the extra complexity is worth introducing right now. For addresses with inlined functions in native binaries, they are ingested as one line unsymbolized and then the inlined functions are extracted by the symbolizer.
2. I haven't reviewed in depth the code that actually generates flamegraph structures, as it may be worth to push that computation to ClickHouse too, but I think it would be too much scope for this PR anyway.

[danipozo]

cc @thorfour @umanwizard

[thorfour]

cc @thorfour @umanwizard

Awesome! Thanks for this. Will review it next week

[thorfour]

This looks great! Thanks for the contribution.

I would like to mark this as an experimental feature for now. As such can we move the clickhouse flags into the FlagsHidden struct so that they aren't not displayed in --help by default?

[danipozo]

Sure! Done in the latest commit

[danipozo]

Hey, kindly pinging, if you get a chance to take another look 👀

[thorfour]

Thanks for the contribution!

[brancz]

I just wanted to point out, this is experimental and hidden, there is a possible future where we might do a major overhaul of Parca, and it's possible that in an effort like that this type of feature may be dropped. Nevertheless, we want to enable users in the meantime to do what they would like, but there is no guarantee for the longevity of this feature.

[danipozo]

there is a possible future where we might do a major overhaul of Parca

It would be interesting to know more about this, do you have some kind of public roadmap?

[brancz]

There isn't anything public yet since we haven't fully made up our mind, but our internal systems at Polar Signals have dramatically diverged from Parca to the point where Parca maintenance has suffered since we are maintaining effectively two projects that achieve much of the same things (hence why we're ok with merging things like this even though it's very far from our original goal of having a single statically linked binary akin to Prometheus for profiling).

An idea has emerged that maybe we will be open-sourcing some of the work we've done at Polar Signals, but that would mean very dramatic changes in Parca, since we've practically fully moved to rust at Polar Signals.