Parquet WAL

[mdisibio]

What this PR does:
This PR introduces support for a parquet-formatted WAL with the goal to support TraceQL search on WALs (which is currently infeasible on v2). The WAL format is controlled by a new storage.Trace.WAL.Version config which is similar to the one used for backend blocks storage.Trace.Block.Version. Setting it to vParquet enables to the new WAL format, default is still v2. All combinations of WAL and Block versions are supported, but performance is best when they match.

Design
Parquet WALs break from the existing design where the WAL is an open file to which new data is appended as it is received. Now a parquet WAL is a folder with a series of sub-files, like this:

• path/wal/<blockID>+<tenantID>+vParquet
  ** meta.json
  ** 00001
  ** 00002
  ** 00003
  ** 00004

Each file is a standalone parquet file, created each time the ingester flushes idle traces from memory to disk (every few seconds). From there the sub-files are identical in format to existing parquet blocks and all the same functionality is reused: search, find trace by id, tag lookup, and soon: traceql. There is also an operational benefit that the standalone files are easy to inspect and troubleshoot with existing parquet tooling.

Refactor
This PR includes a refactor of the tempodb/wal module to support different encodings and is similar to what was done for the backend. A generic interface for a WAL block was introduced in tempodb/encoding/common and now both v2 and vParquet encodings implement it.

Performance Impact
As expected there is an increase in resources for this format, ~2x cpu and ~2x memory. I don't think it can ever be as efficient as v2 which was basically copying bytes from memory to disk, but there are several areas we can explore to mitigate this - especially the case of parquet wal -> parquet block.

TODOs
This PR is stable and usable, and worth merging now since it is already quite large. However there are several TODOs which make sense to do in follow-up PRs:

• Wire up the TraceQL read path to the ingesters (and these new WAL blocks)
• Test replay with partially-written/corrupted files
• Improve performance of parquet wal -> parquet block step. Maybe we can do the same as compaction and copy raw rows between blocks instead of Trace objects. (And use buffer pooling to further reduce memory overhead) Also change the multi-block iterator to not simultaneously buffer the next Trace from all inputs. (i.e. 100 wal sub-files means 100 traces resident in memory at the same time during completion process)

Which issue(s) this PR fixes:
Fixes #

Checklist

• Tests updated
• Documentation added
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

[mapno]

LGTM 👍

Left only a comment, everything else looks ok. But, I may not be the best to review this code, so take it with a grain of salt 😅