Flamegraph returned by SelectMergeStacktraces is not stable

[kolesnikovae]

There are noticeable discrepancies in the flamegraphs built for the same query, even if it only covers immutable blocks. The difference may be in the stack trace structure, proportions, and numbers. Sometimes an entire call path may be missing, which is misleading. We need to verify the correctness of the query path.

Screen.Recording.2023-05-12.at.12.00.53.mov

[kolesnikovae]

Might be related: #2067

[cyriltovena]

@aocenas I think this is actually the javascript rendering doing this, it seems that some rounding might cause this. The total time at the top never changes.

EDIT: There's potential multiple bug, but I'd like to make sure this is fine on the frontend.

[aocenas]

@cyriltovena hmm I remember something like this when we used the pprof format. At that point the response was unstable and I added a sort but still some things switched places. The reason was that some labels are the same but they had different binary/address so they weren't logically the same thing and were represented by 2 separate node next to each other but with the same label. If we sorted by label that would still be unstable. This seems like the same case but we don't have the sort in pyroscope data source anymore so I assume this could be an issue in the Pyroscope API or somewhere where this is sorted?