Skip to content
This repository has been archived by the owner on Jul 19, 2023. It is now read-only.

Block Select Profiles should be correctly merged across row group #707

Closed
cyriltovena opened this issue May 23, 2023 · 2 comments
Closed
Assignees
Labels
kind/bug Something isn't working

Comments

@cyriltovena
Copy link
Collaborator

cyriltovena commented May 23, 2023

For a given block, between each row group we should also do a k-way merge since the series order is not preserved when flushing to disk.

See https://github.com/grafana/phlare/blob/store-gateway-component/pkg/phlaredb/block_querier.go#L901-L925

This can cause some incorrectness when deduping as we expect data to be streamed in order.

@cyriltovena cyriltovena added the kind/bug Something isn't working label May 23, 2023
@kolesnikovae kolesnikovae self-assigned this May 23, 2023
@cyriltovena
Copy link
Collaborator Author

cyriltovena commented May 24, 2023

I think this requires a big refactoring on parquet iterators, we need a way to produce one iterator per rowgroup. Then use this merge https://github.com/grafana/phlare/blob/63d522a723082453d54b1ece0a462f86887fac66/pkg/iter/profiles.go#LL38C27-L38C27

@cyriltovena
Copy link
Collaborator Author

Actually just looked into this again and we actually don't have a bug, we load all profiles in memory for a given block when selecting profiles and merge them.

see https://github.com/grafana/phlare/blob/main/pkg/phlaredb/block_querier.go#L948-L975

So in the end no bug.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants