Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add search block headers #943

Merged
merged 9 commits into from
Sep 15, 2021
Merged

Conversation

mdisibio
Copy link
Contributor

@mdisibio mdisibio commented Sep 9, 2021

What this PR does:
Adds block-level search data which speeds up some searches by eliminating blocks faster. The block header contains the superset of unique tags and duration range within the block. Data is stored in a new file search-header with a dedicated flatbuffer struct.

  • Other notes:
    • Minor refactoring of search pipeline to be more structured around blocks/pages/traces.
    • Previously it was possible to perform exhaustive searches and benchmarks using unmatching condition like x=y. This is no longer possible since it eliminates the block immediately due to tag not present in the header. Since we still want these abilities it was readded with a secret tag x-dbg-exhaustive. If this tag is present in the search then it will evaluate all data again.

This is likely to conflict with #940 so let's merge that first.

Which issue(s) this PR fixes:
Fixes part of #932

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Signed-off-by: Martin Disibio <mdisibio@gmail.com>
Signed-off-by: Martin Disibio <mdisibio@gmail.com>
Signed-off-by: Martin Disibio <mdisibio@gmail.com>
Signed-off-by: Martin Disibio <mdisibio@gmail.com>
Signed-off-by: Martin Disibio <mdisibio@gmail.com>
tempodb/search/backend_search_block.go Outdated Show resolved Hide resolved
tempodb/search/backend_search_block.go Outdated Show resolved Hide resolved
tempodb/search/streaming_search_block.go Outdated Show resolved Hide resolved
@mdisibio mdisibio changed the title WIP: Add search block headers Add search block headers Sep 14, 2021
@mdisibio mdisibio linked an issue Sep 14, 2021 that may be closed by this pull request
12 tasks
Signed-off-by: Martin Disibio <mdisibio@gmail.com>
Signed-off-by: Martin Disibio <mdisibio@gmail.com>
Copy link
Contributor

@annanay25 annanay25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

tempodb/search/backend_search_block_test.go Show resolved Hide resolved
@mdisibio mdisibio merged commit 0e7e7aa into grafana:main Sep 15, 2021
@mdisibio mdisibio deleted the search-block-headers branch September 15, 2021 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ingester search production readiness
2 participants