Expose all TSDB metrics in the ingester #3412
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pracucci
commented
Oct 28, 2020
| @@ -1858,12 +1858,12 @@ func TestIngester_flushing(t *testing.T) { | |||
| i.FlushHandler(httptest.NewRecorder(), httptest.NewRequest("POST", "/flush", nil)) | |||
|
|
|||
| // Flush handler only triggers compactions, but doesn't wait for them to finish. Let's wait for a moment, and then verify. | |||
| test.Poll(t, 1*time.Second, true, func() interface{} { | |||
| test.Poll(t, 5*time.Second, uint64(0), func() interface{} { | |||
There was a problem hiding this comment.
This test was flaky again on my local machine, so I had:
- Increase polling timeout from 1s to 5s
- Compared on the actual number, so we can see what's the value of
NumSeries()when it fails
codesome
approved these changes
Oct 28, 2020
CHANGELOG.md
Outdated
| @@ -89,6 +89,11 @@ | |||
| * [ENHANCEMENT] Blocks storage: reduced number of bucket listing operations to list block content (applies to newly created blocks only). #3363 | |||
| * [ENHANCEMENT] Ruler: Include the tenant ID on the notifier logs. #3372 | |||
| * [ENHANCEMENT] Blocks storage Compactor: Added `-compactor.enabled-tenants` and `-compactor.disabled-tenants` to explicitly enable or disable compaction of specific tenants. #3385 | |||
| * [ENHANCEMENT] Blocks storage ingester: exported more TSDB-related metrics. #3412 | |||
There was a problem hiding this comment.
👋 The v1.5 release just started and you will need to rebase to the latest master and move any CHANGELOG entries into the unreleased section. Thanks!
Signed-off-by: Marco Pracucci <marco@pracucci.com>
pracucci
force-pushed
the
expose-all-tsdb-metrics
branch
from
e86a923 to
a786ac0
Compare
gouthamve
approved these changes
Oct 29, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does:
In past few days we experienced TSDB WAL corruption twice. Given the root cause investigation (and possible fix) is a separate issue, I've also realised we're not mapping all TSDB metrics so in this PR I'm fixing this gap.
Which issue(s) this PR fixes:
N/A
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]