Skip to content

Cody littley/db metrics#3010

Merged
cody-littley merged 23 commits intomainfrom
cody-littley/db-metrics
Mar 4, 2026
Merged

Cody littley/db metrics#3010
cody-littley merged 23 commits intomainfrom
cody-littley/db-metrics

Conversation

@cody-littley
Copy link
Contributor

Describe your changes and provide context

Add lots of PebbleDB Metrics

image

Testing performed to validate your change

Ran locally

@github-actions
Copy link

github-actions bot commented Mar 3, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedMar 4, 2026, 3:58 PM

@github-actions
Copy link

github-actions bot commented Mar 3, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedMar 3, 2026, 9:34 PM

limiter = rate.NewLimiter(rate.Limit(scConfig.HistoricalProofRateLimit), burst)
}
scStore := composite.NewCompositeCommitStore(scDir, logger, scConfig)
ctx := context.Background()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any specific reason for adding this context?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new metrics utility collects metrics periodically in a background goroutine. I use the context to manage that goroutine's lifecycle, so it can tear itself down that goroutine when it is time to do so (e.g. at the end of a unit test).

@cody-littley cody-littley requested a review from blindchaser March 3, 2026 21:54
}

return &pebbleDB{db: db}, nil
ctx, cancel := context.WithCancel(context.Background())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks we want to pass context.Context down through the store constructors, do we want to use context.Background() here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I've piped a context into this method.

Copy link
Contributor

@yzang2019 yzang2019 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one comment though is whether adding so many metrics will have performance impact, maybe add a flag and only turn on metrics when it's true

@codecov
Copy link

codecov bot commented Mar 3, 2026

Codecov Report

❌ Patch coverage is 73.13065% with 327 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.36%. Comparing base (62e679f) to head (9018ad2).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
sei-db/common/metrics/pebble_metrics.go 72.76% 326 Missing ⚠️
sei-db/state_db/sc/composite/store.go 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3010      +/-   ##
==========================================
+ Coverage   58.27%   58.36%   +0.08%     
==========================================
  Files        2108     2109       +1     
  Lines      173730   174854    +1124     
==========================================
+ Hits       101248   102054     +806     
- Misses      63451    63775     +324     
+ Partials     9031     9025       -6     
Flag Coverage Δ
sei-chain-pr 63.36% <73.13%> (?)
sei-db 70.41% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
sei-cosmos/storev2/rootmulti/store.go 46.76% <100.00%> (+0.09%) ⬆️
sei-db/db_engine/pebbledb/db.go 93.97% <100.00%> (+0.46%) ⬆️
sei-db/state_db/sc/flatkv/config.go 100.00% <100.00%> (ø)
sei-db/state_db/sc/flatkv/snapshot.go 66.86% <100.00%> (ø)
sei-db/state_db/sc/flatkv/store.go 58.97% <100.00%> (+0.21%) ⬆️
sei-db/state_db/sc/composite/store.go 48.73% <0.00%> (ø)
sei-db/common/metrics/pebble_metrics.go 72.76% <72.76%> (ø)

... and 24 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cody-littley
Copy link
Contributor Author

@yzang2019

LGTM, one comment though is whether adding so many metrics will have performance impact, maybe add a flag and only turn on metrics when it's true

I think this will be safe performance wise, but I agree it's best to be cautiously defense for this type of thing. Added a config flag we can use to disable this new suite of metrics, if needed.

Base automatically changed from cody-littley/cryptosim-otel to main March 4, 2026 15:50
@cody-littley cody-littley enabled auto-merge (squash) March 4, 2026 15:57
@cody-littley cody-littley merged commit 9d23edf into main Mar 4, 2026
39 checks passed
@cody-littley cody-littley deleted the cody-littley/db-metrics branch March 4, 2026 16:12
yzang2019 pushed a commit that referenced this pull request Mar 19, 2026
Add lots of PebbleDB Metrics

<img width="1536" height="1024" alt="image"
src="https://github.com/user-attachments/assets/7215f885-ba07-418a-aa82-ab29e04e1309"
/>

Ran locally

---------

Co-authored-by: Cody Littley <cody.littley@seinetwork.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants