Skip to content

S3 snapshot metrics #74

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 19, 2024
Merged

S3 snapshot metrics #74

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
7307b83
Metrics for snapshot count and total size
nvaatstra Feb 20, 2024
fbef149
Use (temp) simpleblob main branch and add storage bytes recording
nvaatstra Feb 22, 2024
54737bd
Improved name for snapshots bytes guage
nvaatstra Feb 22, 2024
d027906
Prometheus metrics in adherence with best practices; units as last co…
nvaatstra Feb 23, 2024
59fc0e6
Simpleblob to release 0.2.7
nvaatstra Mar 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ go 1.19
require (
github.com/CrowdStrike/csproto v0.23.1
github.com/PowerDNS/lmdb-go v1.9.0
github.com/PowerDNS/simpleblob v0.2.5
github.com/PowerDNS/simpleblob v0.2.7
github.com/bufbuild/buf v0.56.0
github.com/c2h5oh/datasize v0.0.0-20200825124411-48ed595a09d2
github.com/gogo/protobuf v1.3.2
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@ github.com/PowerDNS/go-tlsconfig v0.0.0-20221101135152-0956853b28df h1:WMUClevRP
github.com/PowerDNS/go-tlsconfig v0.0.0-20221101135152-0956853b28df/go.mod h1:aKP0MVHWl7U3ruSXqAmzsoVVFm8HhYIpOmm1MwkDyDY=
github.com/PowerDNS/lmdb-go v1.9.0 h1:pvdI8lzdeAfWJdkXc3XPZXCewX508awqfe5yMjSKNTE=
github.com/PowerDNS/lmdb-go v1.9.0/go.mod h1:5beHlX2aYqXfMBI0+BBX3LDhV3YGHU5JgoDsirAFPlA=
github.com/PowerDNS/simpleblob v0.2.5 h1:JI3pGI8KyzDXDL75ae5QPCw5588X6z6/nBzehiCe2TE=
github.com/PowerDNS/simpleblob v0.2.5/go.mod h1:demorqIrYIbxW3wsU5SS5NufPyGseDW+xofeQZw4w3o=
github.com/PowerDNS/simpleblob v0.2.7 h1:eslNYitaaaOgIqZdYWnMH1nJIc6HApIvBs63zzKBYQE=
github.com/PowerDNS/simpleblob v0.2.7/go.mod h1:demorqIrYIbxW3wsU5SS5NufPyGseDW+xofeQZw4w3o=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
Expand Down
17 changes: 16 additions & 1 deletion syncer/receiver/metrics.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,20 @@ var (
Help: "Number of bytes downloaded successfully",
},
)
// TODO: add total space used by all snapshots
metricSnapshotsStorageCount = prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "lightningstream_receiver_snapshots_in_storage_count",
Help: "Number of snapshots in storage",
},
[]string{"lmdb"},
)
metricSnapshotsStorageBytes = prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "lightningstream_receiver_snapshots_in_storage_bytes",
Help: "Number of bytes occupied by snapshots in storage",
},
[]string{"lmdb"},
)
)

func init() {
Expand All @@ -74,4 +87,6 @@ func init() {
prometheus.MustRegister(metricSnapshotsLoadFailed)
prometheus.MustRegister(metricSnapshotsListFailed)
prometheus.MustRegister(metricSnapshotsLoadBytes)
prometheus.MustRegister(metricSnapshotsStorageCount)
prometheus.MustRegister(metricSnapshotsStorageBytes)
}
4 changes: 4 additions & 0 deletions syncer/receiver/receiver.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,10 @@ func (r *Receiver) RunOnce(ctx context.Context, includingOwn bool) error {
return fmt.Errorf("list snapshots: %w", err)
}

// Update snapshot metrics
metricSnapshotsStorageCount.WithLabelValues(r.lmdbname).Set(float64(ls.Len()))
metricSnapshotsStorageBytes.WithLabelValues(r.lmdbname).Set(float64(ls.Size()))

// Signal success to health tracker
r.storageListHealth.AddSuccess()

Expand Down