Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: NTRN-204 enhance subscriber monitoring #48

Merged
merged 7 commits into from
Dec 7, 2022

Conversation

foxpy
Copy link
Contributor

@foxpy foxpy commented Oct 23, 2022

Tracking task: NTRN-204.

}

func DecQueriesToProcess() {
queriesToProcess.With(prometheus.Labels{}).Inc()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.Inc()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤡

@@ -73,6 +73,7 @@ func (r *Relayer) Run(ctx context.Context, tasks <-chan neutrontypes.RegisteredQ
)
select {
case query := <-tasks:
neutronmetrics.SetSubscriberTaskQueueNumElements(len(tasks))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it belongs in the relayer, better move it to subscriber

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, I see the same track in the Subscriber. what's the point of two of them? isn't one enough measuring the len just before sending to the queue? also, as far as I can see, both tracks will have the same result since they will occur in the same time — on read from channel

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea how Golang scheduling works under the hood, hence I have placed calls to metrics in both places. What do you think is wise to do in this situation?

}

func IncQueriesToProcess() {
queriesToProcess.With(prometheus.Labels{}).Inc()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not a gauge with .Set(subscriber.activeQueries)? will simplify the if !queryExists part in the processUpdateEvent

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In tracking task it is required to use a counter. If more people from team think it is better to use gauge here — I will switch.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree with @sotnikov-s
since @foxpy is unavailable now, i can fix it

out[restQuery.ID] = neutronQuery
instrumenters.IncQueriesToProcess()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't get why doubled inc?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's definitely a mistake!

@foxpy foxpy requested a review from sotnikov-s November 4, 2022 08:47
zavgorodnii
zavgorodnii previously approved these changes Nov 7, 2022
@zavgorodnii zavgorodnii dismissed their stale review November 7, 2022 08:23

changed my mind

swelf added 2 commits November 25, 2022 16:56
# Conflicts:
#	internal/relay/relayer.go
#	internal/subscriber/subscriber.go
#	internal/subscriber/utils.go
#	internal/txsubmitchecker/txsubmitchecker.go
internal/subscriber/subscriber.go Show resolved Hide resolved
internal/relay/relayer.go Show resolved Hide resolved
cmd/neutron_query_relayer/metrics/metrics.go Show resolved Hide resolved
cmd/neutron_query_relayer/metrics/metrics.go Outdated Show resolved Hide resolved
internal/subscriber/subscriber.go Outdated Show resolved Hide resolved
internal/subscriber/utils.go Outdated Show resolved Hide resolved
Copy link
Contributor

@sotnikov-s sotnikov-s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. tested: metrics are counted up and down properly

internal/subscriber/subscriber.go Show resolved Hide resolved
@zavgorodnii zavgorodnii changed the title Feat: enhance subscriber monitoring feat: NTRN-204 enhance subscriber monitoring Dec 6, 2022
Copy link
Contributor

@NeverHappened NeverHappened left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, and integration tests for kv and tx passed

@zavgorodnii zavgorodnii merged commit 1bf81d9 into main Dec 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants