feat(ai): add AI gateway metrics #3087
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rickstaa
force-pushed
the
ai-gateway-metrics-poc
branch
2 times, most recently
from
7302537
to
a456c04
Compare
rickstaa
force-pushed
the
ai-gateway-metrics-poc
branch
from
7e00384
to
47c088f
Compare
rickstaa
commented
Jul 8, 2024
eliteprox
reviewed
Jul 8, 2024
rickstaa
commented
Jul 9, 2024
5 tasks
rickstaa
force-pushed
the
ai-gateway-metrics-poc
branch
from
2983d08
to
a92e9df
Compare
This commit adds the initial AI gateway metrics so that they can reviewed by others. The code still need to be cleaned up and the buckets adjusted.
This commit improves the AI metrics so that they are easier to work with.
This commit ensures that an error is logged when the Gateway could not find orchestrators for a given model and capability.
This commit ensure that the `ticket_value_sent` abd `tickets_sent` metrics are also created for a AI Gateway.
This commit ensures that the AI gateway metrics contain the orch address label.
This commit ensures that the AI job pricing is calculated correctly and cleans up the codebase.
rickstaa
force-pushed
the
ai-gateway-metrics-poc
branch
from
a92e9df
to
0970d4f
Compare
This commit removes the Orch label from the ai_request_price metrics since that information is better to be retrieved from another endpoint.
eliteprox
added a commit
to eliteprox/go-livepeer
that referenced
this pull request
Jul 26, 2024
* Add gateway metric for roundtrip ai times by model and pipeline * Rename metrics and add unique manifest * Fix name mismatch * modelsRequested not working correctly * feat: add initial POC AI gateway metrics This commit adds the initial AI gateway metrics so that they can reviewed by others. The code still need to be cleaned up and the buckets adjusted. * feat: improve AI metrics This commit improves the AI metrics so that they are easier to work with. * feat(ai): log no capacity error to metrics This commit ensures that an error is logged when the Gateway could not find orchestrators for a given model and capability. * feat(ai): add TicketValueSent and TicketsSent metrics This commit ensure that the `ticket_value_sent` abd `tickets_sent` metrics are also created for a AI Gateway. * fix(ai): ensure that AI metrics have orch address label This commit ensures that the AI gateway metrics contain the orch address label. * fix(ai): fix incorrect Gateway pricing metric This commit ensures that the AI job pricing is calculated correctly and cleans up the codebase. * refactor(ai): remove Orch label from ai_request_price metric This commit removes the Orch label from the ai_request_price metrics since that information is better to be retrieved from another endpoint. --------- Co-authored-by: Elite Encoder <john@eliteencoder.net>
eliteprox
added a commit
to eliteprox/go-livepeer
that referenced
this pull request
Jul 26, 2024
* Add gateway metric for roundtrip ai times by model and pipeline * Rename metrics and add unique manifest * Fix name mismatch * modelsRequested not working correctly * feat: add initial POC AI gateway metrics This commit adds the initial AI gateway metrics so that they can reviewed by others. The code still need to be cleaned up and the buckets adjusted. * feat: improve AI metrics This commit improves the AI metrics so that they are easier to work with. * feat(ai): log no capacity error to metrics This commit ensures that an error is logged when the Gateway could not find orchestrators for a given model and capability. * feat(ai): add TicketValueSent and TicketsSent metrics This commit ensure that the `ticket_value_sent` abd `tickets_sent` metrics are also created for a AI Gateway. * fix(ai): ensure that AI metrics have orch address label This commit ensures that the AI gateway metrics contain the orch address label. * fix(ai): fix incorrect Gateway pricing metric This commit ensures that the AI job pricing is calculated correctly and cleans up the codebase. * refactor(ai): remove Orch label from ai_request_price metric This commit removes the Orch label from the ai_request_price metrics since that information is better to be retrieved from another endpoint. --------- Co-authored-by: Elite Encoder <john@eliteencoder.net>
2 tasks
This was referenced Aug 15, 2024
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this pull request do? Explain your changes. (required)
This pull request introduces new Gateway AI metrics to the
ai-videobranch:Specific updates (required)
census.goto include the new Gateway metrics.ai_processto log these metrics.How did you test each of these updates (required)
I set up both an on-chain and off-chain gateway to validate the metrics. I verified their visibility at
http://localhost:5935/metricsand ensured they were correctly visualized in Grafana.Does this pull request close any open issues?
This implements the functionality outlined in https://livepeer-ai.productlane.com/roadmap?id=58c9cd5d-1c64-4fb3-b8d0-a7e20b7865a2.
Checklist:
makeruns successfully./test.shpassHow to test
http://localhost:5935/metricsto view the new AI Gateway metrics.http://localhost:3000to inspect these metrics in Grafana.