Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: RND-119: Add token usage in inference output #183

Merged
merged 10 commits into from
Sep 4, 2024
Merged

Conversation

niklub
Copy link
Contributor

@niklub niklub commented Aug 15, 2024

Adala table now contains the following system fields:

  • _prompt_tokens
  • _completion_tokens
  • _total_cost_usd

and they're piped to the LSE resulthandler.

TODO need to update all tests again or add some kind of flag which only adds these fields when requested by server

@robot-ci-heartex
Copy link
Collaborator

@robot-ci-heartex
Copy link
Collaborator

@matt-bernstein matt-bernstein marked this pull request as ready for review September 3, 2024 18:39
@robot-ci-heartex
Copy link
Collaborator

adala/runtimes/_litellm.py Show resolved Hide resolved
adala/runtimes/_litellm.py Show resolved Hide resolved
adala/runtimes/_litellm.py Outdated Show resolved Hide resolved
server/handlers/result_handlers.py Outdated Show resolved Hide resolved
@robot-ci-heartex
Copy link
Collaborator

@robot-ci-heartex robot-ci-heartex marked this pull request as draft September 4, 2024 07:03
@robot-ci-heartex
Copy link
Collaborator

@niklub
Copy link
Contributor Author

niklub commented Sep 4, 2024

/git merge master

Workflow run
Successfully merged: 5 files changed, 133 insertions(+), 185 deletions(-)

@robot-ci-heartex
Copy link
Collaborator

@niklub niklub marked this pull request as ready for review September 4, 2024 15:32
@robot-ci-heartex
Copy link
Collaborator

@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 78.46154% with 14 lines in your changes missing coverage. Please review.

Project coverage is 61.57%. Comparing base (459fe95) to head (9c1d952).
Report is 7 commits behind head on master.

Files with missing lines Patch % Lines
adala/runtimes/_litellm.py 76.27% 14 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #183      +/-   ##
==========================================
+ Coverage   61.23%   61.57%   +0.34%     
==========================================
  Files          44       44              
  Lines        2030     2064      +34     
==========================================
+ Hits         1243     1271      +28     
- Misses        787      793       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@niklub niklub merged commit f1cbffb into master Sep 4, 2024
17 of 19 checks passed
@niklub niklub deleted the fb-RND-119 branch September 4, 2024 15:45
@robot-ci-heartex
Copy link
Collaborator

niklub pushed a commit that referenced this pull request Sep 4, 2024
niklub added a commit that referenced this pull request Sep 4, 2024
niklub added a commit that referenced this pull request Sep 4, 2024
Co-authored-by: nik <nik@heartex.net>
Co-authored-by: Matt Bernstein <matt@humansignal.com>
Co-authored-by: niklub <niklub@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants