-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Return highest scoring response to the API #124
Comments
This should be fixed by changing the return object from the list to the first element here right? print("FINAL RESPONSES ------------------------------------------------")
print(response_with_uids)
- return response_with_uids
+ return response_with_uids[0] (Aside from the fact that the return type should be changed as well) |
Isn't this blocked also by masa-bittensor/issues#197? Or could we fix it and then have the scoring be worked on in parallel 🤔 (I think we can but just checking here) |
this is low hanging fruit, and what we could actually do is mutate the behavior such having a debug flag that returns all the scores, and when in product mode, returns only the highest score. Outcome from planning: even better to have a separate endpoint and avoiding any user option, or maybe a parameter to the API endpoint |
From an API perspective, we want to return what the validator judges as the "best" response, or highest scoring, not a list of sanitized responses based on pre-defined types.
The text was updated successfully, but these errors were encountered: