Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Homogenize Generator meta output #7687

Open
nachollorca opened this issue May 13, 2024 · 3 comments
Open

Homogenize Generator meta output #7687

nachollorca opened this issue May 13, 2024 · 3 comments
Labels
Contributions wanted! Looking for external contributions

Comments

@nachollorca
Copy link

Is your feature request related to a problem? Please describe.
The dictionaries returned by generators have matching keys to an extent, but some are using different notations for the same concept, mainly within the metadata / usage keys. For example:

  • OpenAI generators use prompt_tokens/completion tokens, while Cohere/Anthropic/AmazonBedrock use input_tokens/output_tokens.
  • Some generators use finish_reason, while others use stop_reason

Describe the solution you'd like
I would like generators to output the same scheme and keys to the extent that it is possible, i.e. for matching concepts. In particular, I would prefer to use input_tokens/output_tokens and stop_reason.

Describe alternatives you've considered
So far, in apps where we swap generators depending on the use case, the alternative is:

meta = pipe_result["answer_generator"]["replies"][0].meta
usage = meta["usage"]

output.input_tokens = usage["prompt_tokens"] if "prompt_tokens" in usage else usage["input_tokens"]
output.output_tokens = usage["completion_tokens"] if "completion_tokens" in usage else usage["output_tokens"]
output.finish_reason = meta["finish_reason"] if "finish_reason" in meta else meta["stop_reason"]`
@vblagoje
Copy link
Member

Taking into account tendency for other LLM providers/libraries to converge on OpenAI API - let's got with OpenAI naming scheme.

@nachollorca
Copy link
Author

Alright, I'll go for OpenAI's convention :)

@masci masci added the Contributions wanted! Looking for external contributions label May 31, 2024
@CarlosFerLo
Copy link
Contributor

Been having a look at all generators and all the keywords seem to keep the OpenAI standard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Contributions wanted! Looking for external contributions
Projects
Development

No branches or pull requests

4 participants