fix: polishing raise_exception and max_workers
#486
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jjmachan
commented
Jan 19, 2024
Comment on lines
+51
to
+74
| llm: BaseRagasLLM, optional | ||
| The language model to use for the metrics. If not provided then ragas will use | ||
| the default language model. This can we overridden by the llm specified in | ||
| the metric level with `metric.llm`. | ||
| embeddings: BaseRagasEmbeddings, optional | ||
| The embeddings to use for the metrics. If not provided then ragas will use | ||
| the default embeddings. This can we overridden by the embeddings specified in | ||
| the metric level with `metric.embeddings`. | ||
| callbacks: Callbacks, optional | ||
| Lifecycle Langchain Callbacks to run during evaluation. Check the | ||
| [langchain documentation](https://python.langchain.com/docs/modules/callbacks/) | ||
| for more information. | ||
| is_async: bool, optional | ||
| Whether to run the evaluation in async mode or not. If set to True then the | ||
| evaluation is run by calling the `metric.ascore` method. In case the llm or | ||
| embeddings does not support async then the evaluation can be run in sync mode | ||
| with `is_async=False`. Default is False. | ||
| max_workers: int, optional | ||
| The number of workers to use for the evaluation. This is used by the | ||
| `ThreadpoolExecutor` to run the evaluation in sync mode. | ||
| raise_exceptions: bool, optional | ||
| Whether to raise exceptions or not. If set to True then the evaluation will | ||
| raise an exception if any of the metrics fail. If set to False then the | ||
| evaluation will return `np.nan` for the row that failed. Default is True. |
There was a problem hiding this comment.
the new capabilities have been documented
shahules786
approved these changes
Jan 20, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
raise_exceptionwhich can control if you want to raise an exception or return np.nan if an error occursmax_workerscan used control how many requests you do - incase you run into rate limits with ur endpoints