Contradiction in evaluate is_async parameter docstring and code

[dschwalm]

[ x] I checked the documentation and related resources and couldn't find an answer to my question.

Your Question
Contradiction in evaluate is_async parameter. By default this parameter is True, but according to the documentation, it is False by default. Either the doc or the code is wrong.
I am happy to raise a PR once the decision is made.

Code Examples

`def evaluate(
dataset: Dataset,
metrics: list[Metric] | None = None,
llm: t.Optional[BaseRagasLLM | LangchainLLM] = None,
embeddings: t.Optional[BaseRagasEmbeddings | LangchainEmbeddings] = None,
callbacks: Callbacks = None,
in_ci: bool = False,
is_async: bool = True,
run_config: t.Optional[RunConfig] = None,
raise_exceptions: bool = True,
column_map: t.Optional[t.Dict[str, str]] = None,
) -> Result:
"""
Run the evaluation on the dataset with different metrics

Parameters
----------
dataset : Dataset[question: list[str], contexts: list[list[str]], answer: list[str], ground_truth: list[list[str]]]
    The dataset in the format of ragas which the metrics will use to score the RAG
    pipeline with
metrics : list[Metric] , optional
    List of metrics to use for evaluation. If not provided then ragas will run the
    evaluation on the best set of metrics to give a complete view.
llm: BaseRagasLLM, optional
    The language model to use for the metrics. If not provided then ragas will use
    the default language model for metrics which require an LLM. This can we overridden by the llm specified in
    the metric level with `metric.llm`.
embeddings: BaseRagasEmbeddings, optional
    The embeddings to use for the metrics. If not provided then ragas will use
    the default embeddings for metrics which require embeddings. This can we overridden by the embeddings specified in
    the metric level with `metric.embeddings`.
callbacks: Callbacks, optional
    Lifecycle Langchain Callbacks to run during evaluation. Check the
    [langchain documentation](https://python.langchain.com/docs/modules/callbacks/)
    for more information.
in_ci: bool
    Whether the evaluation is running in CI or not. If set to True then some
    metrics will be run to increase the reproducability of the evaluations. This
    will increase the runtime and cost of evaluations. Default is False.
is_async: bool
    Whether to run the evaluation in async mode or not. If set to True then the
    evaluation is run by calling the `metric.ascore` method. In case the llm or
    embeddings does not support async then the evaluation can be run in sync mode
    with `is_async=False`. Default is False.`

[jjmachan]

Hey @dschwalm 👋🏽

thanks for taking the time for bringing this up. You are right there is an inconsistency with that flag and the reason is that it was something that we didn't remove after a big change to the excutor

I've created #975 to remove it and will take it up this week hopefully

[dschwalm]

Cool, thank you for responding and keeping the code more consistent.

[jjmachan]

thanks a lot for reporting this too 🙂, the project has been slow for a while but now its back on track 🚀

closing this for now, feel free to subscribe to the other issue to keep track

we'll get it merged!