From 791f1a4e09066c959bd494f8f04c0804119c6c71 Mon Sep 17 00:00:00 2001 From: TruscaPetre <37754402+TruscaPetre@users.noreply.github.com> Date: Tue, 26 Dec 2023 17:11:15 +0200 Subject: [PATCH] Small improvements to documentation custom LLMs --- docs/howtos/customisations/llms.ipynb | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/docs/howtos/customisations/llms.ipynb b/docs/howtos/customisations/llms.ipynb index a5a6fa7e7..81bc50882 100644 --- a/docs/howtos/customisations/llms.ipynb +++ b/docs/howtos/customisations/llms.ipynb @@ -7,12 +7,12 @@ "source": [ "# Bring your own LLMs\n", "\n", - "Ragas uses langchain under the hood for connecting to LLMs for metrices that require them. This means you can swap out the default LLM we use (`gpt-3.5-turbo-16k`) to use any 100s of API supported out of the box with langchain.\n", + "Ragas uses langchain under the hood for connecting to LLMs for metrics that require them. This means you can swap out the default LLM we use (`gpt-3.5-turbo-16k`) with any 100s of API supported out of the box by langchain:\n", "\n", "- [Completion LLMs Supported](https://api.python.langchain.com/en/latest/api_reference.html#module-langchain.llms)\n", "- [Chat based LLMs Supported](https://api.python.langchain.com/en/latest/api_reference.html#module-langchain.chat_models)\n", "\n", - "This guide will show you how to use another or LLM API for evaluation." + "This guide will show you how to use another LLM API for evaluation." ] }, { @@ -32,9 +32,9 @@ "source": [ "## Evaluating with GPT4\n", "\n", - "Ragas uses gpt3.5 by default but using gpt4 for evaluation can improve the results so lets use that for the `Faithfulness` metric\n", + "Ragas uses gpt3.5 by default but using gpt4 for evaluation can improve the results so lets use that for the `Faithfulness` metric.\n", "\n", - "To start-off, we initialise the gpt4 `chat_model` from langchain" + "To start-off, we initialise the gpt4 `chat_model` from langchain." ] }, { @@ -67,12 +67,14 @@ "id": "f1fdb48b", "metadata": {}, "source": [ - "In order to you the Langchain LLM you have to use the `RagasLLM` wrapper. This help the Ragas library specify the interfaces that will be used internally by the metrics and what is exposed via the Langchain library. You can also use other LLM APIs in tools like LlamaIndex and LiteLLM but creating your own implementation of `RagasLLM` that supports it." + "`RagasLLM` wrapper is required to use the Langchain LLM. This helps the Ragas library specify the interfaces that will be used internally by the metrics and what is exposed via the Langchain library. \n", + "\n", + "You can also use other LLM APIs from tools like LlamaIndex or LiteLLM but that requires creating your own implementation of `RagasLLM` that supports it." ] }, { "cell_type": "code", - "execution_count": 3, + "execution_count": null, "id": "25c72521-3372-4663-81e4-c159b0edde40", "metadata": {}, "outputs": [], @@ -87,7 +89,7 @@ "id": "62645da8-6a52-4cbb-bec7-59f7e153cd38", "metadata": {}, "source": [ - "Substitute the llm in `Metric` instance with the newly create GPT4 model." + "Substitute the `llm` in `Metric` instance with the newly create GPT4 model.\n" ] }, { @@ -107,7 +109,7 @@ "id": "1930dd49", "metadata": {}, "source": [ - "That's it! faithfulness will now be using GPT-4 under the hood for evaluations.\n", + "That's it! `Faithfulness` will now be using GPT-4 under the hood for evaluations.\n", "\n", "Now lets run the evaluations using the example from [quickstart](../quickstart.ipnb)." ] @@ -213,7 +215,7 @@ "source": [ "## Evaluating with Open-Source LLMs\n", "\n", - "You can also use any of the Open-Source LLM for evaluating. Ragas support most the the deployment methods like [HuggingFace TGI](https://python.langchain.com/docs/integrations/llms/huggingface_textgen_inference), [Anyscale](https://python.langchain.com/docs/integrations/llms/anyscale), [vLLM](https://python.langchain.com/docs/integrations/llms/vllm) and many [more](https://python.langchain.com/docs/integrations/llms/) through Langchain. \n", + "You can also use any of the Open-Source LLM for evaluation. Ragas support most the the deployment methods like [HuggingFace TGI](https://python.langchain.com/docs/integrations/llms/huggingface_textgen_inference), [Anyscale](https://python.langchain.com/docs/integrations/llms/anyscale), [vLLM](https://python.langchain.com/docs/integrations/llms/vllm) and many [more](https://python.langchain.com/docs/integrations/llms/) through Langchain. \n", "\n", "When it comes to selecting open-source language models, there are some rules of thumb to follow, given that the quality of evaluation metrics depends heavily on the model's quality:\n", "\n", @@ -310,7 +312,7 @@ "id": "58a610f2-19e5-40ec-bb7d-760c1d608a85", "metadata": {}, "source": [ - "Now you can run the evaluations with and analyse the results." + "Now you can run the evaluations with `HuggingFaceH4/zephyr-7b-alpha` and analyse the results." ] }, {