[pull] main from elastic:main

supporting-blog-content/using-ragas-with-elasticsearch/README.md

@@ -0,0 +1,51 @@
+# Using Ragas with Elasticsearch
+Ragas is an evaluation framework that enables you to gain a deeper understanding of how your LLM application performs. Using evaluation methods, such as the Ragas framework, can help you determine if your LLM application is performing as intended and provide a sense of its accuracy. It can enable data-driven decisions about model selection, prompt engineering effectiveness, and retrieval system optimization.
+
+This repository contains a demo using a sample book dataset and Elasticsearch.
+
+## Setting up
+- The version of Python that is used is Python 3.12.1 but you can use any version of Python higher than 3.10.
+- This demo uses Elasticsearch version 9.0.3, but you can use any version of Elasticsearch that is higher than 8.10.
+- You will also need an OpenAI API key for LLM-based metrics. You will want to configure an environment variable for your OpenAI API Key, which you can find on the API keys page in [OpenAI's developer portal](https://platform.openai.com/api-keys).
+
+## Stucture of the example
+- **[books.json](books.json)**: The dataset used containing an sample of books. This is a subset of 25 books from Goodreads including the book title, the author's name, book description, publication year, and a Goodreads URL.
+- **[ragas-elasticsearch-demo.ipynb](ragas-elasticsearch-demo.ipynb)**: Main Jupyter notebook for running ragas evaluations. It sets up the environment, loads data, runs sample queries, and computes evaluation metrics (context precision, faithfulness, answer relevancy) using ragas.
+- **[ragas_evaluation.csv](ragas_evaluation.csv)**: Output file generated by the notebook, containing detailed results for each evaluation query, including metrics for context precision, faithfulness, and answer relevancy.
+
+## Changing the model
+This example is using `GPT-4o`, you can easily change the model to another by adjusting the parameter `model=”model name”`.
+
+```python
+chat_llm = ChatOpenAI(
+   model="gpt-4o",
+   temperature=0.1,
+   api_key=API_KEY
+)
+```
+
+## Usage
+
+1. **Install Dependencies**
+   The notebook will install required dependencies automatically, but you can also install them manually:
+
+   ```bash
+   pip install -q ragas datasets langchain elasticsearch openai langchain-openai
+   ```
+
+3. **Run the Notebook**
+   Open `ragas-elasticsearch-demo.ipynb` in Jupyter and follow the instructions to run each cell. The notebook will:
+
+   - Query your book index (via Elasticsearch)
+   - Run sample RAG queries
+   - Evaluate the responses using Ragas
+   - Output results to `ragas_evaluation.csv`
+
+5. **View Results**
+
+   The results file contains detailed metrics for each test query. Use it to analyze the quality of your RAG pipeline and compare different configurations. The results will also be in a `csv` file.
+
+
+## Troubleshooting 
+- Your model will need to be deployed first before the following code could run. To learn more about this be sure to check out our [documention on the subject](https://www.elastic.co/docs/explore-analyze/machine-learning/nlp/ml-nlp-deploy-model).
+- If you encounter any problems running this in Colab or locally, it might be due to the dataset requiring a separate download. It can be found in the same folder as this example.