Repo cleanup: consolidate/rename folders, add/update READMEs

README.md

@@ -1,10 +1,23 @@
-# Elasticsearch Labs 
+# Elasticsearch Labs
 
-This repository contains several sample applications for demonstrating the features of Elasticsearch and Elasticsearch Relevance Engine (ESRE) to build modern conversational search experiences powered by generative AI. 
-Elasticsearch is a distributed RESTful search and analytics platform at the heart of our  [ESRE](https://www.elastic.co/enterprise-search/generative-ai) capabilities. 
+This repository contains executable Python notebooks, sample apps, and resources for testing out the Elastic platform:
+
+- Learn how to use Elasticsearch as a vector database to store embeddings, power hybrid and semantic search experiences, and more.
+- Test Elastic's leading-edge, out-of-the-box capabilities like the [Elastic Learned Sparse Encoder](https://www.elastic.co/guide/en/machine-learning/current/ml-nlp-elser.html) and [reciprocal rank fusion (RRF)](https://www.elastic.co/blog/whats-new-elastic-enterprise-search-8-9-0#hybrid-search-with-reciprocal-rank-fusion-(rrf)-combines-multiple-search-techniques-for-better-results), which produce best-in-class results without training or tuning.
+- Integrate with projects like OpenAI, Hugging Face, and LangChain to use Elasticsearch as the backbone of your LLM-powered applications. For use cases like retrieval augmented generation (RAG), summarization, and question answering (QA).
+
+# Python notebooks 📒
+
+The [`notebooks` folder](notebooks/README.md) contains a range of executable Python notebooks, so you can test these features out for yourself. Colab provides an easy-to-use Python virtual environment in the browser.
+
+# Example apps 💻
+
+The [`example-apps`](example-apps/README.md) folder contains example apps that demonstrate Elasticsearch for a number of use cases, using different programming languages and frameworks.
+
+# Support 🛟
+
+Learn how to [get Support](docs/SUPPORT.md).
+
+# License ⚖️
 
-# Guides 
-- [Getting Support](docs/SUPPORT.md)
-
-# License
 This software is licensed under the [Apache License, version 2 ("ALv2")](https://github.com/elastic/elasticsearch-labs/blob/main/LICENSE).

example-apps/ElasticDocs_GPT/README.md

@@ -0,0 +1,33 @@
+# ElasticDocs GPT
+
+Combine the search power of Elasticsearch with the Question Answering power of GPT.
+
+This application supports the popular Elastic blog [ChatGPT and Elasticsearch: OpenAI meets private data](https://www.elastic.co/blog/chatgpt-elasticsearch-openai-meets-private-data).
+
+## Overview
+
+The following diagram shows the high level architecture of the application:
+
+![diagram](https://raw.githubusercontent.com/jeffvestal/ElasticDocs_GPT/main/images/ElasticChat%20GPT%20Diagram%20-%20No%20line%20text.jpeg)
+
+1. A **Python interface** accepts user questions
+   - Generates a hybrid search request for Elasticsearch
+   - BM25 match on the `title` field
+   - kNN searches on the `title-vector` field
+   - Boost `kNN` search results to align scores
+   - Set `size=1` to return only the top scored document
+2. **Search request** is sent to Elasticsearch
+3. **Documentation body and original url** are returned to Python
+4. **API call** is made to OpenAI ChatCompletion with the prompt:
+   - _"answer this question <question> using only this document <body_content from top search result>"_
+5. **Generated response** is returned to Python
+6. Python adds **original documentation source url** to generated response and **prints it to the screen** for the user
+
+# Examples
+  ![autoscale](https://raw.githubusercontent.com/jeffvestal/ElasticDocs_GPT/main/images/elasticDocs%20GPT%20-%20elastic%20cloud%20autoscaling.png)
+
+  ![apm](https://raw.githubusercontent.com/jeffvestal/ElasticDocs_GPT/main/images/elasticDocs%20GPT%20-%20elastic%20jvm%20apm.png)
+
+  ![inference](https://github.com/jeffvestal/ElasticDocs_GPT/blob/main/images/elasticDocs%20GPT%20-%20inference%20processor.png)
+
+  ![pii](https://raw.githubusercontent.com/jeffvestal/ElasticDocs_GPT/main/images/elasticDocs%20GPT%20-%20redact%20pii.png)

example-apps/ElasticGPT_Plugin/README.md

@@ -0,0 +1,19 @@
+# ElasticGPT plugin
+
+Implement a ChatGPT plugin and extend ChatGPT usage to any content indexed into Elasticsearch:
+
+- Implement a ChatGPT plugin that allows access to Elasticsearch data for context-relevant responses.
+- ChatGPT plugins are extensions developed to assist the model in completing its knowledge or executing actions.
+- The plugin architecture involves making a call to the `/search` endpoint of the plugin, which sends a search request to Elasticsearch.
+- The plugin then returns the document body and URL to ChatGPT, which uses this information to craft its response.
+- Deployment of the plugin on Google Cloud Platform (GCP) is demonstrated using Cloud Run.
+- Users can install the plugin in ChatGPT and query Elasticsearch data to enhance ChatGPT's knowledge and functionality.
+
+For full details refer to the original Elastic Blog Post:
+
+- [ChatGPT and Elasticsearch: A plugin to use ChatGPT with your Elastic data](https://www.elastic.co/blog/chatgpt-elasticsearch-plugin-elastic-data)
+
+
+> ℹ️ **Note**:
+> _This is an Elastic Labs project. Elastic Labs projects are for illustrative and experimental purposes only. This Elastic Labs project is not part of any product or services offering provided or supported under a commercial license or subscription. This project is made available as-is under the terms of the license associated with this project._
+> _The release and timing of any features or functionality described in this project remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all._

esre-examples/openai-integration-example-javascript/README.md → example-apps/OpenAI-embeddings/OpenAI-JS/README.md

@@ -1,4 +1,4 @@
-# Elastic OpenAI integration example - JavaScript
+# Elastic OpenAI embeddings example - JavaScript
 
 ## Overview
 
@@ -26,7 +26,7 @@ Download the project from Github and extract the openai-integration-example-pyth
 
 ```bash
 curl https://codeload.github.com/elastic/elasticsearch-labs/tar.gz/main | \
-tar -xz --strip=2 elasticsearch-labs-main/esre-examples/openai-integration-example-javascript
+tar -xz --strip=2 elasticsearch-labs-main/example-apps/OpenAI-embeddings/OpenAI-Py
 ```
 
 ### 2. Create OpenAI account and API key

esre-examples/openai-integration-example-python/README.md → example-apps/OpenAI-embeddings/OpenAI-Py/README.md

@@ -26,7 +26,7 @@ Download the project from Github and extract the openai-integration-example-pyth
 
 ```bash
 curl https://codeload.github.com/elastic/elasticsearch-labs/tar.gz/main | \
-tar -xz --strip=2 elasticsearch-labs-main/esre-examples/openai-integration-example-python
+tar -xz --strip=2 elasticsearch-labs-main/example-apps/OpenAI-embeddings/OpenAI-Py
 ```
 
 ### 2. Create OpenAI account and API key

example-apps/OpenAI-embeddings/README.md

@@ -0,0 +1,14 @@
+## Overview
+
+Tiny example applications that integrate Elastic with [OpenAI embeddings](https://platform.openai.com/docs/guides/embeddings):
+
+- Process JSON files
+- Generate text embeddings using OpenAI [embeddings API](https://platform.openai.com/docs/api-reference/embeddings)
+- Store data in Elasticsearch
+- Run semantic search in web app using Elasticsearch kNN search
+- Return relevant search results
+
+## Programming languages
+
+- [Python](./OpenAI-Py/README.md)
+- [JavaScript](./OpenAI-JS/README.md)

example-apps/README.md

@@ -0,0 +1,14 @@
+# Elasticsearch example apps
+
+## Overview
+
+The following example apps demonstrate Elasticsearch for a variety of use cases, such as semantic search and retrieval augmented generation, using different programming languages and frameworks.
+
+Use these apps as a starting point for your own projects.
+
+## Apps
+
+- [ElasticDocs GPT](./ElasticDocs_GPT/README.md). Combine Elasticsearch with the question answering power of ChatGPT.
+- [ElasticGPT plugin](./ElasticGPT_Plugin/README.md). Implement a ChatGPT plugin and extend ChatGPT usage to any content indexed into Elasticsearch.
+- [OpenAI embeddings](./OpenAI-embeddings/README.md). Use OpenAI embeddings at index time and in Elastic kNN queries. Available in [JavaScript](./OpenAI-embeddings/OpenAI-JS/README.md) and [Python](./OpenAI-embeddings/OpenAI-Py/README.md) versions.
+- [Workplace search](./Workplace-Search/README.MD). Build a semantic search experience for organizational/workplace data.

esre-examples/python-flask-example/README.md → example-apps/Workplace-Search/README.md

@@ -6,11 +6,12 @@ This is a sample app that combines Elasticsearch and OpenAI to create a semantic
 
 ## 1. Download the Project
 
-Download the project from Github and extract the python-flask-example folder.
+Download the project from Github and extract the Workplace-Search folder.
 
 ```bash
 curl https://codeload.github.com/elastic/elasticsearch-labs/tar.gz/main | \
-tar -xz --strip=2 elasticsearch-labs-main/esre-examples/python-flask-example
+tar -xz --strip=2 elasticsearch-labs-main/example-apps/Workplace-Search
+```
 ```
 
 ## 2. Index Data

notebooks/README.md

@@ -0,0 +1,20 @@
+# Python notebooks 📒
+
+This folder contains a range of executable Python notebooks, so you can test everything out for yourself. Use these examples as a blueprint for testing on your own data.
+
+Run notebooks locally using [Jupyter](https://jupyter.org/install), or use the links provided in each notebook to run them in Google [Colab](https://colab.research.google.com).
+Colab provides an easy-to-use Python virtual environment in the browser.
+
+Notebooks are organized into the following folders:
+
+- [`search`](./search/). Learn the fundamentals of Elasticsearch, so you can index embeddings, run lexical, semantic and _hybrid_ searches, and more.
+
+- [`generative-ai`](./generative-ai/): Notebooks that demonstrate various use cases for Elasticsearch as the retrieval engine and vector store for LLM-powered applications.
+
+- [`integrations`](./integrations/): Notebooks that demonstrate how to integrate popular services and projects with Elasticsearch:
+  - [OpenAI](./integrations/openai)
+  - [Hugging Face](./integrations/hugging-face)
+  - [LlamaIndex](./integrations/llama-index)
+
+- [`langchain`](./langchain/): Notebooks that demonstrate how to integrate Elastic with [LangChain](https://langchain-langchain.vercel.app/docs/get_started/introduction.html), a framework for developing applications powered by language models.
+