| page_type | languages | products | urlFragment | name | description | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
sample |
|
|
azure-search-openai-demo-java |
RAG chat app with Azure OpenAI and Azure AI Search with Java and Langchain4J |
A Java sample app that chats with your data using Azure AI services. |
This repo is the Java version of the well known ChatGPT + Enterprise data code sample originally written in python.
It demonstrates best practices for creating ChatGPT-like experiences over your own data using the Retrieval Augmented Generation pattern. It uses Azure OpenAI Service to access the ChatGPT model gpt-4o-mini, and Azure 0AI Search for data indexing and retrieval and Azure Document Intelligence for document structured text parsing.
This repository includes sample data so it's ready to try end to end. In this sample application we use a fictitious company called Contoso Electronics, and the experience allows its employees to ask questions about the benefits, internal policies, as well as job descriptions and roles.
- Chat interface with search result citations and follow-up questions.
- Documents uploads and management.
- Response streaming.
- Response metadata with source content and grounded prompt tracking, search keywords, token usages.
- Citation panel with page level detail (pdf, html, markdown).
- Several documents formats supported: pdf, html, markdown, Office (docx,xslx,pptx).
- Explores various retrievial options to help users evaluate search results: search score, search top results, text/vector/hybrid retrievial strategy, semantic caption and ranking (Azure AI search specific features).
- Chat history with Azure CosmosDB or browser local database (for unauthenticated users).
- Login and documents access control with EntraID.
- Paged document text extraction with table data support with Azure Document Intelligence.
This sample supports different architectural styles. It can be deployed as a microservice event driven architecture with web frontend, AI orchestration and document ingestion apps hosted by Azure Container Apps or Azure Kubernetes Service. It uses Langchain4J AI orchestration java framework.
Note
Previous java Semantic Kernel version has been moved in a dedicated branch
The solution components and Azure services are:
- The API app is implemented as springboot 2.7.x app using the Microsoft JDK.It implements the Chat flow in RAG pattern and provides both a synch and asynch streaming chat REST endpoint for the webapp.It's responsible for orchestrating the interaction between the LLM models (Azure Open AI) and the content retriever (Azure AI Search).
- The chat web app is built in React and deployed as a static web app on nginx. In ACA deployment option, Nginx act as reverse proxy for api calls to the API app. This also solves the CORS issue. In AKS deployment option, a managed Nginx ingress controller is used as entry point for both thw web app and api.
- The indexer APP is implemented as springboot 2.7.x app using the Microsoft JDK.It implement the Ingestion/Indexing flow in RAG pattern and it's responsible for indexing the data into Azure AI Search.It's asynchnously triggered by new BlobUploaded messages from serviceBus or synchrnously called by API to handle User document upload request from the web app. The indexer is also responsible for chunking the documents into smaller pieces, embed them and store them in the index. Azure Document Intelligence can be configured to extract text from PDF documents (including tables and images) or you can leverage document parsers available in Langchain4j. See here for ingestion flow configuration example.
- Azure AI Search is used as RAG retrieval system. Different search options are available: you have traditional full text (with semantic search) search, or vector based search and finally you can opt for hybrid search which brings together the best of the previous ones.
- EventGrid system topic is used to implement a real time mechanism to trigger the indexer app when a new document is uploaded to the Azure blob called staging.It's responsible to read BlobUploaded notification from azure storage container and push a message to the service bus queue containing the blob url. All the documents loaded through service bus messages will be moved to default blob folder and will be available to all users.
- CosmosDB is used to store chat conversations.
For specific deployment options see:
All the available architectures and implementations come with few options to start developing and running the application locally using dev containers or in the cloud using github codespaces. For more detailed instructions, please refer to the specific architecture implementation documentation:
For platform specific guidance, please refer to the following documentation:
This sample is designed to get you started quickly and let you experiment with Java RAG architectures on Azure. For production deployment, you can use the Azure Application Landing Zones (LZA) to automatically deploy the solution in production leveraging best practices for security, monitoring, networking and operational excellence.
Check the chat-with-your-data-lza-app-accelerator to see how you can deploy this solution on Azure Container Apps LZA.
Note
Support for deploying on landing zone is available for previous semantic kernel version at this repo. A new version based on Azure Verified Modules for Azure Container Apps is WIP ☀️ ☁️ :construction_worker_man.
- 📖 Revolutionize your Enterprise Data with ChatGPT: Next-gen Apps w/ Azure OpenAI and AI Search
- 📖 Azure OpenAI Service
- 📖 RAG in Azure AI Search
- 📖 Azure OpenAI client library for Java
- 📖 Semantic Kernel for Java 1.0
- 📖 Evaluating a RAG chat App
- 📖 Well Architected Framework on Azure OpenAI Service
- 📺 Chat with your data using Java and Azure Open AI
- 📺 Vector Search, RAG, And Azure AI Search