OPEA Highlights

Add 4 MegaService examples: CodeGen, ChatQnA, CodeTrans and Docsum, you can deploy them on Kubernetes
Enable 10 microservices for LLM, RAG, security...etc
Support text generation, code generation and end-to-end evaluation

GenAIExamples

Build 4 reference solutions for some classic GenAI applications, like code generation, chat Q&A, code translation and document summarization, through orchestration interface in GenAIComps.
Support seamlessly deployment on Intel Xeon and Gaudi platform through Kubernetes and Docker Compose.

Activate a suite of microservices including ASR, LLMS, Rerank, Embedding, Guardrails, TTS, Telemetry, DataPrep, Retrieval, and VectorDB. ASR functionality is fully operational on Xeon architecture, pending readiness on Gaudi. Retrieval capabilities are functional on LangChain, awaiting readiness on LlamaIndex. VectorDB functionality is supported on Redis, Chroma, and Qdrant, with readiness pending on SVS.
Added 14 file formats support in data preparation microservices and enabled Safeguard of conversation in guardrails.
Added the Ray Gaudi Supported for LLM Service.

Add evaluating the models on text-generation tasks(lm-evaluation-harness) and coding tasks (bigcode-evaluation-harness)
Add end-to-end evaluation with microservice

Add Helm Charts redis-vector-db, TEI, TGI and CodeGen for deploying GenAIExamples on Kubernetes
Add Manifests for deploying GenAIExamples CodeGen, ChatQnA and Docsum on Kubernetes and on Docker Compose

Highlights

Provides a collective list of Generative AI (GenAI) and Retrieval-Augmented Generation (RAG) examples such as chatbot with question and answering (ChatQnA), code generation (CodeGen), document summary (DocSum), etc.
All the examples are well-validated and optimized on Intel platforms.
Use ecosystem-compliant APIs to build the end-to-end GenAI examples.
Deploy the GenAI examples with performance on Intel platforms.

Examples

ChatQnA: an example of chatbot for question and answering through retrieval argumented generation (RAG)
CodeGen: an example of copilot designed for code generation in Visual Studio Code.
DocSum: an example of chatbot for summarizing the content of documents or reports.
SearchQnA: an example of chatbot for using search engine to enhance QA quality.
VisualQnA is an example of chatbot for question and answering based on the images.

Validated Configurations