Next-Gen GPT-ize Intelligent Application Architecture - A prompt engineering and RAG approach with LLM AI Dev Framework
The importance of Prompt Engineering in the field of large-scale language model applications is gradually becoming apparent. Prompt Engineering is a technique based on human language intuition that can help startup, corporate, and enterprises quickly achieve specific functions when building large language models. Based on the formal representation of natural language questions, Prompt Engineering can effectively guide the learning process of large language models, thereby improving their performance. Some papers have shown that in certain complex reasoning and thought-chain contexts, prompts even perform better than fine-tuning.
Retrieval Augmented Generation (RAG) is a method that greatly enhances the accuracy of GPT in answering questions. It combines pre-trained models with vectorized data storage, allowing for direct retrieval and manipulation of external knowledge during text generation. RAG can assist GPT models in better understanding and utilizing given knowledge, thus improving their performance in knowledge-intensive tasks. In practical applications, the RAG method can significantly improve the precision of model output and effectively prevent model hallucination and fabrication.
Based on these research findings, we have proposed a framework for enterprise-level next generation GPT intelligent application engineering, which is based on Retrieval Augmented Generation and Prompt Engineering methodology. The framework, as shown in the diagram below, allows developers to quickly build their own large-scale intelligent applications without modifying the model itself, and immediately enjoy the productivity dividend brought by the new generation of artificial intelligence for any business scenarios.
As shown in the diagram, this architecture utilizes a flexible semantic compute engine as prompt engine and supports various deployment forms such as PaaS, Serverless, and containers, enabling develoeprs to optimize the prompt engine with modern LLM AI application development framework and elastically scale computing resources according to real-time business needs. The session and token data service layer, based on Redis and CosmosDB, adds contextual caching, session persistence, prompt persistence, and other capabilities to the application, leaving room for future model or engine optimization based on prompts. The API encapsulation, load balancing, and gateway at the front end further enhance application security and reliability, allowing the intelligent entity to integrate with various front-end apps in a more secure and stable manner. Most importantly, this architecture is designed with a dual-engine architecture based on Azure Cognitive Search and vector embedding, two additional private knowledge base technologies. On the one hand, Azure Cognitive Search can quickly index unstructured data such as PDF and WORD files, allowing existing data to be used immediately. On the other hand, by utilizing Azure Database with vector storage and processing capabilities and combining them with Azure OpenAI's embedding vector generation model, the enterprise's existing structured knowledge base can be integrated easily, which producing more accurate, stable, and reliable results from the cutting-edge GPT model.
Semantic Kernel (SK) is a lightweight SDK enabling integration of AI Large Language Models (LLMs) with conventional programming languages. The SK extensible programming model combines natural language semantic functions, traditional code native functions, and embeddings-based memory unlocking new potential and adding value to applications with AI.
SK supports prompt templating, function chaining, vectorized memory, and intelligent planning capabilities out of the box.
Semantic Kernel is designed to support and encapsulate several design patterns from the latest in AI research, such that developers can infuse their applications with complex skills like prompt chaining, recursive reasoning, summarization, zero/few-shot learning, contextual memory, long-term memory, embeddings, semantic indexing, planning, and accessing external knowledge stores as well as your own data.
For more information, please visit Semantic Kernel Project
This is an app innovation workshop focused on exploring the use of Azure PaaS and OpenAI GPT model to build modern large language modelintelligent applications. Workshop aims to provide practical skills and experience in developing innovative intelligent applications on Azure for IT and architecture decision-makers, business decision-makers, and developers.
In this workshop, attendees will learn how to select Azure workloads to design a modern GPT intelligent application, understand Azure OpenAI services, explore the capabilities of large model prompt engineering, understand common parameters and options in the OpenAI API, and land an next generation GPT intelligent application architecture inside their organization through hands-on experimentation.
Whether you are a novice or an experienced professional, this workshop will meet the expectations and needs of participants. Attendees will be able to interact with experts, share experiences, and gain inspiration and insights from real project cases. Together, we will explore exciting GPT intelligent applications and usher in a new era of intelligent applications!
- Workshop content Chinese
- Workshop content English
- Demo notebook
- Workshop code base
- ChatGPT Embeddings QnA
- LLM App Pattern - Project Miyagi
- Knowledge Mining with OpenAI Architecture
- Azure OpenAI Samples
- Experimental: Uncle John's Semantic Kernel Recipes
- Learn Azure OpenAI Service with .NET
- Azure Openai
- Azure Cognitive Search
- Azure App Service
- Azure API Management
- Azure Container App
- Azure Kubernetes Service
- Azure CosmosDB
- Azure Cache for Redis
- Azure Azure Database for PostgreSQL
- Qdrant Vector Database on Azure Cloud
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Sparks of Artificial General Intelligence: Early experiments with GPT-4
- HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace
- Automatic Generation of Grammar-Agnostic Visualizations and Infographics
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv, 2022
- Emergent Analogical Reasoning in Large Language Models. arXiv:2212.09196, 2022
- Generated Knowledge Prompting for Commonsense Reasoning. arXiv:2110.08387, 2022
- Multitask Prompted Training Enables Zero-Shot Task Generalization. arXiv:2110.08207, 2022
- Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm. arXiv:2102.07350, 2021
- Self-Consistency Improves Chain of Thought Reasoning in Language Models. arXiv:2203.11171, 2022
- How does GPT Obtain its Ability? Tracing Emergent Abilities of Language Models to their Sources
- A Closer Look at Large Language Models Emergent Abilities
- Illustrating Reinforcement Learning from Human Feedback (RLHF)
- Best practices for prompt engineering with openai api
- Awesome ChatGPT Prompts
- Durable Task Framework
- Hackathon Materials for Semantic Kernel