prompflow1.md

Building Large Language Models Applications using Prompt Flow in Azure ML - Simple LLMOps

Introduction

• Build Large Language Models using Prompt Flow in Azure ML
• Using Prompt Flow
• Using Azure Cognitive search vector services
• using Azure open ai service

Concept

• First create a vector store and provide cognitive search service
• Create a connections to open ai and also cognitive search
• Create a prompt flow
• Create a pipeline to run the prompt flow
• i am using existing vector store and cognitive search service

Pre-requisites

• Azure Cognitive search
• Azure machine learning service
• Azure Open AI service

Code

• Go to Azure Machine Learning services
• Create a connection for open ai in prompt flow sections

• Now create vector index search connections
• i am using existing azure cognitive search service
• Start with uploading data to vector store

• Give a name
• Upload the pdf files
• Select the cognitive search service for vector index
• Select the vector index
• Now Click Next
• In the next screen select Azure Open ai services

• Select compute as serverless

• Review and create the vector store connection

Prompt flow or LLM application

• Now create a prompt flow
• Create a Standard flow

• Input and output will automatically be created

• Select the Azure Open AI connection
• Now add flow to search the index

• now generate prompt context

• Code

from typing import List
from promptflow import tool
from embeddingstore.core.contracts import SearchResultEntity

@tool
def generate_prompt_context(search_result: List[dict]) -> str:
    def format_doc(doc: dict):
        return f"Content: {doc['Content']}\nSource: {doc['Source']}"
    
    SOURCE_KEY = "source"
    URL_KEY = "url"
    
    retrieved_docs = []
    for item in search_result:

        entity = SearchResultEntity.from_dict(item)
        content  = entity.text or ""
        
        source = ""
        if entity.metadata is not None:
            if SOURCE_KEY in entity.metadata:
                if URL_KEY in entity.metadata[SOURCE_KEY]:
                    source = entity.metadata[SOURCE_KEY][URL_KEY] or ""
        
        retrieved_docs.append({
            "Content": content,
            "Source": source
        })
    doc_string = "\n\n".join([format_doc(doc) for doc in retrieved_docs])
    return doc_string

• now specfiy the input for the above code
• Input is shown in below image

• now create a flow for prompt variant
• Specify the prompt to use in the application
• Next setup to answer the question

• Setup the azure open ai connection
• select the model deployment to use, i am using chat for gpt 3.5 turbo
• Set the temperature and max token output
• Send the output as prompt_text
• Now on the right top corner click on the run button
• Wait for the run to complete
• You will see the graph in the right side

• you can change the prompt and run again
• Try with different variations one shot, few shot examples in prompt
• Try to chain the prompt and experiment
• if you are satified with the results output then proceed to next step to deploy

Deploy

• Next Deploy the code is all is well
• Give name to the endpoint

• now choose the outputs for the endpoint

• Now select the connection and models to use
• We need embedding and also gpt 3.5 turbo model

• Select compute
• Provide number of instances and also the size of the instance

• Now review and then deploy
• Deployment will take few minutes to deploy the endpoint and deploy the traffic
• Test the model to make sure it is working as expected