Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

content refresh #60

Merged
merged 1 commit into from Mar 4, 2024
Merged

content refresh #60

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
14 changes: 7 additions & 7 deletions README.md
View file
Open in desktop
Expand Up @@ -51,19 +51,19 @@ Customers in regulated industries must define and enforce required controls in o

### Prescriptive use cases

#### Image and Video recognition
Leverage and validate the initial use case you selected for the Enterprise Azure OpenAI Hub. The use case you selected will determine the deployment and validation steps you need to follow. We will continue to iterate and add more use cases to the library as we continue to evolve and refine commun customer scenarios, aligned with the development of the Azure AI platform.

#### Content creation and design
For a detailed walkthrough, you can go through the validation and realization steps [in this article](./docs/use-cases.md).

#### Accelerated automation
#### Image and Video recognition

#### Personalized marketing
GPT4 Vision together with Azure AI Vision services can be used to recognize and understand the content of images and videos. This use case is designed to provide a comprehensive understanding of how to leverage the power of Azure AI Vision services to solve complex problems and develop cutting-edge AI solutions.

#### Language translation and natural language processing
#### 'On Your Data'

#### Chatbots and virtual agents
Azure native RAG Architecture leveraging Azure AI search for vectorization and using Azure OpenAI for generating embeddings and decoding the embeddings to generate text. This use case is designed to provide a comprehensive understanding of how to leverage the power of Azure AI services to solve complex problems and develop cutting-edge AI solutions on your own Enterprise data.

#### [ChatGPT on your Enterprise data](./docs/rag.md)
[Read the following guidance](./docs/rag.md) for the architecture setup and how to use.

## Contributing

Expand Down
211 changes: 40 additions & 171 deletions docs/EnterpriseAzureOpenAIHub.md
View file
Open in desktop
Expand Up @@ -99,7 +99,7 @@ This section will explain the deployment experience and the options provided for

Once the pre-requisites have been completed, you can deploy the reference implementation using this link [*Deploy to Microsoft Cloud*](https://aka.ms/DeploySecureGenAI), it will start the deployment experience in the Azure portal into your default Azure tenant. In case you have access to multiple tenants, ensure you are selecting the right one.

### 1 - Architecture Setup
### Architecture setup

The first tab will allow you to specify the intent of the deployment, and the overall architecture setup. You can select the deployment intent as "Production" or "Proof of Concept", and subject to the selection, the deployment experience will be tailored to the respective intent.

Expand All @@ -108,68 +108,71 @@ Provide a prefix for the naming convention that will be used for the resources.

> Note: the naming convention will primarily consist of 'prefix'-'region'-'resourcetype' where possible.

![Deployment location](./ArchSetup.png)
<img src="./ArchSetup.png" alt="Architecture setup" width="800" height="600">

When selecting "Production" as the intent, provide the prefix for the naming convention and select the target Azure region.

![Deployment Prod](./ArchSetupSingleParam.png)
<img src="./ArchSetupSingleParam.png" alt="Architecture setup" width="800" height="200">

When selecting "Proof of Concept" as the intent for the deployment, you can optionally select to deploy to a single region, or to multiple regions. When deploying to multiple regions, Azure API Management will be used to facilitate the load balancing, and also provides the retry logic and error handling in case of a failure or unavailability of the Azure Open AI instance in one of the regions. This option requires the Azure services to use Public Endpoints as the APIM service is not available over Private Endpoints in the V2 version currently.

![Deployment intent](./ArchSetupMulti.png)
<img src="./ArchSetupMulti.png" alt="Architecture multi" alt="Architecture setup" width="800" height="600">

When selecting this option, you must provide region input for the initial deployment, the secondary deployment, and also select one of the available regions for Azure API Management V2 service.

![Deployment PoC Multi region](./ArchSetupMultiLocationParam.png)
<img src="./ArchSetupMultiLocationParam.png" alt="Architecture multi" alt="Architecture setup" width="800" height="200">

### 2 - Key Vault Configuration

### Key Vault Configuration

Configure the Key Vault that will be used to store the keys used by the storage account for encryption at rest, as well as the Azure Open AI service. It is recommended to leave with the default recommendations as it relates to the security and compliance recommendations. If needed, you can opt out of the recommendations, assuming you are aware of the implications.

![Key Vault](./KVSetup1.png
<img src="./KVSetup1.png" alt="Key Vault" width="800" height="600">

In the networking section when deploying using a Private Endpoint, you must provide the resourceId of an existing subnet in the same region where you are deploying into.
If you want to deploy the Azure Open AI workloads into a different region vs where you have your virtual network, select the region for the Private Endpoint (i.e., "Deploy the Private Endpoint for Key Vault into the same region as the Key Vault" option must be set to "No", and the regional parameter will appear in the portal)

![Key Vault](./KVSetup2.png)
<img src="./KVSetup2.png>" alt="Key Vault" width="800" height="600"

### 3 - Storage Configuration
### Storage Configuration

This page will create and configure the storage account that will be used in conjunction with the Key Vault and the Azure Open AI service. This enables you to bring your own data that you can use to fine-tune and train the Azure Open AI service for enterprise-specific contexts, fully encrypted at rest using a customer-managed key.

Provide a key name, and the resourceId for an existing subnet when deploying with Private Endpoint. Same as with the Key Vault configuration, if you are deploying to a different region vs where the virtual network is created, select a different region for the private endpoint.

![Storage Account](./SASetup1.png)
<img src="./SASetup1.png" alt="Storage Account" width="800" height="600">

### 4 - Azure OpenAI Configuration
### Azure OpenAI Configuration

Configure the Azure OpenAI instance that will be created, by providing a name for the customer-managed key, and the resourceId to the subnet where the Private Endpoint will be deployed. Same as with the Key Vault and Storage Account configuration, if you are deploying to a different region vs where the virtual network is created, select a different region for the private endpoint.

![Azure Open AI](./AOAISetup1.png)
<img src="./AOAISetup1.png" alt="Azure Open AI" width="800" height="600">

### 5 - Model Deployment and Content Filtering
### Model Deployment and Content Filtering

On this page, you can optionally select to deploy an available model to your Azure OpenAI instance, subject to the available models in the region you have selected. Should there be any capacity constraints with the selected model, the validation API will catch that and inform you before you can submit the deployment.

![Model Deployment](./ModelSetup1.png)
<img src="./ModelSetup1.png" alt="Model deployment" width="800" height="300">

Select the intial model deployment from the drop down list, and provide a name for the deployment.

![Model Deployment](./ModelSetup2.png)
<img src="./ModelSetup2.png" alt="Model deployment" width="800" height="400">

Additionally, you can configure content filtering and advanced filtering settings, that are running on top of the general filtering settings. This is to ensure that the generated content is compliant with the organization's policies and guidelines.

![Model Deployment](./ModelSetup3.png)
<img src="./ModelSetup3.png" alt="Model deployment" width="800" height="500">

Select your preferred options and capabilities you want to be enabled for the content filter.

![Model Deployment](./ModelSetup4.png)
<img src="./ModelSetup4.png" alt="Model deployment" width="800" height="500">

### 5 - Use Cases and Additional Services
### Use Cases and Additional Services

On this page, you can optionally select your initial use case, and additional services that you may want to deploy alongside the Azure OpenAI instance. The list of services will dynamically appear based on the use case you have selected. Each Azure service will provide similar configuration options as the previous pages, and you can configure them as needed in order to meet your security and compliance needs for the overall architecture and setup.

To learn more about the use cases and additional services that are available for the "Enterprise Azure OpenAI Hub" reference implementation, see the [use cases](./use-cases.md) documentation.

![Use Cases and Additional Services](./UseCaseSetup1.png)
<img src="./UseCaseSetup1.png" alt="Use cases 1" width="800" height="500">

In the drop down, you can currently select between the following use cases:

Expand All @@ -182,197 +185,63 @@ Both use cases will provide you with required details for the services that must

When selecting the Image and Video Recognition use case, you can configure the Azure AI Vision, Azure AI Search, and Azure AI Document Intelligence services, Azure Data Factory, and Azure CosmosDB services, and provide the necessary details for the configuration.

![Use Cases and Additional Services](./UseCaseSetup2.png)
<img src="./UseCaseSetup2.png" alt="Use cases" width="800" height="600">

** 'On Your Data' **

When selecting the 'On Your Data' use case, you can configure the Azure AI Search, Azure Document Intelligence, Azure Data Factory, and Azure CosmosDB services, and provide the necessary details for the configuration.

![Use Cases and Additional Services](./UseCaseSetup3.png)
<img src="./UseCaseSetup3.png" alt="Use cases" width="800" height="600">

When selecting 'On Your Data' as the use case, you have an option to use a dedicated Azure OpenAI instance for the purpose of orchestrating indexing and content generation based on the data that you have ingested in the storage account, and indexed in Azure AI Search, and generate the embeddings. You can deploy this to a remote subscription (recommended to not compete about quota with the main Azure OpenAI instance), and provide the necessary details for the configuration.

![Use Cases and Additional Services](./UseCaseSetup4.png)
<img src="./UseCaseSetup4.png" alt="Use cases" width="800" height="600">

** 'Proof of Concept' and 'On Your Data' use case **

If you selected the intent of the deployment to be "Proof of Concept", you can optionally deploy a sample Web Application into its own resoucre group, that will be configured to interact with Azure OpenAI instance, and start generating content based on the data that you have ingested in an Azure native RAG architecture.

For this to work, it requires an Application Registration in Entra ID. You can choose to use an existing one, or create a new one, where you provide the secret value (as secure string) to the UX, and the Application ID will be used to configure the Web Application.
This requires an Application Registration in Entra ID. You can choose to use an existing one, or create a new one, where you provide the secret value (as secure string) to the UX, and the Application ID will be used to configure the Web Application.

>Note: You can also create the App registration later if you don't have the required permission in your Entra ID. Follow the instructions in the [Getting started post deployment](#getting-started-post-deployment) section to create the App registration and configure the Web Application.

![Use Cases and Additional Services](./AppSetup1.png)
<img src="./AppSetup1.png" alt="Use cases" width="800" height="600">

When creating a new App Registration, this can be done directly in the deployment UX. Select New, and provide the necessary details for the App Registration, and the secret value.

![Use Cases and Additional Services](./AppSetup2.png)
<img src="./AppSetup2.png" alt="Use cases" width="1100" height="600">

Once you click register, a new window will open where you can copy the secret value, and provide it to the UX.

Create the new secret, and copy the value of the secret to your clip board. Once you close the window (upper right corner), you can paste the value into the UX in the password field.

![Use Cases and Additional Services](./AppSetup3.png)
<img src="./AppSetup3.png" alt="Use cases" width="1100" height="600">

Paste the secret value and continue with the configuration for the rest of the services required for 'On Your Data' use case.

![Use Cases and Additional Services](./AppSetup4.png)
<img src="./AppSetup4.png" alt="Use cases" width="600" height="200">

### Review + create

*Review + Create* page will validate your permission and configuration before you can click deploy. Once it has been validated successfully, you can click *Create*
*Review + Create* page will validate your permission and configuration before you can click deploy. Once it has been validated successfully, you can click *Create* and the deployment will start.

>Note: Subject to your configuration, the deployment can take up to 20-30 minutes to complete, and you can monitor the deployment progress in the Azure portal.

## Getting started post deployment

Subject to the deployment options you have selected, you may need to do additional configuration in your environment, such as Private DNS Zones creation and conditional forwarding if used over private endpoints, as well as additional RBAC assignment for users/groups/SPNs to interact with the services.

## Azure native RAG on your own data

If you are interested in getting started with the initial Gen AI use case (e.g., Azure native RAG architecture and setup) to accelerate the adoption of Generative AI in your organization, the following instructions and script examples can be used to:

1. Upload files, such as text, images, and videos, to the storage account that you have created as part of the deployment, subject to the configuration you have selected (e.g., the setup highly recommends using Azure RBAC, disabling SAS tokens, and using customer-managed keys for encryption at rest, but if you have selected anything differently, you need to cater for that while getting the data into your storage account).

2. Use the Azure Open AI ingestion API to create one or more indexes in Azure AI Search, to start indexing the data that you have uploaded to the storage account, and make it available for the Azure Open AI instance to enable typical RAG use cases.

3. Use the Azure Open AI API to interact with the Azure Open AI instance, and start generating content based on the data that you have uploaded to the storage account, and indexed in Azure AI Search.

### PowerShell script for RAG enablement

The following PowerShell scripts can be used to 1) start an ingestion job on Azure Open AI to ingest the data from the storage account into Azure AI Search, and 2) access the Azure Open AI API to start generating content based on the data that you have ingested.

#### Ingestion job using Azure Open AI, AI Search, and Storage Account

Modify this script to provide the necessary values for the Azure Open AI endpoint, the embedding deployment name, the ingestion job name, the storage account endpoint, the storage container name, the storage resource ID, and the Azure AI Search endpoint.

```powershell

# Ingestion job using Azure Open AI, AI Search, and Storage Account. The following snippet assumes Managed Identity is properly configured and has the necessary permissions to access the resources, and that the user has Open AI contributor role on the Azure Open AI resource.

# Azure Open AI configuration

$AzureOpenAIEndpoint = ""
$EmbeddingDeploymentName = ""
$IngestionJobName = ""

# Storage Configuration

$StorageAccountEndpoint = ""
$StorageContainerName = ""
$StorageResourceId = ""
Depending on the options you have selected, you can continue the validation following the instructions in the [use cases](./use-cases.md) documentation, such as:

# Azure AI search configuraton
*For Production:*

$AzureAiSearchEndpoint = ""
* [Image and Video Recognition](./use-cases.md#image-and-video-recognition)
* ['On Your Data'](./use-cases.md#on-your-data)

# Get Token
*For Proof of Concept:*

$TokenRequest = Get-AzAccessToken -ResourceUrl "https://cognitiveservices.azure.com"
$MyToken = $TokenRequest.token

# Set Body (body must be present but empty for the request)
$Body = @'
{
}
'@

# AI Ingestion Request
$AzureOAIRequest = @{
Uri = "https://$($AzureOpenAIEndpoint)/openai/extensions/on-your-data/ingestion-jobs/$($IngestionJobName)?api-version=2023-10-01-preview"
Headers = @{
Authorization = "Bearer $($MyToken)"
'Content-Type' = 'application/json'
'storageEndpoint' = "https://$($StorageAccountEndpoint)"
'storageConnectionString' = "ResourceId=$($StorageResourceId)"
'storageContainer' = $StorageContainerName
'searchServiceEndpoint' = "https://$($AzureAiSearchEndpoint)"
'embeddingDeploymentName' = $EmbeddingDeploymentName
}
Body = $Body
Method = 'PUT'
}

$Response = Invoke-WebRequest @AzureOAIRequest
[Newtonsoft.Json.Linq.JObject]::Parse($Response.Content).ToString()

# Get Status on the ingestion job

$GetStatusRequest = @{
Uri = "https://$($AzureOpenAIEndpoint)/openai/extensions/on-your-data/ingestion-jobs/$($IngestionJob)?api-version=2023-10-01-preview"
Headers = @{
Authorization = "Bearer $($MyToken)"
}
Method = 'GET'
}
$GetResponse = Invoke-WebRequest @GetStatusRequest
[Newtonsoft.Json.Linq.JObject]::Parse($GetResponse.Content).ToString()

```

#### Content generation using Azure Open AI API endpoint

Modify this script to provide the necessary values for the Azure Open AI endpoint, the embedding deployment name, and the model name.

```powershell

#T he following snippet assumes Managed Identity is properly configured and has the necessary permissions to access the resources, and that the user has Open AI reader role on the Azure Open AI resource.

# Azure Open AI configuration

$AzureOpenAIEndpoint = ""
$DeploymentName = ""
$EmbeddingDeploymentName = ""
$Prompt = ""

# Azure AI search configuraton

$AzureAiSearchEndpoint = ""
$IndexName = ""

# Get Token
$TokenRequest = Get-AzAccessToken -ResourceUrl "https://cognitiveservices.azure.com"
$MyToken = $TokenRequest.token

# Form the request body towards the Azure Open AI API endpoint, with AzureCognitiveSearch added as dataSource for RAG
$Body = @"
{
"dataSources": [
{
"type": "AzureCognitiveSearch",
"parameters": {
"endpoint": "https://$($AzureAiSearchEndpoint)",
"indexName": "$($IndexName)",
"embeddingDeploymentName": "$($EmbeddingDeploymentName)"
}
}
],
"messages": [
{
"role": "system",
"content": "You are an AI assistant that helps people find information."
},
{
"role": "user",
"content": "$($Prompt)"
}
]
}
"@

# AI Request
$AzureOAIRequest = @{
Uri = "https://$($AzureOpenAIEndpoint)/openai/deployments/$($DeploymentName)/extensions/chat/completions?api-version=2023-10-01-preview"
Headers = @{
Authorization = "Bearer $($MyToken)"
'Content-Type' = 'application/json'
}
Method = 'POST'
Body = $Body
#UseBasicParsing = $true
}
$Response = Invoke-WebRequest @AzureOAIRequest
[Newtonsoft.Json.Linq.JObject]::Parse($Response.Content).ToString()

```
* [Multi-region deployment with APIM](./use-cases.md#multi-region-deployment-with-apim)
* ['On Your Data' with sample Web Application](./use-cases.md#on-your-data-with-sample-web-application)

## Next Steps

Expand Down
2 changes: 0 additions & 2 deletions docs/use-cases-overview.md
View file
Open in desktop

This file was deleted.