Skip to content

Latest commit

 

History

History
171 lines (128 loc) · 34 KB

README.md

File metadata and controls

171 lines (128 loc) · 34 KB
Raw

Module: Azure OpenAI Private ChatGPT

NOTE: Your Azure subscription will need to be whitelisted for Azure Open AI. At the release time of this module (August 2023) you will need to request access via this form and a further form for GPT 4. Once you have access deploy either GPT-35-Turbo, GPT-35-Turbo-16k or if you have access to GPT-4-32k, go forward with that model.

Easily construct a ChatGPT-style interface using Azure OpenAI and a suite of Azure services. Simplifying the complex.

image.png

Introduction

Under OpenAI's terms when using the public version of ChatGPT, any questions you pose—referred to as "prompts"—may contribute to the further training of OpenAI's Large Language Model (LLM). Given this, it's crucial to ask: Are you comfortable with this precious data flow leaving your organization? If you're a decision-maker or hold responsibility over your organization's security measures, what steps are you taking to ensure proprietary information remains confidential?

An effective solution lies in utilising a hosted version of the popular LLM on Azure OpenAI. While there are numerous advantages to Azure OpenAI, I'd like to spotlight two:

  • Data Privacy: By hosting OpenAI's models on Azure, your prompts will never serve as a source for training the LLM. It's simply a self-contained version running on Azure tailored for your use.

  • Enhanced Security: Azure OpenAI offers robust security measures, from the capability to secure specific endpoints to intricate role-based access controls. For a deeper dive, refer to this Microsoft Learn article.

While Azure OpenAI does come with a cost, it's highly affordable—often, a conversation costs under 10 cents. You can review Azure OpenAI's pricing details here.

This terraform module helps establishing a secure ChatGPT-like interface. This uses Azure OpenAI, combined with an array of Azure's other services, such as Azure Container App, Azure Front Door / CDN, Web Application Firewall, Key Vault and Azure DNS, ensuring a confidential and dedicated ChatGPT experience for you.

Diagram

image.png

Description

This flexible terraform module is an OpenAI accelerator that can be used to deploy a privately hosted instance of a ChatBot similar to ChatGPT hosted on Azure using Azure Container Apps, Azure OpenAI and optionally fronted by Azure CDN/Front Door with a WAF / Firewall and custom allowed IP list.

This module can be used to create the following

Create OpenAI Service

  1. Create an Azure Key Vault to store the OpenAI account details.

  2. Create an OpenAI service account.
    Other options include:

    • Specify an already existing OpenAI service account to use.

  3. Create OpenAI language model deployments on the OpenAI service. (e.g. GPT-3, GPT-4, etc.)

  4. Store the OpenAI account and model details in the key vault for consumption.

Create a container app ChatBot UI linked with OpenAI service hosted in Azure

  1. Create a container app log analytics workspace (to link with container app).
  2. Create a container app environment.
  3. Create a container app instance hosting chatbot-ui from image/container.
  4. Link chatbot-ui with corresponding OpenAI account and language model deployment.
  5. Grant the container app access to the key vault to retrieve secrets (optional).

Front solution with an Azure front door (optional)

  1. Deploy Azure Front Door to front solution with CDN + WAF.

  2. Setup a custom domain in Azure Front Door with AFD managed certificate. Other options include:

    • This example specifies an already existing DNZ zone to use. (e.g. existingzone.com - see common.auto.tfvars)
    • Note: Remember to add the zone to your DNS registrar as the module creates a TXT auth. (Certificates fully managed by AFD)

  3. Create a CNAME and TXT record in the custom DNS zone. (e.g. privategpt.existingzone.com)

  4. Setup and apply an AFD WAF policy with IPAllow list for allowed IPs to connect using a custom rule.

ChatBot Demo

image.png

Examples

See Private ChatGPT with Azure Front Door + Firewall on existing DNS zone:
For an example of how to create a Privately hosted instance of ChatBot/ChatGPT on Azure OpenAI with AFD + WAF using an existing DNS zone for the custom domain configuration.

See Private ChatGPT with Azure Front Door + Firewall on new DNS zone:
For an example of how to create a Privately hosted instance of ChatBot/ChatGPT on Azure OpenAI with AFD + WAF using a new DNS zone for the custom domain configuration.

See Private ChatGPT instance only:
For an example of how to create a Privately hosted instance of ChatBot/ChatGPT on Azure OpenAI only. (No AFD + WAF + DNS zone)

This module is published on the Public Terraform Registry - openai-private-chatgpt

Enjoy!

Requirements

Name Version
terraform >= 1.6.5
azurerm ~> 3.87.0

Providers

No providers.

Modules

Name Source Version
azure_frontdoor_cdn ./modules/cdn_frontdoor n/a
openai Pwd9000-ML/openai-service/azurerm >= 1.1.0
privategpt_chatbot_container_apps ./modules/container_app n/a

Resources

No resources.

Inputs

Name Description Type Default Required
ca_container_config type = object({
name = (Required) The name of the container.
image = (Required) The name of the container image.
cpu = (Required) The number of CPU cores to allocate to the container.
memory = (Required) The amount of memory to allocate to the container in GB.
min_replicas = (Optional) The minimum number of replicas to run. Defaults to 0.
max_replicas = (Optional) The maximum number of replicas to run. Defaults to 10.
env = list(object({
name = (Required) The name of the environment variable.
secret_name = (Optional) The name of the secret to use for the environment variable.
value = (Optional) The value of the environment variable.
}))
})		 
object({
    name         = string
    image        = string
    cpu          = number
    memory       = string
    min_replicas = optional(number, 0)
    max_replicas = optional(number, 10)
    env = optional(list(object({
      name        = string
      secret_name = optional(string)
      value       = optional(string)
    })))
  })
 
{
  "cpu": 1,
  "env": [],
  "image": "ghcr.io/pwd9000-ml/chatbot-ui:main",
  "max_replicas": 10,
  "memory": "2Gi",
  "min_replicas": 0,
  "name": "gpt-chatbot-ui"
}
 no
ca_identity type = object({
type = (Required) The type of the Identity. Possible values are SystemAssigned, UserAssigned, SystemAssigned, UserAssigned.
identity_ids = (Optional) Specifies a list of User Assigned Managed Identity IDs to be assigned to this OpenAI Account.
})		 
object({
    type         = string
    identity_ids = optional(list(string))
  })
 null no
ca_ingress type = object({
allow_insecure_connections = (Optional) Allow insecure connections to the container app. Defaults to false.
external_enabled = (Optional) Enable external access to the container app. Defaults to true.
target_port = (Required) The port to use for the container app. Defaults to 3000.
transport = (Optional) The transport protocol to use for the container app. Defaults to auto.
type = object({
percentage = (Required) The percentage of traffic to route to the container app. Defaults to 100.
latest_revision = (Optional) The percentage of traffic to route to the container app. Defaults to true.
})		 
object({
    allow_insecure_connections = optional(bool)
    external_enabled           = optional(bool)
    target_port                = number
    transport                  = optional(string)
    traffic_weight = optional(object({
      percentage      = number
      latest_revision = optional(bool)
    }))
  })
 
{
  "allow_insecure_connections": false,
  "external_enabled": true,
  "target_port": 3000,
  "traffic_weight": {
    "latest_revision": true,
    "percentage": 100
  },
  "transport": "auto"
}
 no
ca_name Name of the container app to create. string "gptca" no
ca_resource_group_name Name of the resource group to create the Container App and supporting solution resources in. string n/a yes
ca_revision_mode Revision mode of the container app to create. string "Single" no
ca_secrets type = list(object({
name = (Required) The name of the secret.
value = (Required) The value of the secret.
}))		 
list(object({
    name  = string
    value = string
  }))
 
[
  {
    "name": "secret1",
    "value": "value1"
  },
  {
    "name": "secret2",
    "value": "value2"
  }
]
 no
cae_name Name of the container app environment to create. string "gptcae" no
cdn_endpoint typp = object({
name = (Required) The name of the CDN endpoint to create.
enabled = (Optional) Is the CDN endpoint enabled? Defaults to true.
})		 
object({
    name    = string
    enabled = optional(bool, true)
  })
 
{
  "enabled": true,
  "name": "PrivateGPT"
}
 no
cdn_firewall_policy The CDN firewall policies to create. 
object({
    create_waf                        = bool
    name                              = string
    enabled                           = optional(bool, true)
    mode                              = optional(string, "Prevention")
    redirect_url                      = optional(string)
    custom_block_response_status_code = optional(number, 403)
    custom_block_response_body        = optional(string)
    custom_rules = optional(list(object({
      name                           = string
      action                         = string
      enabled                        = optional(bool, true)
      priority                       = number
      type                           = string
      rate_limit_duration_in_minutes = optional(number, 1)
      rate_limit_threshold           = optional(number, 10)
      match_conditions = list(object({
        match_variable     = string
        match_values       = list(string)
        operator           = string
        selector           = optional(string)
        negation_condition = optional(bool)
        transforms         = optional(list(string))
      }))
    })))
  })
 
{
  "create_waf": true,
  "custom_block_response_body": "WW91ciByZXF1ZXN0IGhhcyBiZWVuIGJsb2NrZWQu",
  "custom_block_response_status_code": 403,
  "custom_rules": [
    {
      "action": "Block",
      "enabled": true,
      "match_conditions": [
        {
          "match_values": [
            "10.0.1.0/24",
            "10.0.2.0/24"
          ],
          "match_variable": "RemoteAddr",
          "negation_condition": null,
          "operator": "IPMatch",
          "selector": null,
          "transforms": []
        }
      ],
      "name": "PrivateGPTFirewallPolicyCustomRule",
      "priority": 100,
      "rate_limit_duration_in_minutes": 1,
      "rate_limit_threshold": 10,
      "type": "MatchRule"
    }
  ],
  "enabled": true,
  "mode": "Prevention",
  "name": "PrivateGPTFirewallPolicy",
  "redirect_url": null
}
 no
cdn_gpt_origin type = object({
name = (Required) The name which should be used for this Front Door Origin. Changing this forces a new Front Door Origin to be created.
origin_group_name = (Required) The name of the CDN origin group to associate this origin with.
enabled = (Optional) Is the CDN origin enabled? Defaults to true.
certificate_name_check_enabled = (Required) Specifies whether certificate name checks are enabled for this origin. Defaults to true.
http_port = (Optional) The HTTP port of the origin. (e.g. 80)
https_port = (Optional) The HTTPS port of the origin. (e.g. 443)
priority = (Optional) The priority of the origin. (e.g. 1)
weight = (Optional) The weight of the origin. (e.g. 1000)
})		 
object({
    name                           = string
    origin_group_name              = string
    enabled                        = optional(bool, true)
    certificate_name_check_enabled = optional(bool, true)
    http_port                      = optional(number, 80)
    https_port                     = optional(number, 443)
    priority                       = optional(number, 1)
    weight                         = optional(number, 1000)
  })
 
{
  "certificate_name_check_enabled": true,
  "enabled": true,
  "http_port": 80,
  "https_port": 443,
  "name": "PrivateGPTOrigin",
  "origin_group_name": "PrivateGPTOriginGroup",
  "priority": 1,
  "weight": 1000
}
 no
cdn_origin_groups type = list(object({
name = (Required) The name of the CDN origin group to create.
session_affinity_enabled = (Optional) Is session affinity enabled? Defaults to false.
restore_traffic_time_to_healed_or_new_endpoint_in_minutes = (Optional) The time in minutes to restore traffic to a healed or new endpoint. Defaults to 5.
health_probe = (Optional) The health probe settings.
type = object({
interval_in_seconds = (Optional) The interval in seconds between health probes. Defaults to 100.
path = (Optional) The path to use for health probes. Defaults to /.
protocol = (Optional) The protocol to use for health probes. Possible values include 'Http' and 'Https'. Defaults to Http.
request_type = (Optional) The request type to use for health probes. Possible values include 'GET', 'HEAD', and 'OPTIONS'. Defaults to HEAD.
}))
load_balancing = (Optional) The load balancing settings.
type = object({
additional_latency_in_milliseconds = (Optional) The additional latency in milliseconds for probes to fall into the lowest latency bucket. Defaults to 50.
sample_size = (Optional) The number of samples to take for load balancing decisions. Defaults to 4.
successful_samples_required = (Optional) The number of samples within the sample period that must succeed. Defaults to 3.
}))
}))		 
list(object({
    name                                                      = string
    session_affinity_enabled                                  = optional(bool, false)
    restore_traffic_time_to_healed_or_new_endpoint_in_minutes = optional(number, 5)
    health_probe = optional(object({
      interval_in_seconds = optional(number, 100)
      path                = optional(string, "/")
      protocol            = optional(string, "Http")
      request_type        = optional(string, "HEAD")
    }))
    load_balancing = optional(object({
      additional_latency_in_milliseconds = optional(number, 50)
      sample_size                        = optional(number, 4)
      successful_samples_required        = optional(number, 3)
    }))
  }))
 
[
  {
    "health_probe": {
      "interval_in_seconds": 100,
      "path": "/",
      "protocol": "Http",
      "request_type": "HEAD"
    },
    "load_balancing": {
      "additional_latency_in_milliseconds": 50,
      "sample_size": 4,
      "successful_samples_required": 3
    },
    "name": "PrivateGPTOriginGroup",
    "restore_traffic_time_to_healed_or_new_endpoint_in_minutes": 5,
    "session_affinity_enabled": false
  }
]
 no
cdn_profile_name The name of the CDN profile to create. string "example-cdn-profile" no
cdn_resource_group_name Name of the resource group to create the CDN Front Door in. string "cdn-rg-01" no
cdn_route type = object({
name = (Required) The name of the CDN route to create.
enabled = (Optional) Is the CDN route enabled? Defaults to true.
forwarding_protocol = (Optional) The protocol this rule will use when forwarding traffic to backends. Possible values include MatchRequest, HttpOnly and HttpsOnly. Defaults to HttpsOnly.
https_redirect_enabled = (Optional) Is HTTPS redirect enabled? Defaults to false.
patterns_to_match = (Optional) The list of patterns to match for this rule. Defaults to ["/*"].
supported_protocols = (Optional) The list of supported protocols for this rule. Defaults to ["Http", "Https"].
cdn_frontdoor_origin_path = (Optional) The path to use when forwarding traffic to backends. Defaults to null.
cdn_frontdoor_rule_set_ids = (Optional) The list of rule set IDs to associate with this rule. Defaults to null.
link_to_default_domain = (Optional) Is the CDN route linked to the default domain? Defaults to false.
cache = (Optional) The CDN route cache settings.
type = object({
query_string_caching_behavior = (Required) The query string caching behavior. Possible values include 'IgnoreQueryString', 'BypassCaching', 'UseQueryString', and 'NotSet'. Defaults to 'IgnoreQueryString'.
query_strings = (Optional) The list of query strings to include or exclude from caching. Defaults to [].
compression_enabled = (Required) Is compression enabled? Defaults to false.
content_types_to_compress = (Optional) The list of content types to compress. Defaults to [].
})
})		 
object({
    name                       = string
    enabled                    = optional(bool, true)
    forwarding_protocol        = optional(string, "HttpsOnly")
    https_redirect_enabled     = optional(bool, false)
    patterns_to_match          = optional(list(string), ["/*"])
    supported_protocols        = optional(list(string), ["Http", "Https"])
    cdn_frontdoor_origin_path  = optional(string, null)
    cdn_frontdoor_rule_set_ids = optional(list(string), null)
    link_to_default_domain     = optional(bool, false)
    cache = optional(object({
      query_string_caching_behavior = string
      query_strings                 = optional(list(string), [])
      compression_enabled           = bool
      content_types_to_compress     = optional(list(string), [])
    }))
  })
 
{
  "cache": {
    "compression_enabled": false,
    "content_types_to_compress": [],
    "query_string_caching_behavior": "IgnoreQueryString",
    "query_strings": []
  },
  "cdn_frontdoor_origin_path": null,
  "cdn_frontdoor_rule_set_ids": null,
  "enabled": true,
  "forwarding_protocol": "HttpsOnly",
  "https_redirect_enabled": false,
  "link_to_default_domain": false,
  "name": "PrivateGPTRoute",
  "patterns_to_match": [
    "/*"
  ],
  "supported_protocols": [
    "Http",
    "Https"
  ]
}
 no
cdn_security_policy type = object({
name = (Required) The name of the CDN security policy to create.
patterns_to_match = (Required) The list of patterns to match for this policy. Defaults to ["/*"].
})		 
object({
    name              = string
    patterns_to_match = list(string)
  })
 
{
  "name": "PrivateGPTSecurityPolicy",
  "patterns_to_match": [
    "/*"
  ]
}
 no
cdn_sku_name Specifies the SKU for the CDN Front Door Profile. Possible values include 'Standard_AzureFrontDoor' and 'Premium_AzureFrontDoor'. string "Standard_AzureFrontDoor" no
create_dns_zone Create a DNS zone for the CDN profile. If set to false, an existing DNS zone must be provided. bool false no
create_front_door_cdn Create a Front Door profile. bool false no
create_model_deployment Create the model deployment. bool false no
create_openai_service Create the OpenAI service. bool false no
custom_domain_config type = object({
zone_name = (Required) The name of the DNS zone to create the CNAME and TXT record in for the CDN Front Door Custom domain.
host_name = (Required) The host name of the DNS record to create. (e.g. Contoso)
ttl = (Optional) The TTL of the DNS record to create. (e.g. 3600)
tls = optional(list(object({
certificate_type = (Optional) Defines the source of the SSL certificate. Possible values include 'CustomerCertificate' and 'ManagedCertificate'. Defaults to 'ManagedCertificate'.
NOTE: It may take up to 15 minutes for the Front Door Service to validate the state and Domain ownership of the Custom Domain.
minimum_tls_version = (Optional) TLS protocol version that will be used for Https. Possible values include TLS10 and TLS12. Defaults to TLS12.
}))))
})		 
object({
    zone_name = string
    host_name = string
    ttl       = optional(number, 3600)
    tls = optional(list(object({
      certificate_type    = optional(string, "ManagedCertificate")
      minimum_tls_version = optional(string, "TLS12")
    })))
  })
 
{
  "host_name": "PrivateGPT",
  "tls": [
    {
      "certificate_type": "ManagedCertificate",
      "minimum_tls_version": "TLS12"
    }
  ],
  "ttl": 3600,
  "zone_name": "mydomain7335.com"
}
 no
dns_resource_group_name The name of the resource group to create the DNS zone in / or where the existing zone is hosted. string "dns-rg-01" no
key_vault_access_permission The permission to grant the container app to the key vault. Set this variable to null if no Key Vault access is needed. Defaults to Key Vault Secrets User. list(string) 
[
  "Key Vault Secrets User"
]
 no
key_vault_id (Optional) - The id of the key vault to grant access to. Only required if key_vault_access_permission is set. string "" no
keyvault_firewall_allowed_ips value of key vault firewall allowed ip rules. list(string) [] no
keyvault_firewall_bypass List of key vault firewall rules to bypass. string "AzureServices" no
keyvault_firewall_default_action Default action for key vault firewall rules. string "Deny" no
keyvault_firewall_virtual_network_subnet_ids value of key vault firewall allowed virtual network subnet ids. list(string) [] no
keyvault_resource_group_name Name of the resource group to create the Key Vault that will store OpenAI service account details. string n/a yes
kv_config Key Vault configuration object to create azure key vault to store openai account details. 
object({
    name = string
    sku  = string
  })
 
{
  "name": "kvname",
  "sku": "standard"
}
 no
laws_name Name of the log analytics workspace to create. string "gptlaws" no
laws_retention_in_days Retention in days of the log analytics workspace to create. number 30 no
laws_sku SKU of the log analytics workspace to create. string "PerGB2018" no
location Azure region where resources will be hosted. string "uksouth" no
model_deployment type = list(object({
deployment_id = (Required) The name of the Cognitive Services Account Model Deployment. Changing this forces a new resource to be created.
model_name = {
model_format = (Required) The format of the Cognitive Services Account Deployment model. Changing this forces a new resource to be created. Possible value is OpenAI.
model_name = (Required) The name of the Cognitive Services Account Deployment model. Changing this forces a new resource to be created.
model_version = (Required) The version of Cognitive Services Account Deployment model.
}
scale = {
scale_type = (Required) Deployment scale type. Possible value is Standard. Changing this forces a new resource to be created.
scale_tier = (Optional) Possible values are Free, Basic, Standard, Premium, Enterprise. Changing this forces a new resource to be created.
scale_size = (Optional) The SKU size. When the name field is the combination of tier and some other value, this would be the standalone code. Changing this forces a new resource to be created.
scale_family = (Optional) If the service has different generations of hardware, for the same SKU, then that can be captured here. Changing this forces a new resource to be created.
scale_capacity = (Optional) Tokens-per-Minute (TPM). If the SKU supports scale out/in then the capacity integer should be included. If scale out/in is not possible for the resource this may be omitted. Default value is 1. Changing this forces a new resource to be created.
}
rai_policy_name = (Optional) The name of RAI policy. Changing this forces a new resource to be created.
}))		 
list(object({
    deployment_id   = string
    model_name      = string
    model_format    = string
    model_version   = string
    scale_type      = string
    scale_tier      = optional(string)
    scale_size      = optional(number)
    scale_family    = optional(string)
    scale_capacity  = optional(number)
    rai_policy_name = optional(string)
  }))
 [] no
openai_account_name Name of the OpenAI service. string "demo-account" no
openai_custom_subdomain_name The subdomain name used for token-based authentication. Changing this forces a new resource to be created. (normally the same as the account name) string "demo-account" no
openai_identity type = object({
type = (Required) The type of the Identity. Possible values are SystemAssigned, UserAssigned, SystemAssigned, UserAssigned.
identity_ids = (Optional) Specifies a list of User Assigned Managed Identity IDs to be assigned to this OpenAI Account.
})		 
object({
    type = string
  })
 
{
  "type": "SystemAssigned"
}
 no
openai_local_auth_enabled Whether local authentication methods is enabled for the Cognitive Account. Defaults to true. bool true no
openai_outbound_network_access_restricted Whether or not outbound network access is restricted. Defaults to false. bool false no
openai_public_network_access_enabled Whether or not public network access is enabled. Defaults to false. bool true no
openai_resource_group_name Name of the resource group to create the OpenAI service / or where an existing service is hosted. string n/a yes
openai_sku_name SKU name of the OpenAI service. string "S0" no
tags A map of key value pairs that is used to tag resources created. map(string) {} no

Outputs

Name Description
container_app_enviornment_id The ID of the container app enviornment.
container_app_id The ID of the container app.
key_vault_id The ID of the Key Vault used to store OpenAI account and model details.
key_vault_uri The URI of the Key Vault used to store OpenAI account and model details..
latest_revision_fqdn The FQDN of the Latest Revision of the Container App.
latest_revision_name The Name of the Latest Revision of the Container App.
openai_account_name The name of the Cognitive Service Account.
openai_endpoint The endpoint used to connect to the Cognitive Service Account.
openai_primary_key The primary access key for the Cognitive Service Account.
openai_secondary_key The secondary access key for the Cognitive Service Account.
openai_subdomain The subdomain used to connect to the Cognitive Service Account.