Skip to content

Commit

Permalink
docs: pointing cognitive apis to azure ai (#2119)
Browse files Browse the repository at this point in the history
* docs: pointing cognitive apis to azure ai

* Update docs/Explore Algorithms/AI Services/Quickstart - Creare a Visual Search Engine.ipynb

* Update docs/Explore Algorithms/AI Services/Quickstart - Creare a Visual Search Engine.ipynb

---------

Co-authored-by: JessicaXYWang <108437381+JessicaXYWang@users.noreply.github.com>
  • Loading branch information
mhamilton723 and JessicaXYWang committed Nov 2, 2023
1 parent 77be641 commit d240cbb
Show file tree
Hide file tree
Showing 11 changed files with 155 additions and 138 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
"from synapse.ml.core.platform import find_secret\n",
"\n",
"service_key = find_secret(\n",
" secret_name=\"cognitive-api-key\", keyvault=\"mmlspark-build-keys\"\n",
" secret_name=\"ai-services-api-key\", keyvault=\"mmlspark-build-keys\"\n",
")\n",
"service_loc = \"eastus\""
]
Expand Down Expand Up @@ -203,7 +203,7 @@
"\n",
"<img src=\"https://mmlspark.blob.core.windows.net/graphics/Cog%20Service%20NB/cog_service.svg\" width=\"300\" />\n",
"\n",
"When we call the cognitive service transformer, we start cognitive service clients on each of your spark workers.\n",
"When we call the AI service transformer, we start cognitive service clients on each of your spark workers.\n",
"These clients send requests to the cloud, and turn the JSON responses into Spark Struct Types so that you can access any field that the service returns."
]
},
Expand Down Expand Up @@ -308,7 +308,7 @@
" [\n",
" (\"I am so happy today, its sunny!\",),\n",
" (\"I am frustrated by this rush hour traffic\",),\n",
" (\"The cognitive services on spark is pretty lit\",),\n",
" (\"The AI services on spark is pretty lit\",),\n",
" ],\n",
" [\"text\"],\n",
")\n",
Expand Down Expand Up @@ -361,7 +361,7 @@
"import random\n",
"\n",
"service_key_2 = find_secret(\n",
" secret_name=\"cognitive-api-key-2\", keyvault=\"mmlspark-build-keys\"\n",
" secret_name=\"ai-services-api-key-2\", keyvault=\"mmlspark-build-keys\"\n",
")\n",
"keys = [service_key, service_key_2]\n",
"\n",
Expand Down
22 changes: 11 additions & 11 deletions docs/Explore Algorithms/AI Services/Geospatial Services.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,18 @@
"\n",
"# Azure Maps Geospatial Services\n",
"\n",
"[Microsoft Azure Maps ](https://azure.microsoft.com/en-us/services/azure-maps/) provides developers from all industries with powerful geospatial capabilities. Those geospatial capabilities are packed with the freshest mapping data. Azure Maps is available for web, mobile (iOS and Android), Microsoft Power BI, Microsoft Power Apps and Microsoft Synapse. Azure Maps is an Open API compliant set of REST APIs. The following are only a high-level overview of the services which Azure Maps offers - Maps, Search, Routing, Traffic, Weather, Time Zones, Geolocation, Geofencing, Map Data, Creator, and Spatial Operations.\n",
"[Microsoft Azure Maps ](https://azure.microsoft.com/services/azure-maps/) provides developers from all industries with powerful geospatial capabilities. Those geospatial capabilities are packed with the freshest mapping data. Azure Maps is available for web, mobile (iOS and Android), Microsoft Power BI, Microsoft Power Apps and Microsoft Synapse. Azure Maps is an Open API compliant set of REST APIs. The following are only a high-level overview of the services which Azure Maps offers - Maps, Search, Routing, Traffic, Weather, Time Zones, Geolocation, Geofencing, Map Data, Creator, and Spatial Operations.\n",
"\n",
"## Usage\n",
"\n",
"### Geocode addresses\n",
"[**Address Geocoding**](https://docs.microsoft.com/en-us/rest/api/maps/search/post-search-address-batch) The Search Address Batch API sends batches of queries to Search Address API using just a single API call. This API geocodes text addresses or partial addresses and the geocoding search index will be queried for everything above the street level data. **Note** that the geocoder is very tolerant of typos and incomplete addresses. It will also handle everything from exact street addresses or street or intersections as well as higher level geographies such as city centers, counties, states etc.\n",
"[**Address Geocoding**](https://docs.microsoft.com/rest/api/maps/search/post-search-address-batch) The Search Address Batch API sends batches of queries to Search Address API using just a single API call. This API geocodes text addresses or partial addresses and the geocoding search index will be queried for everything above the street level data. **Note** that the geocoder is very tolerant of typos and incomplete addresses. It will also handle everything from exact street addresses or street or intersections as well as higher level geographies such as city centers, counties, states etc.\n",
"\n",
"### Reverse Geocode Coordinates\n",
"[**Reverse Geocoding**](https://docs.microsoft.com/en-us/rest/api/maps/search/post-search-address-reverse-batch) The Search Address Reverse Batch API sends batches of queries to Search Address Reverse API using just a single API call. This API takes in location coordinates and translates them into human readable street addresses. Most often this is needed in tracking applications where you receive a GPS feed from the device or asset and wish to know what address where the coordinate is located.\n",
"[**Reverse Geocoding**](https://docs.microsoft.com/rest/api/maps/search/post-search-address-reverse-batch) The Search Address Reverse Batch API sends batches of queries to Search Address Reverse API using just a single API call. This API takes in location coordinates and translates them into human readable street addresses. Most often this is needed in tracking applications where you receive a GPS feed from the device or asset and wish to know what address where the coordinate is located.\n",
"\n",
"### Get Point In Polygon\n",
"[**Get Point in Polygon**](https://docs.microsoft.com/en-us/rest/api/maps/spatial/get-point-in-polygon) This API returns a boolean value indicating whether a point is inside a set of polygons. The set of polygons can we pre-created by using the [**Data Upload API**](https://docs.microsoft.com/en-us/rest/api/maps/data/upload-preview) referenced by a unique udid."
"[**Get Point in Polygon**](https://docs.microsoft.com/rest/api/maps/spatial/get-point-in-polygon) This API returns a boolean value indicating whether a point is inside a set of polygons. The set of polygons can we pre-created by using the [**Data Upload API**](https://docs.microsoft.com/rest/api/maps/data/upload-preview) referenced by a unique udid."
]
},
{
Expand All @@ -28,12 +28,12 @@
"source": [
"## Prerequisites\n",
"\n",
"1. Sign into the [Azure Portal](https://portal.azure.com) and create an Azure Maps account by following these [instructions](https://docs.microsoft.com/en-us/azure/azure-maps/how-to-manage-account-keys#create-a-new-account).\n",
"1. Once the Maps account is created, provision a Maps Creator Resource by following these [instructions](https://docs.microsoft.com/en-us/azure/azure-maps/how-to-manage-creator#create-creator-resource). Creator is a [geographically scoped service](https://docs.microsoft.com/en-us/azure/azure-maps/creator-geographic-scope). Pick appropriate location while provisioning the creator resource. \n",
"1. Follow these [instructions](https://docs.microsoft.com/en-us/azure/cognitive-services/big-data/getting-started#create-an-apache-spark-cluster) to set up your Azure Databricks environment and install SynapseML.\n",
"1. Sign into the [Azure Portal](https://portal.azure.com) and create an Azure Maps account by following these [instructions](https://docs.microsoft.com/azure/azure-maps/how-to-manage-account-keys#create-a-new-account).\n",
"1. Once the Maps account is created, provision a Maps Creator Resource by following these [instructions](https://docs.microsoft.com/azure/azure-maps/how-to-manage-creator#create-creator-resource). Creator is a [geographically scoped service](https://docs.microsoft.com/azure/azure-maps/creator-geographic-scope). Pick appropriate location while provisioning the creator resource. \n",
"1. Follow these [instructions](https://learn.microsoft.com/azure/synapse-analytics/machine-learning/overview-cognitive-services#create-an-apache-spark-cluster) to set up your Azure Databricks environment and install SynapseML.\n",
"1. After you create a new notebook in Azure Databricks, copy the **Shared code** below and paste into a new cell in your notebook.\n",
"1. Choose a service sample, below, and copy paste it into a second new cell in your notebook.\n",
"1. Replace the `AZUREMAPS_API_KEY` placeholders with your own [Maps account key](https://docs.microsoft.com/en-us/azure/azure-maps/how-to-manage-authentication#view-authentication-details).\n",
"1. Replace the `AZUREMAPS_API_KEY` placeholders with your own [Maps account key](https://docs.microsoft.com/azure/azure-maps/how-to-manage-authentication#view-authentication-details).\n",
"1. Choose the run button (triangle icon) in the upper right corner of the cell, then select **Run Cell**.\n",
"1. View results in a table below the cell."
]
Expand Down Expand Up @@ -91,7 +91,7 @@
"source": [
"## Geocoding sample\n",
"\n",
"The azure maps geocoder sends batches of queries to the [Search Address API](https://docs.microsoft.com/en-us/rest/api/maps/search/getsearchaddress). The API limits the batch size to 10000 queries per request. "
"The azure maps geocoder sends batches of queries to the [Search Address API](https://docs.microsoft.com/rest/api/maps/search/getsearchaddress). The API limits the batch size to 10000 queries per request. "
]
},
{
Expand Down Expand Up @@ -155,7 +155,7 @@
"source": [
"## Reverse Geocoding sample\n",
"\n",
"The azure maps reverse geocoder sends batches of queries to the [Search Address Reverse API](https://docs.microsoft.com/en-us/rest/api/maps/search/get-search-address-reverse) using just a single API call. The API allows caller to batch up to 10,000 queries per request"
"The azure maps reverse geocoder sends batches of queries to the [Search Address Reverse API](https://docs.microsoft.com/rest/api/maps/search/get-search-address-reverse) using just a single API call. The API allows caller to batch up to 10,000 queries per request"
]
},
{
Expand Down Expand Up @@ -214,7 +214,7 @@
"source": [
"## Check Point In Polygon sample\n",
"\n",
"This API returns a boolean value indicating whether a point is inside a set of polygons. The polygon can be added to your creator account using the [**Data Upload API**](https://docs.microsoft.com/en-us/rest/api/maps/data/upload-preview). The API then returns a unique udid to reference the polygon."
"This API returns a boolean value indicating whether a point is inside a set of polygons. The polygon can be added to your creator account using the [**Data Upload API**](https://docs.microsoft.com/rest/api/maps/data/upload-preview). The API then returns a unique udid to reference the polygon."
]
},
{
Expand Down
4 changes: 2 additions & 2 deletions docs/Explore Algorithms/AI Services/Overview.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -160,8 +160,8 @@
"\n",
"# A general AI services key for Text Analytics, Computer Vision and Form Recognizer (or use separate keys that belong to each service)\n",
"service_key = find_secret(\n",
" secret_name=\"cognitive-api-key\", keyvault=\"mmlspark-build-keys\"\n",
") # Replace it with your ai service key, check prerequisites for more details\n",
" secret_name=\"ai-services-api-key\", keyvault=\"mmlspark-build-keys\"\n",
") # Replace the call to find_secret with your key as a python string. e.g. service_key=\"27snaiw...\"\n",
"service_loc = \"eastus\"\n",
"\n",
"# A Bing Search v7 subscription key\n",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@
"from synapse.ml.core.platform import find_secret\n",
"\n",
"# put your service keys here\n",
"cognitive_key = find_secret(\n",
" secret_name=\"cognitive-api-key\", keyvault=\"mmlspark-build-keys\"\n",
"ai_service_key = find_secret(\n",
" secret_name=\"ai-services-api-key\", keyvault=\"mmlspark-build-keys\"\n",
")\n",
"cognitive_location = \"eastus\"\n",
"ai_service_location = \"eastus\"\n",
"bing_search_key = find_secret(\n",
" secret_name=\"bing-search-key\", keyvault=\"mmlspark-build-keys\"\n",
")"
Expand Down Expand Up @@ -96,8 +96,8 @@
"source": [
"celebs = (\n",
" RecognizeDomainSpecificContent()\n",
" .setSubscriptionKey(cognitive_key)\n",
" .setLocation(cognitive_location)\n",
" .setSubscriptionKey(ai_service_key)\n",
" .setLocation(ai_service_location)\n",
" .setModel(\"celebrities\")\n",
" .setImageUrlCol(\"url\")\n",
" .setOutputCol(\"celebs\")\n",
Expand Down Expand Up @@ -131,8 +131,8 @@
"\n",
"recognizeText = (\n",
" RecognizeText()\n",
" .setSubscriptionKey(cognitive_key)\n",
" .setLocation(cognitive_location)\n",
" .setSubscriptionKey(ai_service_key)\n",
" .setLocation(ai_service_location)\n",
" .setImageUrlCol(\"url\")\n",
" .setMode(\"Printed\")\n",
" .setOutputCol(\"ocr\")\n",
Expand Down Expand Up @@ -174,8 +174,8 @@
"source": [
"sentimentTransformer = (\n",
" TextSentiment()\n",
" .setLocation(cognitive_location)\n",
" .setSubscriptionKey(cognitive_key)\n",
" .setLocation(ai_service_location)\n",
" .setSubscriptionKey(ai_service_key)\n",
" .setTextCol(\"text\")\n",
" .setOutputCol(\"sentiment\")\n",
")\n",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# AI Services - Analyze Text\n",
"\n"
"# Azure AI Services - Analyze Text"
]
},
{
Expand All @@ -16,10 +15,10 @@
"source": [
"from synapse.ml.core.platform import find_secret\n",
"\n",
"cognitive_key = find_secret(\n",
" secret_name=\"cognitive-api-key\", keyvault=\"mmlspark-build-keys\"\n",
"ai_service_key = find_secret(\n",
" secret_name=\"ai-services-api-key\", keyvault=\"mmlspark-build-keys\"\n",
")\n",
"cognitive_location = \"eastus\""
"ai_service_location = \"eastus\""
]
},
{
Expand Down Expand Up @@ -56,8 +55,8 @@
"\n",
"text_analyze = (\n",
" TextAnalyze()\n",
" .setLocation(cognitive_location)\n",
" .setSubscriptionKey(cognitive_key)\n",
" .setLocation(ai_service_location)\n",
" .setSubscriptionKey(ai_service_key)\n",
" .setTextCol(\"text\")\n",
" .setOutputCol(\"textAnalysis\")\n",
" .setErrorCol(\"error\")\n",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@
"from pyspark.sql.functions import lit, udf, col, split\n",
"from synapse.ml.core.platform import *\n",
"\n",
"cognitive_key = find_secret(\n",
" secret_name=\"cognitive-api-key\", keyvault=\"mmlspark-build-keys\"\n",
"ai_service_key = find_secret(\n",
" secret_name=\"ai-services-api-key\", keyvault=\"mmlspark-build-keys\"\n",
")\n",
"cognitive_loc = \"eastus\"\n",
"ai_service_loc = \"eastus\"\n",
"azure_search_key = find_secret(\n",
" secret_name=\"azure-search-key\", keyvault=\"mmlspark-build-keys\"\n",
")\n",
Expand Down Expand Up @@ -73,8 +73,8 @@
"# define pipeline\n",
"describeImage = (\n",
" AnalyzeImage()\n",
" .setSubscriptionKey(cognitive_key)\n",
" .setLocation(cognitive_loc)\n",
" .setSubscriptionKey(ai_service_key)\n",
" .setLocation(ai_service_loc)\n",
" .setImageUrlCol(\"PrimaryImageUrl\")\n",
" .setOutputCol(\"RawImageDescription\")\n",
" .setErrorCol(\"Errors\")\n",
Expand Down Expand Up @@ -105,9 +105,9 @@
{
"cell_type": "markdown",
"source": [
"Before writing the results to a Search Index, you must define a schema which must specify the name, type, and attributes of each field in your index. Refer [Create a basic index in Azure Search](https://docs.microsoft.com/en-us/azure/search/search-what-is-an-index) for more information."
"Before writing the results to a Search Index, you must define a schema which must specify the name, type, and attributes of each field in your index. Refer [Create a basic index in Azure Search](https://docs.microsoft.com/azure/search/search-what-is-an-index) for more information."
],
"metadata": {}
"metadata": {}
},
{
"cell_type": "code",
Expand All @@ -131,9 +131,9 @@
{
"cell_type": "markdown",
"source": [
"The Search Index can be queried using the [Azure Search REST API](https://docs.microsoft.com/rest/api/searchservice/) by sending GET or POST requests and specifying query parameters that give the criteria for selecting matching documents. For more information on querying refer [Query your Azure Search index using the REST API](https://docs.microsoft.com/en-us/rest/api/searchservice/Search-Documents)"
"The Search Index can be queried using the [Azure Search REST API](https://docs.microsoft.com/rest/api/searchservice/) by sending GET or POST requests and specifying query parameters that give the criteria for selecting matching documents. For more information on querying refer [Query your Azure Search index using the REST API](https://docs.microsoft.com/rest/api/searchservice/Search-Documents)"
],
"metadata": {}
"metadata": {}
},
{
"cell_type": "code",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@
"\n",
"# Fill this in with your Azure AI service information\n",
"service_key = find_secret(\n",
" secret_name=\"cognitive-api-key\", keyvault=\"mmlspark-build-keys\"\n",
" secret_name=\"ai-services-api-key\", keyvault=\"mmlspark-build-keys\"\n",
") # Replace this line with a string like service_key = \"dddjnbdkw9329\"\n",
"service_loc = \"eastus\"\n",
"\n",
Expand Down

0 comments on commit d240cbb

Please sign in to comment.