diff --git a/media/azure-sql-db-elastic-pools-custom-dashboard-architecture-1.png b/media/azure-sql-db-elastic-pools-custom-dashboard-architecture-1.png new file mode 100644 index 0000000000..8fa18c341d Binary files /dev/null and b/media/azure-sql-db-elastic-pools-custom-dashboard-architecture-1.png differ diff --git a/media/azure-sql-db-elastic-pools-custom-dashboard-architecture-2.png b/media/azure-sql-db-elastic-pools-custom-dashboard-architecture-2.png new file mode 100644 index 0000000000..0d2bfdd4f9 Binary files /dev/null and b/media/azure-sql-db-elastic-pools-custom-dashboard-architecture-2.png differ diff --git a/media/azure-sql-db-elastic-pools-custom-dashboard-architecture-3.png b/media/azure-sql-db-elastic-pools-custom-dashboard-architecture-3.png new file mode 100644 index 0000000000..c5532f0c42 Binary files /dev/null and b/media/azure-sql-db-elastic-pools-custom-dashboard-architecture-3.png differ diff --git a/media/azure-sql-db-elastic-pools-custom-dashboard-custom-control-1.png b/media/azure-sql-db-elastic-pools-custom-dashboard-custom-control-1.png new file mode 100644 index 0000000000..5c219223e3 Binary files /dev/null and b/media/azure-sql-db-elastic-pools-custom-dashboard-custom-control-1.png differ diff --git a/media/azure-sql-db-elastic-pools-custom-dashboard-pricing.png b/media/azure-sql-db-elastic-pools-custom-dashboard-pricing.png new file mode 100644 index 0000000000..0784a32fb7 Binary files /dev/null and b/media/azure-sql-db-elastic-pools-custom-dashboard-pricing.png differ diff --git a/samples/manage/README.md b/samples/manage/README.md index cdc410e175..4e5d526574 100644 --- a/samples/manage/README.md +++ b/samples/manage/README.md @@ -5,3 +5,13 @@ Contains samples for managing Microsoft's SQL databases including SQL Server, Az Samples are coming soon! +## Collect and monitor resource usage data across multiple pools in a subscription +This Solution Quick Start provides a solution for collecting and monitoring Azure SQL Database resource usage accross multiple pools in a subscription. When you have a large number of databases in a subscription, it is cumbersome to monitor each elastic pool separately. To solve this, you can combine SQL database PowerShell cmdlets and T-SQL queries to collect resource usage data from multiple pools and their databases for monitoring and analysis of resource usage. + +[Manage Mulitiple Elastic Pools in SQL Database Using PowerShell and Power BI](https://github.com/Microsoft/sql-server-samples/tree/master/samples/manage/azure-sql-db-elastic-pools) in the GitHub SQL Server samples repository provides a set of powershell scripts and T-SQL queries along with documentation on what it does and how to use it. + +## Get started using Elastic Pools in a SaaS scenario + +This Solution Quick Start provides a solution for a Softwware-as-a-Solution (SaaS) scenario that leverages Elastic Pools to provide a cost-effective, scalable database back-end of a SaaS application. In this solution, you will walk-though the implementation of a web app that lets you visualize the load created on an Elastic Pool by a load generator using a custom dashboard that supplements the Azure Portal. + +[saas-scenario-with-elastic-pools](https://github.com/Microsoft/sql-server-samples/tree/master/samples/manage/azure-sql-db-elastic-pools-custom-dashboard) in the GitHub SQL Server samples repository provides a load generator and monitoring web app along with the documentation on what it does and how to use it. diff --git a/samples/manage/azure-sql-db-elastic-pools-custom-dashboard/Contoso ShopKeeper.zip b/samples/manage/azure-sql-db-elastic-pools-custom-dashboard/Contoso ShopKeeper.zip new file mode 100644 index 0000000000..3a563f001e Binary files /dev/null and b/samples/manage/azure-sql-db-elastic-pools-custom-dashboard/Contoso ShopKeeper.zip differ diff --git a/samples/manage/azure-sql-db-elastic-pools-custom-dashboard/Solution QuickStart Guide Managing Elastic Pools using Custom Dashboard.docx b/samples/manage/azure-sql-db-elastic-pools-custom-dashboard/Solution QuickStart Guide Managing Elastic Pools using Custom Dashboard.docx new file mode 100644 index 0000000000..43e6169b79 Binary files /dev/null and b/samples/manage/azure-sql-db-elastic-pools-custom-dashboard/Solution QuickStart Guide Managing Elastic Pools using Custom Dashboard.docx differ diff --git a/samples/manage/azure-sql-db-elastic-pools-custom-dashboard/readme.md b/samples/manage/azure-sql-db-elastic-pools-custom-dashboard/readme.md new file mode 100644 index 0000000000..f796133e45 --- /dev/null +++ b/samples/manage/azure-sql-db-elastic-pools-custom-dashboard/readme.md @@ -0,0 +1,160 @@ +# Solution Quick Start: Elastic Pool Custom Dashboard for Saas + +The goal of this Solution Quick Start is to help developers get started using Elastic Pools in a SaaS scenario. Therefore, this quick start focuses on leveraging Elastic Pools to provide a cost-effective, scalable database back-end of a SaaS application, showing how the monitoring of Elastic Pool and constituent databases could be monitored via a custom dashboard that supplements the Azure Portal. + +This readme applies to: + +- Solution Quick Start Guide Managing Elastic Pools using Custom Dashboard.docx - contains the documentation for the Solution QuickStart +- Contoso ShopKeeper.zip - contains the Visual Studio 2105 solutions for the project + +> [AZURE.NOTE] The requirements for building the solution are as follows: +- Visual Studio 2015 Update 1 or later +- Azure Subscription + +## About this sample + +- ***Applies to:*** Azure SQL Database +- *** Key features:*** Elastic Pools +- ***Workload:*** SaaS workload generator +- ***Programming Language:*** ADO.NET, XML, C#, Transact-SQL +- ***Authors:*** Zoiner Tejada, Carl Rabeler, Srini Acharya +- ***Update history:*** n/a + +## Solution Quick Start Overview + +The Solution Quick Start consists of a single Visual Studio 2015 solution with two projects, as follows: + +- LoadGeneratorConsole: A console application that creates a configurable load against a specified set of databases. +- MonitoringWeb: A Web App that shows gathering and reporting on telemetry collected from Elastic Pools and Database instances. + +### Contents + +[Scenario](#scenario)
+[Solution Overview](#solution-overview)
+[Scenario Guidance](#scenario-guidance)
+[Performing Schema Maintenance on Pooled Databases](#schema-maintenance)
+[Monitoring & Alerting](#monitoring-alerting)
+[Database Recovery](#database-recovery)
+[Summary](#summary)
+[Learn More](#learn-more)
+ + + + + + + +## Scenario + +Contoso Shopkeeper provides business small and mid-size an easy to use, cost-effective shopping virtual store front and e-commerce solution that merchants can use to sell their products online. ShopKeeper is a multi-tenant Software-as-a-Service (SaaS) application that is entirely hosted in Azure and managed by Contoso on behalf of their merchant customers. + +The fundamentals behind the architecture of ShopKeeper are resource sharing amongst tenants (which helps keep costs down for both Contoso and its merchant customers), and isolation between tenants (which aims to guarantee that one merchants code or data is never mixed in with another’s). Take the example below, where a customer is using her Web Browser to shop Fabrikam Fabrics. In the process of placing an order she would be interacting with a Web App that only contains Fabrikam Fabric’s code, and the Web App would interact with the database instance that only contains Fabrikam Fabric’s data- this is the isolation aspect. The fact that various Web Apps share the resources from an App Service Plan or that multiple SQL Databases instances share resources from an Elastic Pool demonstrates the resource sharing aspect. + + + +## Solution Overview + +The focus of this Solution Quick Start is on leveraging Elastic Pools and understanding how the support the backend for a SaaS application like Contoso ShopKeeper, the design and implementation of the App Services component is considered out of scope. + +Since the best way to understand the behavior of Elastic Pools is to experience using them under load, we provide a load generator. The load generator is a console application that targets one or more elastic database instances in an Elastic Pool with a specific write load. You can run multiple instances of the load generator with different settings if you want to create a blended load, e.g., a mix of heavy a light load. In addition, you do not need to target all databases in the pool by the load generator, so you can leave databases you choose without any load. + + + +## Scenario Guidance + +Before exploring Elastic Pools “hands-on” by running the load generator and Monitoring Web App in the ShopKeeper scenario, you should review the following guidance. + +### Balance costs vs tenant performance + +In a multi-tenant scenario, appropriately balancing the cost of resources versus tenant performance is critical. An imbalance in one direction means that too much is being spent on resources and costs are high. An imbalance in the other direction means that tenants experience unacceptably slow performance. + +### Freemium Models and Elastic Pools + +In most SaaS application scenarios, like ShopKeeper, there is a notion of a freemium subscription model. In this model, the solution as it is sold to end-customers is priced in different tiers, such as free and paid, with the notion that there is a friction free upgrade path from the free subscription (where most customers start out) to the paid subscription. +In the ShopKeeper scenario, for example, assume Contoso has a Free Subscription and a Paid Subscription. Contoso’s goal for the Free Subscription is to keep the per tenant costs as low as possible because the tenants are not paying for this consumption directly (it might be paid from revenue generated by paying customers, or by other means such as a transaction fees or advertising). The Paid Subscription should still aim for cost-effectiveness, but because tenants are paying for usage in this Subscription, it is likely the per tenant costs can be higher (and provide improved peak performance or storage capacity). + +So how does this map to how Contoso might leverage Elastic Pools? The database cost per tenant is effectively the cost of the pool divided by the number of tenant databases in the pool. The number of databases that can be added to any given pool is limited by the pricing tier of the pool. For example, a Standard 200 pool supports up to 400 databases. Currently this tier is priced at $446 USD per month. If Contoso were to fully utilize the pool by adding 400 databases, the cost per tenant would near $1.12 per tenant per month. Similarly, if they used the Basic 200 tier (which currently is priced at $298 USD per month), then the cost per tenant- month would near $0.75. + + [AZURE.NOTE] Examples of Pool Management: For examples of management with the above options, see https://azure.microsoft.com/en-us/documentation/articles/sql-database-elastic-pool-manage-portal/ + + + +## Performing Schema Maintenance on Pooled Databases + +The Contoso ShopKeeper solution demonstrates an example of the challenge faces by SaaS solutions with regards to managing schemas. In this Solution Quick Start, it deploys an instance of the AdventureWorks database for each tenant. Therefore, Contoso would have many copies of a database with the same schema, albeit different data. This raises the question, how should Contoso roll out schema updates when required (e.g., because of application updates), without affecting the per tenant data? + +Using the Azure Portal, as illustrated in this Solution Quick Start, Contoso can use Elastic Jobs to coordinate the execution of a T-SQL script against all of the databases in an Elastic Pool. The important consideration when taking this approach is that the T-SQL script must be written so it is idempotent. That is, running the script multiple times against a single database does not corrupt the target database or raise errors. The approach shown in this Solution Quick Start is to perform checks that examine if the script has already been run against the target database, and to gracefully complete (instead of executing any changes) if the script has previously completed. + +Besides using the Portal, Contoso may consider using PowerShell to control the execution of Elastics Database Jobs. In addition to the automation opportunities this allows for the deployment of updates, using PowerShell has one characteristic that is not present in the Portal: custom groups. With custom groups Contoso can target its T-SQL script to execute on a specific set of databases, instead of all the databases within the Pool. +NOTE: For examples of using PowerShell to create and manage Elastic Database jobs, see https://azure.microsoft.com/en-us/documentation/articles/sql-database-elastic-jobs-overview/ + + + +## Monitoring & Alerting + +For Contoso, balancing performance versus cost is critical, and for their ShopKeeper application the cost is directly correlated with the number of Elastic Pools they have allocated. Therefore, they want to monitor their Pools closely so they know when to take actions such as creating new Pools or moving databases between Pools. + +This begs the question, how can they setup notifications if their load is overwhelming the pool? In this Solution Quick Start, we demonstrate configuring alerts on the Pool using the Azure Portal. Contoso can configure alerts for metrics such CPU %, eDTU %, sessions %, storage %, workers % which when can be tracked in the portal when they trigger, or they can be used to send an email out to admins. + +If the capacity concern has more to do with specific tenant database instances, Contoso can also configure alerts on a per database level, in a similar fashion as they do for Pools, by using the Azure Portal. + +For both Pool alerts and per Database alerts, Contoso can configure Web Hooks that will perform an HTTP POST to the endpoint of their choosing when the alert is triggered, in addition to having an email sent out to administrators. The payload of this HTTP POST contains the information about the alert that was configured (e.g., the alert name, description and metric configuration), but also the value that caused the alert to trigger. This enables Contoso to extend their ability to react to alerts by using their own management web app or to send out notifications (e.g. using Azure Notification Services) or SMS text messages (e.g., using Twilio). + +When it comes to monitoring the Pool and the databases Contoso can choose to use the Azure Portal as well as T-SQL, as we show in this Solution Quick Start with the Monitoring Web App. They can use the T-SQL options to collect the telemetry from Azure and store it in their own log analytics solution. This would enable them to perform analysis on the telemetry that spans much longer periods of time than that permitted by the retention policy of the data when it is managed by Azure—for example, enabling them to review pool usage over the course of months instead of the 14 days that is maintained by Azure. + + + +## Database Recovery + +By utilizing Elastic Pools, Contoso gets an improved ability to juggle cost versus tenant performance, but does not lose any of the features supporting availability and disaster recovery that are available to SQL Databases outside of a Pool. For example, they can use Point in Time Restore to recover from user error, such as a DBA accidentally dropping the customers table in a tenant’s database. To accomplish this with minimal down-time, they would ensure Pool to which they will restore has capacity for another database instance, rename the original Database to a temporary name and then restore to new database with same name as original. In this fashion, they could restore the database without having to make any application level changes, such as altering connection strings, because the restored database name would be unchanged. + +They also get to benefit from the restore deleted database feature to provide a window of time during which their system could actually delete the database when a merchant cancels, but be able to restore that database should the customer re-join within the retention window. This retention window is controlled by the service tier of the pool: 7 days for Basic, 14 days for Standard and 35 days for Premium. + + + +## Summary + + This Solution Quick Start provides guidance on how to leverage Elastic Pools to support the backend of a SaaS application. In addition, this Solution Quick Start provides a tool to simulate load on elastic databases so that you can understand how the effects of load on a database affect the Elastic Pool. Finally, this solution Quick Start demonstrates how you can build collect your Elastic Pool and database telemetry programmatically so that you can implement a monitoring solution that suites the needs of your application. + + + +## Learn More + +- SQL Database Forum on MSDN: https://social.msdn.microsoft.com/Forums/azure/en-US/home?forum=ssdsgetstarted +- Stack Overflow: http://stackoverflow.com/questions/tagged/azure-sql-database +- Azure Documentation on Elastic Pools: https://azure.microsoft.com/en-us/documentation/articles/sql-database-elastic-pool/ diff --git a/samples/manage/azure-sql-db-elastic-pools/readme.md b/samples/manage/azure-sql-db-elastic-pools/readme.md index 64312a9345..e4fa7a473e 100644 --- a/samples/manage/azure-sql-db-elastic-pools/readme.md +++ b/samples/manage/azure-sql-db-elastic-pools/readme.md @@ -1,11 +1,20 @@ -# Elastic Pool Telemetry using PowerShell +# Solution Quick Start: Elastic Pool Telemetry using PowerShell -This sample provides a set of PowerShell scripts for off-loading elastic pool and elastic database telemetry data into a separate telemetry database. +This Solution Quick Start sample provides a set of PowerShell scripts for off-loading elastic pool and elastic database telemetry data into a separate telemetry database. This readme applies to the PowerShell scripts: PoolTelemetryJobRunner.ps1 and PoolTelemetry.ps1. +## About this sample + +- ***Applies to:*** Azure SQL Database +- *** Key features:*** Elastic Pools +- ***Workload:*** n/a +- ***Programming Language:*** PowerShell, Transact-SQL, DAX +- ***Authors:*** Carl Rabeler, Srini Acharya +- ***Update history:*** n/a + ### Contents [What do the PowerShell scripts do?](#what-do-the-powershell-scripts-do?)