Metrics Advisor is a scalable real-time time series monitoring, alerting, and root cause analysis platform. Use Metrics Advisor to:
- Analyze multi-dimensional data from multiple data sources
- Identify and correlate anomalies
- Configure and fine-tune the anomaly detection model used on your data
- Diagnose anomalies and help with root cause analysis
Source code | Package (Pypi) | Package (Conda) | API reference documentation | Product documentation | Samples
Azure SDK Python packages support for Python 2.7 has ended 01 January 2022. For more information and questions, please refer to #20691
Install the Azure Metrics Advisor client library for Python with pip:
pip install azure-ai-metricsadvisor
- Python 3.7 or later is required to use this package.
- You need an Azure subscription, and a Metrics Advisor service to use this package.
You will need two keys to authenticate the client:
- The subscription key to your Metrics Advisor resource. You can find this in the Keys and Endpoint section of your resource in the Azure portal.
- The API key for your Metrics Advisor instance. You can find this in the web portal for Metrics Advisor, in API keys on the left navigation menu.
We can use the keys to create a new MetricsAdvisorClient
or MetricsAdvisorAdministrationClient
.
from azure.ai.metricsadvisor import MetricsAdvisorKeyCredential, MetricsAdvisorClient
service_endpoint = os.getenv("METRICS_ADVISOR_ENDPOINT")
subscription_key = os.getenv("METRICS_ADVISOR_SUBSCRIPTION_KEY")
api_key = os.getenv("METRICS_ADVISOR_API_KEY")
client = MetricsAdvisorClient(service_endpoint,
MetricsAdvisorKeyCredential(subscription_key, api_key))
from azure.ai.metricsadvisor import MetricsAdvisorKeyCredential, MetricsAdvisorAdministrationClient
service_endpoint = os.getenv("METRICS_ADVISOR_ENDPOINT")
subscription_key = os.getenv("METRICS_ADVISOR_SUBSCRIPTION_KEY")
api_key = os.getenv("METRICS_ADVISOR_API_KEY")
client = MetricsAdvisorAdministrationClient(service_endpoint,
MetricsAdvisorKeyCredential(subscription_key, api_key))
MetricsAdvisorClient
helps with:
- listing incidents
- listing root causes of incidents
- retrieving original time series data and time series data enriched by the service.
- listing alerts
- adding feedback to tune your model
MetricsAdvisorAdministrationClient
allows you to
- manage data feeds
- manage anomaly detection configurations
- manage anomaly alerting configurations
- manage hooks
A DataFeed
is what Metrics Advisor ingests from your data source, such as Cosmos DB or a SQL server. A data feed contains rows of:
- timestamps
- zero or more dimensions
- one or more measures
A DataFeedMetric
is a quantifiable measure that is used to monitor and assess the status of a specific business process. It can be a combination of multiple time series values divided into dimensions. For example a web health metric might contain dimensions for user count and the en-us market.
AnomalyDetectionConfiguration
is required for every time series, and determines whether a point in the time series is an anomaly.
After a detection configuration is applied to metrics, AnomalyIncident
s are generated whenever any series within it has an DataPointAnomaly
.
You can configure which anomalies should trigger an AnomalyAlert
. You can set multiple alerts with different settings. For example, you could create an alert for anomalies with lower business impact, and another for more important alerts.
Metrics Advisor lets you create and subscribe to real-time alerts. These alerts are sent over the internet, using a notification hook like EmailNotificationHook
or WebNotificationHook
.
- Add a data feed from a sample or data source
- Check ingestion status
- Configure anomaly detection configuration
- Configure alert configuration
- Query anomaly detection results
- Query incidents
- Query root causes
- Add hooks for receiving anomaly alerts
Metrics Advisor supports connecting different types of data sources. Here is a sample to ingest data from SQL Server.
from azure.ai.metricsadvisor import MetricsAdvisorKeyCredential, MetricsAdvisorAdministrationClient
from azure.ai.metricsadvisor.models import (
SqlServerDataFeedSource,
DataFeedSchema,
DataFeedMetric,
DataFeedDimension,
DataFeedRollupSettings,
DataFeedMissingDataPointFillSettings,
)
service_endpoint = os.getenv("METRICS_ADVISOR_ENDPOINT")
subscription_key = os.getenv("METRICS_ADVISOR_SUBSCRIPTION_KEY")
api_key = os.getenv("METRICS_ADVISOR_API_KEY")
sql_server_connection_string = os.getenv("METRICS_ADVISOR_SQL_SERVER_CONNECTION_STRING")
query = os.getenv("METRICS_ADVISOR_SQL_SERVER_QUERY")
client = MetricsAdvisorAdministrationClient(service_endpoint,
MetricsAdvisorKeyCredential(subscription_key, api_key))
data_feed = client.create_data_feed(
name="My data feed",
source=SqlServerDataFeedSource(
connection_string=sql_server_connection_string,
query=query,
),
granularity="Daily",
schema=DataFeedSchema(
metrics=[
DataFeedMetric(name="cost", display_name="Cost"),
DataFeedMetric(name="revenue", display_name="Revenue")
],
dimensions=[
DataFeedDimension(name="category", display_name="Category"),
DataFeedDimension(name="region", display_name="region")
],
timestamp_column="Timestamp"
),
ingestion_settings=datetime.datetime(2019, 10, 1),
data_feed_description="cost/revenue data feed",
rollup_settings=DataFeedRollupSettings(
rollup_type="AutoRollup",
rollup_method="Sum",
rollup_identification_value="__CUSTOM_SUM__"
),
missing_data_point_fill_settings=DataFeedMissingDataPointFillSettings(
fill_type="SmartFilling"
),
access_mode="Private"
)
return data_feed
After we start the data ingestion, we can check the ingestion status.
import datetime
from azure.ai.metricsadvisor import MetricsAdvisorKeyCredential, MetricsAdvisorAdministrationClient
service_endpoint = os.getenv("METRICS_ADVISOR_ENDPOINT")
subscription_key = os.getenv("METRICS_ADVISOR_SUBSCRIPTION_KEY")
api_key = os.getenv("METRICS_ADVISOR_API_KEY")
data_feed_id = os.getenv("METRICS_ADVISOR_DATA_FEED_ID")
client = MetricsAdvisorAdministrationClient(service_endpoint,
MetricsAdvisorKeyCredential(subscription_key, api_key))
ingestion_status = client.list_data_feed_ingestion_status(
data_feed_id,
datetime.datetime(2020, 9, 20),
datetime.datetime(2020, 9, 25)
)
for status in ingestion_status:
print("Timestamp: {}".format(status.timestamp))
print("Status: {}".format(status.status))
print("Message: {}\n".format(status.message))
While a default detection configuration is automatically applied to each metric, we can tune the detection modes used on our data by creating a customized anomaly detection configuration.
from azure.ai.metricsadvisor import MetricsAdvisorKeyCredential, MetricsAdvisorAdministrationClient
from azure.ai.metricsadvisor.models import (
ChangeThresholdCondition,
HardThresholdCondition,
SmartDetectionCondition,
SuppressCondition,
MetricDetectionCondition,
)
service_endpoint = os.getenv("METRICS_ADVISOR_ENDPOINT")
subscription_key = os.getenv("METRICS_ADVISOR_SUBSCRIPTION_KEY")
api_key = os.getenv("METRICS_ADVISOR_API_KEY")
metric_id = os.getenv("METRICS_ADVISOR_METRIC_ID")
client = MetricsAdvisorAdministrationClient(service_endpoint,
MetricsAdvisorKeyCredential(subscription_key, api_key))
change_threshold_condition = ChangeThresholdCondition(
anomaly_detector_direction="Both",
change_percentage=20,
shift_point=10,
within_range=True,
suppress_condition=SuppressCondition(
min_number=5,
min_ratio=2
)
)
hard_threshold_condition = HardThresholdCondition(
anomaly_detector_direction="Up",
upper_bound=100,
suppress_condition=SuppressCondition(
min_number=2,
min_ratio=2
)
)
smart_detection_condition = SmartDetectionCondition(
anomaly_detector_direction="Up",
sensitivity=10,
suppress_condition=SuppressCondition(
min_number=2,
min_ratio=2
)
)
detection_config = client.create_detection_configuration(
name="my_detection_config",
metric_id=metric_id,
description="anomaly detection config for metric",
whole_series_detection_condition=MetricDetectionCondition(
condition_operator="OR",
change_threshold_condition=change_threshold_condition,
hard_threshold_condition=hard_threshold_condition,
smart_detection_condition=smart_detection_condition
)
)
return detection_config
Then let's configure in which conditions an alert needs to be triggered.
from azure.ai.metricsadvisor import MetricsAdvisorKeyCredential, MetricsAdvisorAdministrationClient
from azure.ai.metricsadvisor.models import (
MetricAlertConfiguration,
MetricAnomalyAlertScope,
TopNGroupScope,
MetricAnomalyAlertConditions,
SeverityCondition,
MetricBoundaryCondition,
MetricAnomalyAlertSnoozeCondition,
)
service_endpoint = os.getenv("METRICS_ADVISOR_ENDPOINT")
subscription_key = os.getenv("METRICS_ADVISOR_SUBSCRIPTION_KEY")
api_key = os.getenv("METRICS_ADVISOR_API_KEY")
detection_configuration_id = os.getenv("METRICS_ADVISOR_DETECTION_CONFIGURATION_ID")
hook_id = os.getenv("METRICS_ADVISOR_HOOK_ID")
client = MetricsAdvisorAdministrationClient(service_endpoint,
MetricsAdvisorKeyCredential(subscription_key, api_key))
alert_config = client.create_alert_configuration(
name="my alert config",
description="alert config description",
cross_metrics_operator="AND",
metric_alert_configurations=[
MetricAlertConfiguration(
detection_configuration_id=detection_configuration_id,
alert_scope=MetricAnomalyAlertScope(
scope_type="WholeSeries"
),
alert_conditions=MetricAnomalyAlertConditions(
severity_condition=SeverityCondition(
min_alert_severity="Low",
max_alert_severity="High"
)
)
),
MetricAlertConfiguration(
detection_configuration_id=detection_configuration_id,
alert_scope=MetricAnomalyAlertScope(
scope_type="TopN",
top_n_group_in_scope=TopNGroupScope(
top=10,
period=5,
min_top_count=5
)
),
alert_conditions=MetricAnomalyAlertConditions(
metric_boundary_condition=MetricBoundaryCondition(
direction="Up",
upper=50
)
),
alert_snooze_condition=MetricAnomalyAlertSnoozeCondition(
auto_snooze=2,
snooze_scope="Metric",
only_for_successive=True
)
),
],
hook_ids=[hook_id]
)
return alert_config
We can query the alerts and anomalies.
import datetime
from azure.ai.metricsadvisor import MetricsAdvisorKeyCredential, MetricsAdvisorClient
service_endpoint = os.getenv("METRICS_ADVISOR_ENDPOINT")
subscription_key = os.getenv("METRICS_ADVISOR_SUBSCRIPTION_KEY")
api_key = os.getenv("METRICS_ADVISOR_API_KEY")
client = MetricsAdvisorClient(service_endpoint,
MetricsAdvisorKeyCredential(subscription_key, api_key))
results = client.list_alerts(
alert_configuration_id=alert_config_id,
start_time=datetime.datetime(2021, 1, 1),
end_time=datetime.datetime(2021, 9, 9),
time_mode="AnomalyTime",
)
tolist = []
for result in results:
tolist.append(result)
print("Alert id: {}".format(result.id))
print("Create time: {}".format(result.created_time))
return tolist
from azure.ai.metricsadvisor import MetricsAdvisorKeyCredential, MetricsAdvisorClient
service_endpoint = os.getenv("METRICS_ADVISOR_ENDPOINT")
subscription_key = os.getenv("METRICS_ADVISOR_SUBSCRIPTION_KEY")
api_key = os.getenv("METRICS_ADVISOR_API_KEY")
client = MetricsAdvisorClient(service_endpoint,
MetricsAdvisorKeyCredential(subscription_key, api_key))
results = client.list_anomalies(
alert_configuration_id=alert_config_id,
alert_id=alert_id,
)
for result in results:
print("Create time: {}".format(result.created_time))
print("Severity: {}".format(result.severity))
print("Status: {}".format(result.status))
We can query the incidents for a detection configuration.
import datetime
from azure.ai.metricsadvisor import MetricsAdvisorKeyCredential, MetricsAdvisorClient
service_endpoint = os.getenv("METRICS_ADVISOR_ENDPOINT")
subscription_key = os.getenv("METRICS_ADVISOR_SUBSCRIPTION_KEY")
api_key = os.getenv("METRICS_ADVISOR_API_KEY")
detection_configuration_id = os.getenv("METRICS_ADVISOR_DETECTION_CONFIGURATION_ID")
client = MetricsAdvisorClient(service_endpoint,
MetricsAdvisorKeyCredential(subscription_key, api_key))
results = client.list_incidents(
detection_configuration_id=detection_configuration_id,
start_time=datetime.datetime(2021, 1, 1),
end_time=datetime.datetime(2021, 9, 9),
)
for result in results:
print("Metric id: {}".format(result.metric_id))
print("Incident ID: {}".format(result.id))
print("Severity: {}".format(result.severity))
print("Status: {}".format(result.status))
We can also query the root causes of an incident
from azure.ai.metricsadvisor import MetricsAdvisorKeyCredential, MetricsAdvisorClient
service_endpoint = os.getenv("METRICS_ADVISOR_ENDPOINT")
subscription_key = os.getenv("METRICS_ADVISOR_SUBSCRIPTION_KEY")
api_key = os.getenv("METRICS_ADVISOR_API_KEY")
detection_configuration_id = os.getenv("METRICS_ADVISOR_DETECTION_CONFIGURATION_ID")
incident_id = os.getenv("METRICS_ADVISOR_INCIDENT_ID")
client = MetricsAdvisorClient(service_endpoint,
MetricsAdvisorKeyCredential(subscription_key, api_key))
results = client.list_incident_root_causes(
detection_configuration_id=detection_configuration_id,
incident_id=incident_id,
)
for result in results:
print("Score: {}".format(result.score))
print("Description: {}".format(result.description))
We can add some hooks so when an alert is triggered, we can get call back.
from azure.ai.metricsadvisor import MetricsAdvisorKeyCredential, MetricsAdvisorAdministrationClient
from azure.ai.metricsadvisor.models import EmailNotificationHook
service_endpoint = os.getenv("METRICS_ADVISOR_ENDPOINT")
subscription_key = os.getenv("METRICS_ADVISOR_SUBSCRIPTION_KEY")
api_key = os.getenv("METRICS_ADVISOR_API_KEY")
client = MetricsAdvisorAdministrationClient(service_endpoint,
MetricsAdvisorKeyCredential(subscription_key, api_key))
hook = client.create_hook(
hook=EmailNotificationHook(
name="email hook",
description="my email hook",
emails_to_alert=["alertme@alertme.com"],
external_link="https://docs.microsoft.com/en-us/azure/cognitive-services/metrics-advisor/how-tos/alerts"
)
)
return hook
This library includes a complete set of async APIs. To use them, you must first install an async transport, such as aiohttp. See azure-core documentation for more information.
from azure.ai.metricsadvisor import MetricsAdvisorKeyCredential
from azure.ai.metricsadvisor.aio import MetricsAdvisorClient
service_endpoint = os.getenv("METRICS_ADVISOR_ENDPOINT")
subscription_key = os.getenv("METRICS_ADVISOR_SUBSCRIPTION_KEY")
api_key = os.getenv("METRICS_ADVISOR_API_KEY")
client = MetricsAdvisorClient(service_endpoint,
MetricsAdvisorKeyCredential(subscription_key, api_key))
from azure.ai.metricsadvisor import MetricsAdvisorKeyCredential
from azure.ai.metricsadvisor.aio import MetricsAdvisorAdministrationClient
service_endpoint = os.getenv("METRICS_ADVISOR_ENDPOINT")
subscription_key = os.getenv("METRICS_ADVISOR_SUBSCRIPTION_KEY")
api_key = os.getenv("METRICS_ADVISOR_API_KEY")
client = MetricsAdvisorAdministrationClient(service_endpoint,
MetricsAdvisorKeyCredential(subscription_key, api_key))
The Azure Metrics Advisor clients will raise exceptions defined in Azure Core.
This library uses the standard logging library for logging.
Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO
level.
Detailed DEBUG
level logging, including request/response bodies and unredacted
headers, can be enabled on the client or per-operation with the logging_enable
keyword argument.
See full SDK logging documentation with examples here.
For more details see the samples README.
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit cla.microsoft.com.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.