Skip to content

Commit

Permalink
Address PR feedback
Browse files Browse the repository at this point in the history
  • Loading branch information
@gunnaraasen
gunnaraasen committed Aug 8, 2018
1 parent 50854bf commit 08d6e68
Show file tree
Hide file tree
Showing 3 changed files with 103 additions and 111 deletions.
117 changes: 70 additions & 47 deletions plugins/outputs/azure_monitor/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,33 +1,54 @@
## Azure Monitor Custom Metrics Output for Telegraf

This plugin will send custom metrics to Azure Monitor.
Azure Monitor has a metric resolution of one minute.
To handle this in Telegraf, the Azure Monitor output plugin will automatically aggregates metrics into one minute buckets, which are then sent to Azure Monitor on every flush interval.

The metrics from each input plugin will be written to a separate Azure Monitor namespace, prefixed with `Telegraf/` by default.
The field name for each metric is written as the Azure Monitor metric name.
All field values are written as a summarized set that includes: min, max, sum, count.
Tags are written as a dimension on each Azure Monitor metric.

Since Azure Monitor only accepts numeric values, string-typed fields are dropped by default.
There is a configuration option (`strings_as_dimensions`) to retain fields that contain strings as extra dimensions.
Azure Monitor allows a maximum of 10 dimensions per metric so any dimensions over that amount will be deterministically dropped.
This plugin will send custom metrics to Azure Monitor. Azure Monitor has a
metric resolution of one minute. To handle this in Telegraf, the Azure Monitor
output plugin will automatically aggregates metrics into one minute buckets,
which are then sent to Azure Monitor on every flush interval.

The metrics from each input plugin will be written to a separate Azure Monitor
namespace, prefixed with `Telegraf/` by default. The field name for each
metric is written as the Azure Monitor metric name. All field values are
written as a summarized set that includes: min, max, sum, count. Tags are
written as a dimension on each Azure Monitor metric.

Since Azure Monitor only accepts numeric values, string-typed fields are
dropped by default. There is a configuration option (`strings_as_dimensions`)
to retain fields that contain strings as extra dimensions. Azure Monitor
allows a maximum of 10 dimensions per metric so any dimensions over that
amount will be deterministically dropped.

## Initial Setup

1. [Register your Azure subscription with the `microsoft.insights` resource
provider.](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-supported-services#portal)
2. [Consult this chart to identify which regions support Azure Monitor.](https://azure.microsoft.com/en-us/global-infrastructure/services/)
3. Only some Azure Monitor regions support Custom Metrics. For regions with
Custom Metrics support, an endpoint will be available with the format
`https://<region>.monitoring.azure.com`. The following regions are
currently known to be supported:
- West Central US, e.g. `https://westcentralus.monitoring.azure.com`
- South Central US, e.g. `https://southcentralus.monitoring.azure.com`

## Azure Authentication

This plugin uses one of several different types of authenticate methods.
The preferred authentication methods are different from the *order* in which each authentication is checked.
Here are the preferred authentication methods:
This plugin uses one of several different types of authenticate methods. The
preferred authentication methods are different from the *order* in which each
authentication is checked. Here are the preferred authentication methods:

1. Managed Service Identity (MSI) token
- This is the prefered authentication method.
- This is the prefered authentication method. Telegraf will automatically
authenticate using this method when running on Azure VMs.
2. AAD Application Tokens (Service Principals)
- Primarily useful if Telegraf is writing metrics for other resources. [More information](https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-application-objects).
- A Service Principal or User Principal needs to be assigned the `Monitoring Contributor` roles.
- Primarily useful if Telegraf is writing metrics for other resources. [More
information](https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-application-objects).
- A Service Principal or User Principal needs to be assigned the `Monitoring
Contributor` roles.
3. AAD User Tokens (User Principals)
- Allows Telegraf to authenticate like a user. It is best to use this method for development.
- Allows Telegraf to authenticate like a user. It is best to use this method
for development.

The plugin will attempt to authenticate with the first available of the following configurations in this order:
The plugin will attempt to authenticate with the first available of the
following configurations in this order:

1. **Client Credentials**: Azure AD Application ID and Secret.

Expand All @@ -50,8 +71,8 @@ The plugin will attempt to authenticate with the first available of the followin
- `AZURE_CERTIFICATE_PATH`: Specifies the certificate Path to use.
- `AZURE_CERTIFICATE_PASSWORD`: Specifies the certificate password to use.

3. **Resource Owner Password**: Azure AD User and Password. This grant type is *not
recommended*, use device login instead if you need interactive login.
3. **Resource Owner Password**: Azure AD User and Password. This grant type is
*not recommended*, use device login instead if you need interactive login.

- `AZURE_TENANT_ID`: Specifies the Tenant to which to authenticate.
- `AZURE_CLIENT_ID`: Specifies the app client ID to use.
Expand All @@ -64,44 +85,46 @@ The plugin will attempt to authenticate with the first available of the followin
Identity](https://docs.microsoft.com/en-us/azure/active-directory/msi-overview)
for more details. Only available on ARM-based resources.

**Note: As shown above, the last option (#5) is the preferred way to
authenticate when running Telegraf on Azure VMs. The VMs will need to be given
access to the Azure Monitor to publish custom metrics. Instructions on how to
grant access can be found [here]()**
**Note: As shown above, the last option (#4) is the preferred way to
authenticate when running Telegraf on Azure VMs. Make sure you've followed the
[initial setup instructions](#initial-setup).**

## Config

The plugin will automatically attempt to discover the region and resource ID using the Azure VM Instance Metadata service.
If Telegraf is not running on a virtual machine or the VM Instance Metadata service is not available, the following variables are required for the output to function.
The plugin will automatically attempt to discover the region and resource ID
using the Azure VM Instance Metadata service. If Telegraf is not running on a
virtual machine or the VM Instance Metadata service is not available, the
following variables are required for the output to function.

* region
* resourceId
* resource_id

### Configuration:

```
## See the [Azure Monitor output plugin README](/plugins/outputs/azure_monitor/README.md)
## for details on authentication options.
[[outputs.azure_monitor]]
## See the [Azure Monitor output plugin README](/plugins/outputs/azure_monitor/README.md)
## for details on authentication options.
## Write HTTP timeout, formatted as a string. Defaults to 20s.
#timeout = "20s"
## Write HTTP timeout, formatted as a string. Defaults to 20s.
#timeout = "20s"
## Set the namespace prefix, defaults to "Telegraf/<input-name>".
#namespace_prefix = "Telegraf/"
## Set the namespace prefix, defaults to "Telegraf/<input-name>".
#namespace_prefix = "Telegraf/"
## Azure Monitor doesn't have a string value type, so convert string
## fields to dimensions (a.k.a. tags) if enabled. Azure Monitor allows
## a maximum of 10 dimensions so Telegraf will only send the first 10
## alphanumeric dimensions.
#strings_as_dimensions = false
## Azure Monitor doesn't have a string value type, so convert string
## fields to dimensions (a.k.a. tags) if enabled. Azure Monitor allows
## a maximum of 10 dimensions so Telegraf will only send the first 10
## alphanumeric dimensions.
#strings_as_dimensions = false
## *The following two fields must be set or be available via the
## Instance Metadata service on Azure Virtual Machines.*
## *The following two fields must be set or be available via the
## Instance Metadata service on Azure Virtual Machines.*
## Azure Region to publish metrics against, e.g. eastus, southcentralus.
#region = ""
## Azure Region to publish metrics against, e.g. eastus, southcentralus.
#region = ""
## The Azure Resource ID against which metric will be logged, e.g.
## "/subscriptions/<subscription_id>/resourceGroups/<resource_group>/providers/Microsoft.Compute/virtualMachines/<vm_name>"
#resource_id = ""
## The Azure Resource ID against which metric will be logged, e.g.
## "/subscriptions/<subscription_id>/resourceGroups/<resource_group>/providers/Microsoft.Compute/virtualMachines/<vm_name>"
#resource_id = ""
```
61 changes: 33 additions & 28 deletions plugins/outputs/azure_monitor/azure_monitor.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,6 @@ import (
"github.com/influxdata/telegraf/plugins/outputs"
)

var _ telegraf.AggregatingOutput = (*AzureMonitor)(nil)
var _ telegraf.Output = (*AzureMonitor)(nil)

// AzureMonitor allows publishing of metrics to the Azure Monitor custom metrics
// service
type AzureMonitor struct {
Expand Down Expand Up @@ -60,22 +57,22 @@ const (
var sampleConfig = `
## See the [Azure Monitor output plugin README](/plugins/outputs/azure_monitor/README.md)
## for details on authentication options.
## Write HTTP timeout, formatted as a string. Defaults to 20s.
#timeout = "20s"
## Set the namespace prefix, defaults to "Telegraf/<input-name>".
#namespace_prefix = "Telegraf/"
## Azure Monitor doesn't have a string value type, so convert string
## fields to dimensions (a.k.a. tags) if enabled. Azure Monitor allows
## a maximum of 10 dimensions so Telegraf will only send the first 10
## alphanumeric dimensions.
#strings_as_dimensions = false
## *The following two fields must be set or be available via the
## Instance Metadata service on Azure Virtual Machines.*
## Azure Region to publish metrics against, e.g. eastus, southcentralus.
#region = ""
Expand Down Expand Up @@ -142,7 +139,7 @@ func (a *AzureMonitor) Connect() error {
func vmInstanceMetadata(c *http.Client) (string, string, error) {
req, err := http.NewRequest("GET", vmInstanceMetadataURL, nil)
if err != nil {
return "", "", fmt.Errorf("Error creating HTTP request")
return "", "", fmt.Errorf("error creating request: %v", err)
}
req.Header.Set("Metadata", "true")

Expand All @@ -157,7 +154,7 @@ func vmInstanceMetadata(c *http.Client) (string, string, error) {
return "", "", err
}
if resp.StatusCode >= 300 || resp.StatusCode < 200 {
return "", "", fmt.Errorf("unable to fetch MSI: %v", body)
return "", "", fmt.Errorf("unable to fetch instance metadata: [%v] %s", resp.StatusCode, body)
}

// VirtualMachineMetadata contains information about a VM from the metadata service
Expand Down Expand Up @@ -232,7 +229,7 @@ func (a *AzureMonitor) Write(metrics []telegraf.Metric) error {
}

var body []byte
for i, m := range azmetrics {
for _, m := range azmetrics {
// Azure Monitor accepts new batches of points in new-line delimited
// JSON, following RFC 4288 (see https://github.com/ndjson/ndjson-spec).
jsonBytes, err := json.Marshal(&m)
Expand All @@ -241,16 +238,21 @@ func (a *AzureMonitor) Write(metrics []telegraf.Metric) error {
}
// Azure Monitor's maximum request body size of 4MB. Send batches that
// exceed this size via separate write requests.
if (len(body) + len(jsonBytes)) > maxRequestBodySize {
err := a.Write(metrics[i:])
if (len(body) + len(jsonBytes) + 1) > maxRequestBodySize {
err := a.send(body)
if err != nil {
return err
}
body = nil
}
body = append(body, jsonBytes...)
body = append(body, '\n')
}

return a.send(body)
}

func (a *AzureMonitor) send(body []byte) error {
var buf bytes.Buffer
g := gzip.NewWriter(&buf)
if _, err := g.Write(body); err != nil {
Expand All @@ -260,7 +262,6 @@ func (a *AzureMonitor) Write(metrics []telegraf.Metric) error {
return err
}

// req, err := http.NewRequest("POST", a.url, bytes.NewBuffer(body))
req, err := http.NewRequest("POST", a.url, &buf)
if err != nil {
return err
Expand All @@ -273,7 +274,7 @@ func (a *AzureMonitor) Write(metrics []telegraf.Metric) error {
// refresh the token if needed.
req, err = autorest.CreatePreparer(a.auth.WithAuthorization()).Prepare(req)
if err != nil {
return fmt.Errorf("E! [outputs.azure_monitor] Unable to fetch authentication credentials: %v", err)
return fmt.Errorf("unable to fetch authentication credentials: %v", err)
}

resp, err := a.client.Do(req)
Expand All @@ -282,13 +283,9 @@ func (a *AzureMonitor) Write(metrics []telegraf.Metric) error {
}
defer resp.Body.Close()

rbody, err := ioutil.ReadAll(resp.Body)
if err != nil {
rbody = nil
}

if resp.StatusCode < 200 || resp.StatusCode > 299 {
return fmt.Errorf("E! Failed to write: %v", string(rbody))
_, err = ioutil.ReadAll(resp.Body)
if err != nil || resp.StatusCode < 200 || resp.StatusCode > 299 {
return fmt.Errorf("failed to write batch: [%v] %s", resp.StatusCode, resp.Status)
}

return nil
Expand All @@ -315,7 +312,6 @@ func translate(m telegraf.Metric, prefix string) *azureMonitorMetric {
for i, tag := range m.TagList() {
// Azure custom metrics service supports up to 10 dimensions
if i > 10 {
log.Printf("W! [outputs.azure_monitor] metric [%s] exceeds 10 dimensions", m.Name())
continue
}
dimensionNames = append(dimensionNames, tag.Key)
Expand All @@ -326,12 +322,21 @@ func translate(m telegraf.Metric, prefix string) *azureMonitorMetric {
max, _ := m.GetField("max")
sum, _ := m.GetField("sum")
count, _ := m.GetField("count")

mn, ns := "Missing", "Missing"
names := strings.SplitN(m.Name(), "-", 2)
if len(names) > 0 {
ns = names[0]
} else if len(names) > 1 {
mn = names[1]
}

return &azureMonitorMetric{
Time: m.Time(),
Data: &azureMonitorData{
BaseData: &azureMonitorBaseData{
Metric: strings.SplitN(m.Name(), "-", 2)[1],
Namespace: prefix + strings.SplitN(m.Name(), "-", 2)[0],
Metric: mn,
Namespace: ns,
DimensionNames: dimensionNames,
Series: []*azureMonitorSeries{
&azureMonitorSeries{
Expand Down Expand Up @@ -361,9 +366,9 @@ func (a *AzureMonitor) Add(m telegraf.Metric) {
// Azure Monitor doesn't have a string value type, so convert string fields
// to dimensions (a.k.a. tags) if enabled.
if a.StringsAsDimensions {
for fk, fv := range m.Fields() {
if v, ok := fv.(string); ok {
m.AddTag(fk, v)
for _, f := range m.FieldList() {
if v, ok := f.Value.(string); ok {
m.AddTag(f.Key, v)
}
}
}
Expand Down
36 changes: 0 additions & 36 deletions plugins/outputs/azure_monitor/azure_monitor_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,42 +11,6 @@ import (
"github.com/influxdata/telegraf/internal"
)

// func TestConnectionMSI(t *testing.T) {
// azm := AzureMonitor{}
// }

// MockMetrics returns a mock []telegraf.Metric object for using in unit tests
// of telegraf output sinks.
// func getMockMetrics() []telegraf.Metric {
// metrics := make([]telegraf.Metric, 0)
// // Create a new point batch
// metrics = append(metrics, getTestMetric(1.0))
// return metrics
// }

// TestMetric Returns a simple test point:
// measurement -> "test1" or name
// tags -> "tag1":"value1"
// value -> value
// time -> time.Date(2009, time.November, 10, 23, 0, 0, 0, time.UTC)
// func getTestMetric(value interface{}, name ...string) telegraf.Metric {
// if value == nil {
// panic("Cannot use a nil value")
// }
// measurement := "test1"
// if len(name) > 0 {
// measurement = name[0]
// }
// tags := map[string]string{"tag1": "value1"}
// pt, _ := metric.New(
// measurement,
// tags,
// map[string]interface{}{"value": value},
// time.Now().UTC(),
// )
// return pt
// }

func TestAzureMonitor_Connect(t *testing.T) {
type fields struct {
Timeout internal.Duration
Expand Down

0 comments on commit 08d6e68

Please sign in to comment.