Skip to content

Commit

Permalink
Doc: Created a separate topic for AWS authentication (#39012)
Browse files Browse the repository at this point in the history
* Created new topic for AWS authentication and fixed broken relrefs.

* Applied suggestion from review.

* Update docs/sources/datasources/aws-cloudwatch/aws-authentication.md

Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>

* Applied some suggestions from doc review.

* Applied all doc review.

* Last of the edits, went through and addressed all comments.

Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>
  • Loading branch information
achatterjee-grafana and GrafanaWriter committed Sep 13, 2021
1 parent 26bb08e commit d3a7e02
Show file tree
Hide file tree
Showing 7 changed files with 82 additions and 71 deletions.
2 changes: 1 addition & 1 deletion docs/sources/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ aliases = ["/docs/grafana/v1.1", "/docs/grafana/latest/guides/reference/admin",
<img src="/static/img/docs/logos/icon_cloudmonitoring.svg">
<h5>Google Cloud Monitoring</h5>
</a>
<a href="{{< relref "datasources/cloudwatch.md" >}}" class="nav-cards__item nav-cards__item--ds">
<a href="{{< relref "datasources/aws-cloudwatch/_index.md" >}}" class="nav-cards__item nav-cards__item--ds">
<img src="/static/img/docs/logos/icon_cloudwatch.svg">
<h5>AWS CloudWatch</h5>
</a>
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/datasources/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Each data source has a specific Query Editor that is customized for the features
The following data sources are officially supported:

- [Alertmanager]({{< relref "alertmanager.md" >}})
- [AWS CloudWatch]({{< relref "cloudwatch.md" >}})
- [AWS CloudWatch]({{< relref "aws-cloudwatch/_index.md" >}})
- [Azure Monitor]({{< relref "azuremonitor/_index.md" >}})
- [Elasticsearch]({{< relref "elasticsearch.md" >}})
- [Google Cloud Monitoring]({{< relref "google-cloud-monitoring/_index.md" >}})
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ weight = 200

Grafana ships with built-in support for CloudWatch. Add it as a data source, then you are ready to build dashboards or use Explore with CloudWatch metrics and CloudWatch Logs.

This topic explains options, variables, querying, and other options specific to this data source. Refer to [Add a data source]({{< relref "add-a-data-source.md" >}}) for instructions on how to add a data source to Grafana. Only users with the organization admin role can add data sources.
This topic describes queries, templates, variables, and other configuration specific to the CloudWatch data source. For instructions on how to add a data source to Grafana, refer to [Add a data source]({{< relref "../add-a-data-source.md" >}}). Only users with the organization admin role can add data sources.

> **Note:** If you have issues with getting this data source to work and Grafana is giving you undescriptive errors, then check your log file (try looking in /var/log/grafana/grafana.log).
> **Note:** If you are having issues setting up the data source and Grafana is returning undescriptive errors, then check the log file located in /var/log/grafana/grafana.log).
## Cloudwatch settings

Expand All @@ -31,21 +31,9 @@ To access data source settings, hover your mouse over the **Configuration** (gea

## Authentication

### AWS credentials
For authentication options and configuration details, see [AWS authentication]({{< relref "aws-authentication.md" >}}) topic.

There are three different authentication methods available. `AWS SDK Default` performs no custom configuration at all and instead uses the [default provider](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html) as specified by the AWS SDK for Go. This requires you to configure your AWS credentials separately, such as if you've [configured the CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html), if you're [running on an EC2 instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html), [in an ECS task](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html) or for a [Service Account in a Kubernetes cluster](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html).

`Credentials file` corresponds directly to the [SharedCredentialsProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#SharedCredentialsProvider) provider in the Go SDK. In short, it will read the AWS shared credentials file and find the given profile. While `AWS SDK Default` will also find the shared credentials file, this option allows you to specify which profile to use without using environment variables. It doesn't have any implicit fallbacks to other credential providers, and will fail if using credentials from the credentials file doesn't work.

`Access & secret key` corresponds to the [StaticProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#StaticProvider) and uses the given access key ID and secret key to authenticate. This method doesn't have any fallbacks, and will fail if the provided key pair doesn't work.

### IAM roles

Currently all access to CloudWatch is done server side by the Grafana backend using the official AWS SDK. Providing you have chosen the _AWS SDK Default_ authentication method, and your Grafana server is running on AWS, you can use IAM Roles to handle authentication automatically.

See the AWS documentation on [IAM Roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html)

### IAM policies
## IAM policies

Grafana needs permissions granted via IAM to be able to read CloudWatch metrics
and EC2 tags/instances/regions. You can attach these permissions to IAM roles and
Expand Down Expand Up @@ -99,40 +87,6 @@ Here is a minimal policy example:
}
```

### Assuming a role

The `Assume Role ARN` field allows you to specify which IAM role to assume, if any. When left blank, the provided credentials are used directly and the associated role or user should have the required permissions. If this field is non-blank, on the other hand, the provided credentials are used to perform an [sts:AssumeRole](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) call.

### Endpoint

The `Endpoint` field allows you to specify a custom endpoint URL that overrides the default generated endpoint for the CloudWatch API. Leave this field blank if you want to use the default generated endpoint. For more information on why and how to use Service endpoints, refer to the [AWS service endpoints documentation](https://docs.aws.amazon.com/general/latest/gr/rande.html).

### EKS IAM roles for service accounts

The Grafana process in the container runs as user 472 (called "grafana"). When Kubernetes mounts your projected credentials, they will by default only be available to the root user. In order to allow user 472 to access the credentials (and avoid it falling back to the IAM role attached to the EC2 instance), you will need to provide a [security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for your pod.

```yaml
securityContext:
fsGroup: 472
runAsUser: 472
runAsGroup: 472
```

### AWS credentials file

Create a file at `~/.aws/credentials`. That is the `HOME` path for user running grafana-server.

> **Note:** If you think you have the credentials file in the right place and it is still not working, you might try moving your .aws file to '/usr/share/grafana/' and make sure your credentials file has at most 0644 permissions.
Example content:

```bash
[default]
aws_access_key_id = asdsadasdasdasd
aws_secret_access_key = dasdasdsadasdasdasdsa
region = us-west-2
```

## Using the Query Editor

The CloudWatch data source can query data from both CloudWatch metrics and CloudWatch Logs APIs, each with its own specialized query editor. You select which API you want to query with using the query mode switch on top of the editor.
Expand All @@ -146,22 +100,18 @@ To create a valid query, you need to specify the namespace, metric name and at l

### Dynamic queries using dimension wildcards

> Only available in Grafana v6.5+.
In Grafana 6.5 or higher, you’re able to monitor a dynamic list of metrics by using the asterisk (\*) wildcard for one or more dimension values.
You can monitor a dynamic list of metrics by using the asterisk (\*) wildcard for one or more dimension values.

{{< figure src="/static/img/docs/v65/cloudwatch-dimension-wildcard.png" max-width="800px" class="docs-image--right" caption="CloudWatch dimension wildcard" >}}

In the example, all metrics in the namespace `AWS/EC2` with a metric name of `CPUUtilization` and ANY value for the `InstanceId` dimension are queried. This can help you monitor metrics for AWS resources, like EC2 instances or containers. For example, when new instances get created as part of an auto scaling event, they will automatically appear in the graph without you having to track the new instance IDs. This capability is currently limited to retrieving up to 100 metrics. You can click on `Show Query Preview` to see the search expression that is automatically built to support wildcards. To learn more about search expressions, visit the [CloudWatch documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html).
In this example, the query returns all metrics in the namespace `AWS/EC2` with a metric name of `CPUUtilization` and ANY value for the `InstanceId` dimension are queried. This can help you monitor metrics for AWS resources, like EC2 instances or containers. For example, when new instances are created as part of an auto scaling event, they will automatically appear in the graph without needing to track the new instance IDs. This capability is currently limited to retrieving up to 100 metrics.

By default, the search expression is defined in such a way that the queried metrics must match the defined dimension names exactly. This means that in the example only metrics with exactly one dimension with name ‘InstanceId’ will be returned.
Click on `Show Query Preview` to see the search expression that is automatically built to support wildcards. To learn more about search expressions, visit the [CloudWatch documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html). By default, the search expression is defined in such a way that the queried metrics must match the defined dimension names exactly. This means that in the example only metrics with exactly one dimension with name ‘InstanceId’ will be returned.

You can untoggle `Match Exact` to include metrics that have other dimensions defined. Disabling `Match Exact` also creates a search expression even if you don’t use wildcards. We simply search for any metric that matches at least the namespace, metric name, and all defined dimensions.

### Multi-value template variables

> Only available in Grafana v6.5+.
When defining dimension values based on multi-valued template variables, a search expression is used to query for the matching metrics. This enables the use of multiple template variables in one query and also allows you to use template variables for queries that have the `Match Exact` option disabled.

Search expressions are currently limited to 1024 characters, so your query may fail if you have a long list of values. We recommend using the asterisk (`*`) wildcard instead of the `All` option if you want to query all metrics that have any value for a certain dimension name.
Expand Down Expand Up @@ -192,8 +142,6 @@ This feature is not available for metrics that are based on metric math expressi

## Using the Logs Query Editor

> **Note:** Available in Grafana v7.0+.
To query CloudWatch Logs, select the region and up to 20 log groups which you want to query. Use the main input area to write your query in [CloudWatch Logs Query Language](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html)

You can also write queries returning time series data by using the [`stats` command](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_Insights-Visualizing-Log-Data.html). When making `stats` queries in Explore, you have to make sure you are in Metrics Explore mode.
Expand All @@ -206,8 +154,8 @@ To the right of the query input field is a CloudWatch Logs Insights link that op

### Using template variables

As with several other data sources, the CloudWatch data source supports the use of template variables in queries.
See the [Templating]({{< relref "../variables/_index.md" >}}) documentation for an introduction to the templating feature and the different types of template variables.
The CloudWatch data source supports use of template variables in queries.
For an introduction to templating and template variables, refer to the [Templating]({{< relref "../../variables/_index.md" >}}) documentation.

### Deep linking from Grafana panels to the CloudWatch console

Expand All @@ -218,7 +166,7 @@ If you're not currently logged in to the CloudWatch console, the link will forwa
### Alerting

Since CloudWatch Logs queries can return numeric data, for example through the use of the `stats` command, alerts are supported.
See the [Alerting]({{< relref "../alerting/_index.md" >}}) documentation for more on Grafana alerts.
For more information on Grafana alerts, refer to [Alerting]({{< relref "../../alerting/_index.md" >}}) documentation.

## Curated dashboards

Expand All @@ -236,9 +184,9 @@ To import the pre-configured dashboards, go to the configuration page of your Cl

## Templated queries

Instead of hard-coding things like server, application and sensor name in your metric queries you can use variables in their place. Variables are shown as dropdown select boxes at the top of the dashboard. These dropdowns make it easy to change the data being displayed in your dashboard.
Instead of hard-coding server, application, and sensor names in your metric queries, you can use variables. The variables are listed as dropdown select boxes at the top of the dashboard. These dropdowns make it easy to change the display of data in your dashboard.

See the [Templating]({{< relref "../variables/_index.md" >}}) documentation for an introduction to the templating feature and the different types of template variables.
For an introduction to templating and template variables, refer to the [Templating]({{< relref "../../variables/_index.md" >}}) documentation.

### Query variable

Expand Down Expand Up @@ -371,7 +319,7 @@ Please see the AWS documentation for [Service Quotas](https://docs.aws.amazon.co

## Configure the data source with grafana.ini

In the Grafana [configuration]({{< relref "../administration/configuration.md#aws" >}}) file, there's an `AWS` section that allows you to customize the data source.
The Grafana [configuration]({{< relref "../../administration/configuration.md#aws" >}}) file includes an `AWS` section where you can customize the data source.

### allowed_auth_providers

Expand All @@ -387,7 +335,7 @@ When a custom namespace is specified in the query editor, the [List Metrics API]

## Configure the data source with provisioning

It's now possible to configure data sources using config files with Grafana's provisioning system. You can read more about how it works and all the settings you can set for data sources on the [provisioning docs page]({{< relref "../administration/provisioning/#datasources" >}})
You can configure the CloudWatch data source by customizing configuration files in Grafana's provisioning system. To know more about provisioning and learn about available configuration options, refer to the [Provisioning Grafana]({{< relref "../../administration/provisioning/#datasources" >}}) topic.

Here are some provisioning examples for this data source.

Expand Down
63 changes: 63 additions & 0 deletions docs/sources/datasources/aws-cloudwatch/aws-authentication.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
+++
title = "Authentication"
description = "AWS authentication"
keywords = ["grafana", "aws", "authentication"]
aliases = ["/docs/grafana/latest/datasources/cloudwatch"]
weight = 205
+++

# AWS authentication

You can use one of the following authentication methods. All of these methods are enabled by default. You can disable them if necessary if you have server configuration access. For more information, refer to [allowed_auth_providers]({{< relref "../../administration/configuration.md#allowed_auth_providers" >}}) documentation.

- `AWS SDK Default` performs no custom configuration and instead uses the [default provider](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html) as specified by the AWS SDK for Go. It requires you to configure your AWS credentials separately, such as if you've [configured the CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html), if you're [running on an EC2 instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html), [in an ECS task](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html), or for a [Service Account in a Kubernetes cluster](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html).

- `Credentials file` corresponds directly to the [SharedCredentialsProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#SharedCredentialsProvider) provider in the Go SDK. It reads the AWS shared credentials file to find a given profile. While `AWS SDK Default` will also find the shared credentials file, this option allows you to specify which profile to use without using environment variables. This option doesn't have any implicit fallbacks to other credential providers, and it fails if the credentials provided from the file aren't correct.

- `Access and secret key` corresponds to the [StaticProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#StaticProvider) and uses the given access key ID and secret key to authenticate. This method doesn't have any fallbacks, and will fail if the provided key pair doesn't work.

> **Note:** Grafana also supports the `Workspace IAM role` method. We will be add documentation for this authentication method soon.
## IAM roles

Currently all access to CloudWatch is done server side by the Grafana backend using the official AWS SDK. If you are using the _AWS SDK Default_ authentication method, and your Grafana server is running on AWS, use IAM Roles to handle authentication automatically.

For more information, refer to the AWS documentation on [IAM Roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html).

> **Note:** IAM policies are data source specific. Refer to Authentication section of the [Cloudwatch]({{< relref "./_index.md" >}}) topic for more information.
## Assuming a role

The `Assume Role ARN` field allows you to specify which IAM role to assume. When left blank, the provided credentials are used directly and the associated role or user should have the required permissions. If this field is non-blank, on the other hand, the provided credentials are used to perform an [sts:AssumeRole](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) call.

If you set this field to `false` then AWS authentication no longer uses an assumed role. For more information, refer to [assume_role_enabled]({{< relref "../../administration/configuration.md#assume_role_enabled" >}}) documentation.

## Endpoint

The `Endpoint` field allows you to specify a custom endpoint URL that overrides the default generated endpoint for the CloudWatch API. Leave this field blank if you want to use the default generated endpoint. For more information on why and how to use Service endpoints, refer to the [AWS service endpoints documentation](https://docs.aws.amazon.com/general/latest/gr/rande.html).

## EKS IAM roles for service accounts

The Grafana process in the container runs as user 472 (called "grafana"). When Kubernetes mounts your projected credentials, they will by default only be available to the root user. To allow user 472 to access the credentials (and avoid falling back to the IAM role attached to the EC2 instance), you need to provide a [security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for your pod.

```yaml
securityContext:
fsGroup: 472
runAsUser: 472
runAsGroup: 472
```

## AWS credentials file

Create a file at `~/.aws/credentials`. That is the `HOME` path for user running grafana-server.

> **Note:** If you think you have the credentials file in the right place and it is still not working, you might try moving your .aws file to '/usr/share/grafana/' and make sure your credentials file has at most 0644 permissions.
Example content:

```bash
[default]
aws_access_key_id = asdsadasdasdasd
aws_secret_access_key = dasdasdsadasdasdasdsa
region = us-west-2
```

0 comments on commit d3a7e02

Please sign in to comment.