diff --git a/docs/sources/_index.md b/docs/sources/_index.md index 2964ac70783a..159fe7ecc238 100755 --- a/docs/sources/_index.md +++ b/docs/sources/_index.md @@ -94,7 +94,7 @@ aliases = ["/docs/grafana/v1.1", "/docs/grafana/latest/guides/reference/admin",
Google Cloud Monitoring
- }}" class="nav-cards__item nav-cards__item--ds"> + }}" class="nav-cards__item nav-cards__item--ds">
AWS CloudWatch
 diff --git a/docs/sources/datasources/_index.md b/docs/sources/datasources/_index.md index 8f3c96fa38b0..1ced957ce320 100644 --- a/docs/sources/datasources/_index.md +++ b/docs/sources/datasources/_index.md @@ -17,7 +17,7 @@ Each data source has a specific Query Editor that is customized for the features The following data sources are officially supported: - [Alertmanager]({{< relref "alertmanager.md" >}}) -- [AWS CloudWatch]({{< relref "cloudwatch.md" >}}) +- [AWS CloudWatch]({{< relref "aws-cloudwatch/_index.md" >}}) - [Azure Monitor]({{< relref "azuremonitor/_index.md" >}}) - [Elasticsearch]({{< relref "elasticsearch.md" >}}) - [Google Cloud Monitoring]({{< relref "google-cloud-monitoring/_index.md" >}}) diff --git a/docs/sources/datasources/cloudwatch.md b/docs/sources/datasources/aws-cloudwatch/_index.md similarity index 76% rename from docs/sources/datasources/cloudwatch.md rename to docs/sources/datasources/aws-cloudwatch/_index.md index 5283220d3429..a036cc025d61 100644 --- a/docs/sources/datasources/cloudwatch.md +++ b/docs/sources/datasources/aws-cloudwatch/_index.md @@ -10,9 +10,9 @@ weight = 200 Grafana ships with built-in support for CloudWatch. Add it as a data source, then you are ready to build dashboards or use Explore with CloudWatch metrics and CloudWatch Logs. -This topic explains options, variables, querying, and other options specific to this data source. Refer to [Add a data source]({{< relref "add-a-data-source.md" >}}) for instructions on how to add a data source to Grafana. Only users with the organization admin role can add data sources. +This topic describes queries, templates, variables, and other configuration specific to the CloudWatch data source. For instructions on how to add a data source to Grafana, refer to [Add a data source]({{< relref "../add-a-data-source.md" >}}). Only users with the organization admin role can add data sources. -> **Note:** If you have issues with getting this data source to work and Grafana is giving you undescriptive errors, then check your log file (try looking in /var/log/grafana/grafana.log). +> **Note:** If you are having issues setting up the data source and Grafana is returning undescriptive errors, then check the log file located in /var/log/grafana/grafana.log). ## Cloudwatch settings @@ -31,21 +31,9 @@ To access data source settings, hover your mouse over the **Configuration** (gea ## Authentication -### AWS credentials +For authentication options and configuration details, see [AWS authentication]({{< relref "aws-authentication.md" >}}) topic. -There are three different authentication methods available. `AWS SDK Default` performs no custom configuration at all and instead uses the [default provider](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html) as specified by the AWS SDK for Go. This requires you to configure your AWS credentials separately, such as if you've [configured the CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html), if you're [running on an EC2 instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html), [in an ECS task](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html) or for a [Service Account in a Kubernetes cluster](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html). - -`Credentials file` corresponds directly to the [SharedCredentialsProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#SharedCredentialsProvider) provider in the Go SDK. In short, it will read the AWS shared credentials file and find the given profile. While `AWS SDK Default` will also find the shared credentials file, this option allows you to specify which profile to use without using environment variables. It doesn't have any implicit fallbacks to other credential providers, and will fail if using credentials from the credentials file doesn't work. - -`Access & secret key` corresponds to the [StaticProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#StaticProvider) and uses the given access key ID and secret key to authenticate. This method doesn't have any fallbacks, and will fail if the provided key pair doesn't work. - -### IAM roles - -Currently all access to CloudWatch is done server side by the Grafana backend using the official AWS SDK. Providing you have chosen the _AWS SDK Default_ authentication method, and your Grafana server is running on AWS, you can use IAM Roles to handle authentication automatically. - -See the AWS documentation on [IAM Roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html) - -### IAM policies +## IAM policies Grafana needs permissions granted via IAM to be able to read CloudWatch metrics and EC2 tags/instances/regions. You can attach these permissions to IAM roles and @@ -99,40 +87,6 @@ Here is a minimal policy example: } ``` -### Assuming a role - -The `Assume Role ARN` field allows you to specify which IAM role to assume, if any. When left blank, the provided credentials are used directly and the associated role or user should have the required permissions. If this field is non-blank, on the other hand, the provided credentials are used to perform an [sts:AssumeRole](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) call. - -### Endpoint - -The `Endpoint` field allows you to specify a custom endpoint URL that overrides the default generated endpoint for the CloudWatch API. Leave this field blank if you want to use the default generated endpoint. For more information on why and how to use Service endpoints, refer to the [AWS service endpoints documentation](https://docs.aws.amazon.com/general/latest/gr/rande.html). - -### EKS IAM roles for service accounts - -The Grafana process in the container runs as user 472 (called "grafana"). When Kubernetes mounts your projected credentials, they will by default only be available to the root user. In order to allow user 472 to access the credentials (and avoid it falling back to the IAM role attached to the EC2 instance), you will need to provide a [security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for your pod. - -```yaml -securityContext: - fsGroup: 472 - runAsUser: 472 - runAsGroup: 472 -``` - -### AWS credentials file - -Create a file at `~/.aws/credentials`. That is the `HOME` path for user running grafana-server. - -> **Note:** If you think you have the credentials file in the right place and it is still not working, you might try moving your .aws file to '/usr/share/grafana/' and make sure your credentials file has at most 0644 permissions. - -Example content: - -```bash -[default] -aws_access_key_id = asdsadasdasdasd -aws_secret_access_key = dasdasdsadasdasdasdsa -region = us-west-2 -``` - ## Using the Query Editor The CloudWatch data source can query data from both CloudWatch metrics and CloudWatch Logs APIs, each with its own specialized query editor. You select which API you want to query with using the query mode switch on top of the editor. @@ -146,22 +100,18 @@ To create a valid query, you need to specify the namespace, metric name and at l ### Dynamic queries using dimension wildcards -> Only available in Grafana v6.5+. - -In Grafana 6.5 or higher, you’re able to monitor a dynamic list of metrics by using the asterisk (\*) wildcard for one or more dimension values. +You can monitor a dynamic list of metrics by using the asterisk (\*) wildcard for one or more dimension values. {{< figure src="/static/img/docs/v65/cloudwatch-dimension-wildcard.png" max-width="800px" class="docs-image--right" caption="CloudWatch dimension wildcard" >}} -In the example, all metrics in the namespace `AWS/EC2` with a metric name of `CPUUtilization` and ANY value for the `InstanceId` dimension are queried. This can help you monitor metrics for AWS resources, like EC2 instances or containers. For example, when new instances get created as part of an auto scaling event, they will automatically appear in the graph without you having to track the new instance IDs. This capability is currently limited to retrieving up to 100 metrics. You can click on `Show Query Preview` to see the search expression that is automatically built to support wildcards. To learn more about search expressions, visit the [CloudWatch documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html). +In this example, the query returns all metrics in the namespace `AWS/EC2` with a metric name of `CPUUtilization` and ANY value for the `InstanceId` dimension are queried. This can help you monitor metrics for AWS resources, like EC2 instances or containers. For example, when new instances are created as part of an auto scaling event, they will automatically appear in the graph without needing to track the new instance IDs. This capability is currently limited to retrieving up to 100 metrics. -By default, the search expression is defined in such a way that the queried metrics must match the defined dimension names exactly. This means that in the example only metrics with exactly one dimension with name ‘InstanceId’ will be returned. +Click on `Show Query Preview` to see the search expression that is automatically built to support wildcards. To learn more about search expressions, visit the [CloudWatch documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html). By default, the search expression is defined in such a way that the queried metrics must match the defined dimension names exactly. This means that in the example only metrics with exactly one dimension with name ‘InstanceId’ will be returned. You can untoggle `Match Exact` to include metrics that have other dimensions defined. Disabling `Match Exact` also creates a search expression even if you don’t use wildcards. We simply search for any metric that matches at least the namespace, metric name, and all defined dimensions. ### Multi-value template variables -> Only available in Grafana v6.5+. - When defining dimension values based on multi-valued template variables, a search expression is used to query for the matching metrics. This enables the use of multiple template variables in one query and also allows you to use template variables for queries that have the `Match Exact` option disabled. Search expressions are currently limited to 1024 characters, so your query may fail if you have a long list of values. We recommend using the asterisk (`*`) wildcard instead of the `All` option if you want to query all metrics that have any value for a certain dimension name. @@ -192,8 +142,6 @@ This feature is not available for metrics that are based on metric math expressi ## Using the Logs Query Editor -> **Note:** Available in Grafana v7.0+. - To query CloudWatch Logs, select the region and up to 20 log groups which you want to query. Use the main input area to write your query in [CloudWatch Logs Query Language](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html) You can also write queries returning time series data by using the [`stats` command](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_Insights-Visualizing-Log-Data.html). When making `stats` queries in Explore, you have to make sure you are in Metrics Explore mode. @@ -206,8 +154,8 @@ To the right of the query input field is a CloudWatch Logs Insights link that op ### Using template variables -As with several other data sources, the CloudWatch data source supports the use of template variables in queries. -See the [Templating]({{< relref "../variables/_index.md" >}}) documentation for an introduction to the templating feature and the different types of template variables. +The CloudWatch data source supports use of template variables in queries. +For an introduction to templating and template variables, refer to the [Templating]({{< relref "../../variables/_index.md" >}}) documentation. ### Deep linking from Grafana panels to the CloudWatch console @@ -218,7 +166,7 @@ If you're not currently logged in to the CloudWatch console, the link will forwa ### Alerting Since CloudWatch Logs queries can return numeric data, for example through the use of the `stats` command, alerts are supported. -See the [Alerting]({{< relref "../alerting/_index.md" >}}) documentation for more on Grafana alerts. +For more information on Grafana alerts, refer to [Alerting]({{< relref "../../alerting/_index.md" >}}) documentation. ## Curated dashboards @@ -236,9 +184,9 @@ To import the pre-configured dashboards, go to the configuration page of your Cl ## Templated queries -Instead of hard-coding things like server, application and sensor name in your metric queries you can use variables in their place. Variables are shown as dropdown select boxes at the top of the dashboard. These dropdowns make it easy to change the data being displayed in your dashboard. +Instead of hard-coding server, application, and sensor names in your metric queries, you can use variables. The variables are listed as dropdown select boxes at the top of the dashboard. These dropdowns make it easy to change the display of data in your dashboard. -See the [Templating]({{< relref "../variables/_index.md" >}}) documentation for an introduction to the templating feature and the different types of template variables. +For an introduction to templating and template variables, refer to the [Templating]({{< relref "../../variables/_index.md" >}}) documentation. ### Query variable @@ -371,7 +319,7 @@ Please see the AWS documentation for [Service Quotas](https://docs.aws.amazon.co ## Configure the data source with grafana.ini -In the Grafana [configuration]({{< relref "../administration/configuration.md#aws" >}}) file, there's an `AWS` section that allows you to customize the data source. +The Grafana [configuration]({{< relref "../../administration/configuration.md#aws" >}}) file includes an `AWS` section where you can customize the data source. ### allowed_auth_providers @@ -387,7 +335,7 @@ When a custom namespace is specified in the query editor, the [List Metrics API] ## Configure the data source with provisioning -It's now possible to configure data sources using config files with Grafana's provisioning system. You can read more about how it works and all the settings you can set for data sources on the [provisioning docs page]({{< relref "../administration/provisioning/#datasources" >}}) +You can configure the CloudWatch data source by customizing configuration files in Grafana's provisioning system. To know more about provisioning and learn about available configuration options, refer to the [Provisioning Grafana]({{< relref "../../administration/provisioning/#datasources" >}}) topic. Here are some provisioning examples for this data source. diff --git a/docs/sources/datasources/aws-cloudwatch/aws-authentication.md b/docs/sources/datasources/aws-cloudwatch/aws-authentication.md new file mode 100644 index 000000000000..194aaeace75c --- /dev/null +++ b/docs/sources/datasources/aws-cloudwatch/aws-authentication.md @@ -0,0 +1,63 @@ ++++ +title = "Authentication" +description = "AWS authentication" +keywords = ["grafana", "aws", "authentication"] +aliases = ["/docs/grafana/latest/datasources/cloudwatch"] +weight = 205 ++++ + +# AWS authentication + +You can use one of the following authentication methods. All of these methods are enabled by default. You can disable them if necessary if you have server configuration access. For more information, refer to [allowed_auth_providers]({{< relref "../../administration/configuration.md#allowed_auth_providers" >}}) documentation. + +- `AWS SDK Default` performs no custom configuration and instead uses the [default provider](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html) as specified by the AWS SDK for Go. It requires you to configure your AWS credentials separately, such as if you've [configured the CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html), if you're [running on an EC2 instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html), [in an ECS task](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html), or for a [Service Account in a Kubernetes cluster](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html). + +- `Credentials file` corresponds directly to the [SharedCredentialsProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#SharedCredentialsProvider) provider in the Go SDK. It reads the AWS shared credentials file to find a given profile. While `AWS SDK Default` will also find the shared credentials file, this option allows you to specify which profile to use without using environment variables. This option doesn't have any implicit fallbacks to other credential providers, and it fails if the credentials provided from the file aren't correct. + +- `Access and secret key` corresponds to the [StaticProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#StaticProvider) and uses the given access key ID and secret key to authenticate. This method doesn't have any fallbacks, and will fail if the provided key pair doesn't work. + +> **Note:** Grafana also supports the `Workspace IAM role` method. We will be add documentation for this authentication method soon. + +## IAM roles + +Currently all access to CloudWatch is done server side by the Grafana backend using the official AWS SDK. If you are using the _AWS SDK Default_ authentication method, and your Grafana server is running on AWS, use IAM Roles to handle authentication automatically. + +For more information, refer to the AWS documentation on [IAM Roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html). + +> **Note:** IAM policies are data source specific. Refer to Authentication section of the [Cloudwatch]({{< relref "./_index.md" >}}) topic for more information. + +## Assuming a role + +The `Assume Role ARN` field allows you to specify which IAM role to assume. When left blank, the provided credentials are used directly and the associated role or user should have the required permissions. If this field is non-blank, on the other hand, the provided credentials are used to perform an [sts:AssumeRole](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) call. + +If you set this field to `false` then AWS authentication no longer uses an assumed role. For more information, refer to [assume_role_enabled]({{< relref "../../administration/configuration.md#assume_role_enabled" >}}) documentation. + +## Endpoint + +The `Endpoint` field allows you to specify a custom endpoint URL that overrides the default generated endpoint for the CloudWatch API. Leave this field blank if you want to use the default generated endpoint. For more information on why and how to use Service endpoints, refer to the [AWS service endpoints documentation](https://docs.aws.amazon.com/general/latest/gr/rande.html). + +## EKS IAM roles for service accounts + +The Grafana process in the container runs as user 472 (called "grafana"). When Kubernetes mounts your projected credentials, they will by default only be available to the root user. To allow user 472 to access the credentials (and avoid falling back to the IAM role attached to the EC2 instance), you need to provide a [security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for your pod. + +```yaml +securityContext: + fsGroup: 472 + runAsUser: 472 + runAsGroup: 472 +``` + +## AWS credentials file + +Create a file at `~/.aws/credentials`. That is the `HOME` path for user running grafana-server. + +> **Note:** If you think you have the credentials file in the right place and it is still not working, you might try moving your .aws file to '/usr/share/grafana/' and make sure your credentials file has at most 0644 permissions. + +Example content: + +```bash +[default] +aws_access_key_id = asdsadasdasdasd +aws_secret_access_key = dasdasdsadasdasdasdsa +region = us-west-2 +``` diff --git a/docs/sources/datasources/elasticsearch.md b/docs/sources/datasources/elasticsearch.md index d701fa5e3629..2365423e6028 100644 --- a/docs/sources/datasources/elasticsearch.md +++ b/docs/sources/datasources/elasticsearch.md @@ -274,6 +274,6 @@ For more details on AWS SigV4, refer to the [AWS documentation](https://docs.aws In order to sign requests to your Amazon Elasticsearch Service domain, SigV4 can be enabled in the Grafana [configuration]({{< relref "../administration/configuration.md#sigv4_auth_enabled" >}}). -Once AWS SigV4 is enabled, it can be configured on the Elasticsearch data source configuration page. Refer to [Cloudwatch authentication]({{}}) for more information about authentication options. +Once AWS SigV4 is enabled, it can be configured on the Elasticsearch data source configuration page. Refer to [Cloudwatch authentication]({{< relref "../datasources/aws-cloudwatch/aws-authentication.md" >}}) for more information about authentication options. {{< figure src="/static/img/docs/v73/elasticsearch-sigv4-config-editor.png" max-width="500px" class="docs-image--no-shadow" caption="SigV4 configuration for AWS Elasticsearch Service" >}} diff --git a/docs/sources/installation/upgrading.md b/docs/sources/installation/upgrading.md index 7c37551472d0..48ee28dbce46 100755 --- a/docs/sources/installation/upgrading.md +++ b/docs/sources/installation/upgrading.md @@ -293,7 +293,7 @@ The new authentication method, _AWS SDK Default_, uses the default AWS Go SDK cr The other authentication methods, _Access & secret key_ and _Credentials file_, have changed in regards to fallbacks. If these methods fail, they no longer fallback to other methods. e.g. environment variables. If you want fallbacks, you should use _AWS SDK Default_ instead. -For more information and details, please refer to [Using AWS CloudWatch in Grafana]({{< relref "../datasources/cloudwatch.md#authentication" >}}). +For more information and details, please refer to [Using AWS CloudWatch in Grafana]({{< relref "../datasources/aws-cloudwatch/aws-authentication.md" >}}). ### User invites database migration diff --git a/docs/sources/whatsnew/whats-new-in-v7-5.md b/docs/sources/whatsnew/whats-new-in-v7-5.md index 979d4597b0ea..13d37b8c29a8 100644 --- a/docs/sources/whatsnew/whats-new-in-v7-5.md +++ b/docs/sources/whatsnew/whats-new-in-v7-5.md @@ -83,7 +83,7 @@ For more information, refer to [Deprecating Application Insights and Insights An - You can now enable or disable authentication providers and assume a role other than default by changing the [allowed_auth_providers]({{< relref "../administration/configuration.md#allowed-auth-providers" >}}) and [assume_role_enabled]({{< relref "../administration/configuration.md#assume-role-enabled" >}}) options in the Grafana configuration file. By default, the allowed authentication providers are _AWS SDK Default_, _Access && secret key_, and _Credentials File_, and role is _Assume role (ARN)_. - You can now specify a custom endpoint in the CloudWatch data source configuration page. This field is optional, and if it is left empty, then the default endpoint for CloudWatch is used. By specifying a regional endpoint, you can reduce request latency. - [AWS Cloudwatch data source]({{< relref "../datasources/cloudwatch.md#endpoint" >}}) was updated as a result of this change. + [AWS Cloudwatch data source]({{< relref "../datasources/aws-cloudwatch/_index.md" >}}) was updated as a result of this change. ### Increased API limit for CloudMonitoring Services