Skip to content

Commit

Permalink
Added array/api_token config file.
Browse files Browse the repository at this point in the history 
Added pod replica link metrics.
  • Loading branch information
@genegr
genegr committed Feb 17, 2023
1 parent 44ac57b commit 07f77ff
Show file tree
Hide file tree
Showing 22 changed files with 1,190 additions and 89 deletions.
6 changes: 3 additions & 3 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ GOCMD=go
GOTEST=$(GOCMD) test
GOVET=$(GOCMD) vet
BINARY_NAME=pure-fa-om-exporter
MODULE_NAME=go mod init purestorage/fa-openmetrics-exporter
MODULE_NAME=purestorage/fa-openmetrics-exporter
VERSION?=1.0.3
SERVICE_PORT?=9490
DOCKER_REGISTRY?= quay.io/purestorage/
Expand All @@ -21,12 +21,12 @@ all: help

## Build:
init:
$(GOCMD) init go mod init $(MODULE_NAME)
$(GOCMD) mod init $(MODULE_NAME)
$(GOCMD) mod tidy

build: ## Build your project and put the output binary in out/bin/
mkdir -p out/bin
GO111MODULE=on $(GOCMD) build -o out/bin/$(BINARY_NAME) cmd/fa-om-exporter/main.go
CGO_ENABLED=0 GO111MODULE=on $(GOCMD) build -o out/bin/$(BINARY_NAME) cmd/fa-om-exporter/main.go

clean: ## Remove build related file
rm -fr ./bin
Expand Down
128 changes: 85 additions & 43 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,46 @@ docker build -t pure-fa-ome:$VERSION .

**Authentication**

Authentication is used by the exporter as the mechanism to cross authenticate to the scraped appliance, therefore for each array it is required to provide the REST API token for an account that has a 'readonly' role. The api-token must be provided in the http request using the HTTP Authorization header of type 'Bearer'. This is achieved by specifying the api-token value as the authorization parameter of the specific job in the Prometheus configuration file.
Authentication is used by the exporter as the mechanism to cross authenticate to the scraped appliance, therefore for each array it is required to provide the REST API token for an account that has a 'readonly' role. The api-token can be provided in two ways

- using the HTTP Authorization header of type 'Bearer', or
- via a configuration map in a specific configuration file.

The first option requires to specify the api-token value as the authorization parameter of the specific job in the Prometheus configuration file.
The second option allows to provide the FlashArray/api-token key-pair map for a list of arrays in a simple YAML configuration file that is passed as paramether to the exporter. This makes possible to write more concise Prometheus configuration files and also to configure other scrapers that cannot use the HTTP authentication header.

### Usage

```shell

usage: pure-fa-om-exporter [-h|--help] [-a|--address "<value>"] [-p|--port
<integer>] [-d|--debug] [-t|--tokens <file>]

Pure Storage FA OpenMetrics exporter

Arguments:

-h --help Print help information
-a --address IP address for this exporter to bind to. Default: 0.0.0.0
-p --port Port for this exporter to listen. Default: 9491
-d --debug Enable debug. Default: false
-t --tokens API token(s) map file
```

The array token configuration file must have to following synthax:

```shell
<array_id1>:
address: <ip-address>|<hosname1>
api_token: <api-token1>
<array_id2>:
address: <ip-address2>|<hostname2>
api_token: <api-token2>
...
<array_idN>:
address: <ip-addressN>|<hostnameN>
api_token: <api-tokenN>
```

### Scraping endpoints

Expand Down Expand Up @@ -116,47 +155,50 @@ A complete example monitoring stack implemented in Docker Compose which can be f

### Metrics Collected

|Metric Name |Description |
|--------------------------------------------|-----------------------------------------------------|
|purefa_info |FlashArray system information |
|purefa_alerts_open |FlashArray open alert events |
|purefa_array_performance_average_bytes |FlashArray array average operations size in bytes |
|purefa_array_performance_bandwidth_bytes |FlashArray array throughput in bytes per second |
|purefa_array_performance_latency_usec |FlashArray array latency in microseconds |
|purefa_array_performance_queue_depth_ops |FlashArray array queue depth size |
|purefa_array_performance_throughput_iops |FlashArray array throughput in iops |
|purefa_array_space_bytes |FlashArray array space in bytes |
|purefa_array_space_data_reduction_ratio |FlashArray array space data reduction |
|purefa_array_space_utilization |FlashArray array space utilization in percent |
|purefa_directory_performance_average_bytes |FlashArray directory average operations size in bytes|
|purefa_directory_performance_bandwidth_bytes|FlashArray directory throughput in bytes per second |
|purefa_directory_performance_latency_usec |FlashArray directory latency in microseconds |
|purefa_directory_performance_throughput_iops|FlashArray directory throughput in iops |
|purefa_directory_space_bytes |FlashArray directory space in bytes |
|purefa_directory_space_data_reduction_ratio |FlashArray directory space data reduction |
|purefa_host_connections_info |FlashArray host volumes connections |
|purefa_host_performance_average_bytes |FlashArray host average operations size in bytes |
|purefa_host_performance_bandwidth_bytes |FlashArray host bandwidth in bytes per second |
|purefa_host_performance_latency_usec |FlashArray host latency in microseconds |
|purefa_host_performance_throughput_iops |FlashArray host throughput in iops |
|purefa_host_space_bytes |FlashArray host space in bytes |
|purefa_host_space_data_reduction_ratio |FlashArray host space data reduction |
|purefa_host_space_size_bytes |FlashArray host volumes size |
|purefa_hw_component_status |FlashArray hardware component status |
|purefa_hw_component_temperature_celsius |FlashArray hardware component temperature in C |
|purefa_hw_component_voltage_volt |FlashArray hardware component voltage |
|purefa_pod_performance_average_bytes |FlashArray pod average operations size |
|purefa_pod_performance_bandwidth_bytes |FlashArray pod throughput in bytes per second |
|purefa_pod_performance_latency_usec |FlashArray pod latency in microseconds |
|purefa_pod_performance_throughput_iops |FlashArray pod throughput in iops |
|purefa_pod_space_bytes |FlashArray pod space in bytes |
|purefa_pod_space_data_reduction_ratio |FlashArray pod space data reduction |
|purefa_volume_performance_average_bytes |FlashArray volume average operations size in bytes |
|purefa_volume_performance_bandwidth_bytes |FlashArray volume throughput in bytes per second |
|purefa_volume_performance_latency_usec |FlashArray volume latency in microseconds |
|purefa_volume_performance_throughput_iops |FlashArray volume throughput in iops |
|purefa_volume_space_bytes |FlashArray volume space in bytes |
|purefa_volume_space_data_reduction_ratio |FlashArray volume space data reduction |
|Metric Name |Description |
|-----------------------------------------------------|------------------------------------------------------------|
|purefa_info |FlashArray system information |
|purefa_alerts_open |FlashArray open alert events |
|purefa_array_performance_average_bytes |FlashArray array average operations size in bytes |
|purefa_array_performance_bandwidth_bytes |FlashArray array throughput in bytes per second |
|purefa_array_performance_latency_usec |FlashArray array latency in microseconds |
|purefa_array_performance_queue_depth_ops |FlashArray array queue depth size |
|purefa_array_performance_throughput_iops |FlashArray array throughput in iops |
|purefa_array_space_bytes |FlashArray array space in bytes |
|purefa_array_space_data_reduction_ratio |FlashArray array space data reduction |
|purefa_array_space_utilization |FlashArray array space utilization in percent |
|purefa_directory_performance_average_bytes |FlashArray directory average operations size in bytes |
|purefa_directory_performance_bandwidth_bytes |FlashArray directory throughput in bytes per second |
|purefa_directory_performance_latency_usec |FlashArray directory latency in microseconds |
|purefa_directory_performance_throughput_iops |FlashArray directory throughput in iops |
|purefa_directory_space_bytes |FlashArray directory space in bytes |
|purefa_directory_space_data_reduction_ratio |FlashArray directory space data reduction |
|purefa_host_connections_info |FlashArray host volumes connections |
|purefa_host_performance_average_bytes |FlashArray host average operations size in bytes |
|purefa_host_performance_bandwidth_bytes |FlashArray host bandwidth in bytes per second |
|purefa_host_performance_latency_usec |FlashArray host latency in microseconds |
|purefa_host_performance_throughput_iops |FlashArray host throughput in iops |
|purefa_host_space_bytes |FlashArray host space in bytes |
|purefa_host_space_data_reduction_ratio |FlashArray host space data reduction |
|purefa_host_space_size_bytes |FlashArray host volumes size |
|purefa_hw_component_status |FlashArray hardware component status |
|purefa_hw_component_temperature_celsius |FlashArray hardware component temperature in C |
|purefa_hw_component_voltage_volt |FlashArray hardware component voltage |
|purefa_pod_performance_average_bytes |FlashArray pod average operations size |
|purefa_pod_performance_bandwidth_bytes |FlashArray pod throughput in bytes per second |
|purefa_pod_performance_latency_usec |FlashArray pod latency in microseconds |
|purefa_pod_performance_throughput_iops |FlashArray pod throughput in iops |
|purefa_pod_space_bytes |FlashArray pod space in bytes |
|purefa_pod_space_data_reduction_ratio |FlashArray pod space data reduction |
|purefa_pod_replica_links_performance_bandwidth_bytes |FlashArray pod replica links throughput in bytes per second |
|purefa_pod_replica_links_lag_avg_sec |FlashArray pod replica links average lag in seconds |
|purefa_pod_replica_links_lag_max_sec |FlashArray pod replica links maximum lag in seconds |
|purefa_volume_performance_average_bytes |FlashArray volume average operations size in bytes |
|purefa_volume_performance_bandwidth_bytes |FlashArray volume throughput in bytes per second |
|purefa_volume_performance_latency_usec |FlashArray volume latency in microseconds |
|purefa_volume_performance_throughput_iops |FlashArray volume throughput in iops |
|purefa_volume_space_bytes |FlashArray volume space in bytes |
|purefa_volume_space_data_reduction_ratio |FlashArray volume space data reduction |

## Monitoring On-Premise with Prometheus and Grafana
Take a holistic overview of your Pure Storage FlashArray estate on-premise with Prometheus and Grafana to summarize statistics such as:
Expand All @@ -174,4 +216,4 @@ For more information on dependencies, and notes to deploy -- take look at the ex

# License

This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details
This project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details.
56 changes: 47 additions & 9 deletions cmd/fa-om-exporter/main.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,58 @@ package main

import (
"context"
"flag"
"fmt"
"log"
"net/http"
config "purestorage/fa-openmetrics-exporter/internal/config"
collectors "purestorage/fa-openmetrics-exporter/internal/openmetrics-exporter"
client "purestorage/fa-openmetrics-exporter/internal/rest-client"
"strings"
"os"
"io"

"gopkg.in/yaml.v3"
"github.com/akamensky/argparse"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
)

var version string = "1.0.3"
var version string = "1.0.4"
var debug bool = false
var arraytokens config.FlashArrayList

func main() {

host := flag.String("host", "0.0.0.0", "Address of the exporter")
port := flag.Int("port", 9490, "Port of the exporter")
d := flag.Bool("debug", false, "Debug")
flag.Parse()
addr := fmt.Sprintf("%s:%d", *host, *port)
parser := argparse.NewParser("pure-fa-om-exporter", "Pure Storage FA OpenMetrics exporter")
host := parser.String("a", "address", &argparse.Options{Required: false, Help: "IP address for this exporter to bind to", Default: "0.0.0.0"})
port := parser.Int("p", "port", &argparse.Options{Required: false, Help: "Port for this exporter to listen", Default: 9491})
d := parser.Flag("d", "debug", &argparse.Options{Required: false, Help: "Enable debug", Default: false})
at := parser.File("t", "tokens", os.O_RDONLY, 0600, &argparse.Options{Required: false, Help: "API token(s) map file"})
parser.Parse(os.Args)
if !isNilFile(*at) {
defer at.Close()
buf := make([]byte, 1024)
arrlist := ""
for {
n, err := at.Read(buf)
if err == io.EOF {
break
}
if err != nil {
log.Fatalf("Reading token file: %v", err)
}
if n > 0 {
arrlist = arrlist + string(buf[:n])
}
}
buf = []byte(arrlist)
err := yaml.Unmarshal(buf, &arraytokens)
if err != nil {
log.Fatalf("Unmarshalling token file: %v", err)
}
}
debug = *d
addr := fmt.Sprintf("%s:%d", *host, *port)
log.Printf("Start Pure FlashArray exporter v%s on %s", version, addr)

http.HandleFunc("/", index)
Expand Down Expand Up @@ -78,11 +107,15 @@ func metricsHandler(w http.ResponseWriter, r *http.Request) {
}
authHeader := r.Header.Get("Authorization")
authFields := strings.Fields(authHeader)
if len(authFields) != 2 || strings.ToLower(authFields[0]) != "bearer" {

apitoken := arraytokens.GetApiToken(endpoint)
if len(authFields) == 2 && strings.ToLower(authFields[0]) == "bearer" {
apitoken = authFields[1]
}
if apitoken == "" {
http.Error(w, "Target authorization token is missing", http.StatusBadRequest)
return
}
apitoken := authFields[1]

registry := prometheus.NewRegistry()
faclient := client.NewRestClient(endpoint, apitoken, apiver, debug)
Expand Down Expand Up @@ -154,3 +187,8 @@ func index(w http.ResponseWriter, r *http.Request) {

fmt.Fprintf(w, "%s", msg)
}

func isNilFile(f os.File) bool {
var tf os.File
return f == tf
}
16 changes: 9 additions & 7 deletions go.mod
View file
Original file line number Diff line number Diff line change
@@ -1,22 +1,24 @@
module purestorage/fa-openmetrics-exporter

go 1.19
go 1.20

require (
github.com/akamensky/argparse v1.4.0
github.com/go-resty/resty/v2 v2.7.0
github.com/google/go-cmp v0.5.9
github.com/prometheus/client_golang v1.14.0
github.com/prometheus/client_model v0.3.0
gopkg.in/yaml.v3 v3.0.1
)

require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/cespare/xxhash/v2 v2.1.2 // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/prometheus/common v0.39.0 // indirect
github.com/prometheus/procfs v0.9.0 // indirect
golang.org/x/net v0.5.0 // indirect
golang.org/x/sys v0.4.0 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect
github.com/prometheus/common v0.37.0 // indirect
github.com/prometheus/procfs v0.8.0 // indirect
golang.org/x/net v0.0.0-20220225172249-27dd8689420f // indirect
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a // indirect
google.golang.org/protobuf v1.28.1 // indirect
)
Loading

0 comments on commit 07f77ff

Please sign in to comment.