Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Postgres query metrics #335

Merged
merged 9 commits into from
May 20, 2024
Merged

fix: Postgres query metrics #335

merged 9 commits into from
May 20, 2024

Conversation

chris13524
Copy link
Member

@chris13524 chris13524 commented May 20, 2024

Description

Adds metrics for Postgres queries so we can attempt to identify if create_client is slow or not, potentially leading to draining Postgres connection pool.

Depends on #335

How Has This Been Tested?

Not tested

Due Diligence

  • Breaking change
  • Requires a documentation update
  • Requires a e2e/integration test update

@chris13524 chris13524 self-assigned this May 20, 2024
@chris13524 chris13524 changed the base branch from main to fix/upgrade-otel May 20, 2024 16:27
@arein arein added the accepted The issue has been accepted into the project label May 20, 2024
Copy link
Contributor

Show Plan

[command]/home/runner/work/_temp/02f84c69-f9e8-465f-b352-dad2a1c8f4da/terraform-bin -chdir=terraform show -no-color /tmp/plan.tfplan

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # module.ecs.aws_ecs_service.app_service will be updated in-place
  ~ resource "aws_ecs_service" "app_service" {
        id                                 = "arn:aws:ecs:eu-central-1:898587786287:service/staging-push/staging-push-service"
        name                               = "staging-push-service"
        tags                               = {}
      ~ task_definition                    = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push:189" -> (known after apply)
        # (15 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

  # module.ecs.aws_ecs_task_definition.app_task_definition must be replaced
-/+ resource "aws_ecs_task_definition" "app_task_definition" {
      ~ arn                      = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push:189" -> (known after apply)
      ~ arn_without_revision     = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push" -> (known after apply)
      ~ container_definitions    = (sensitive value) # forces replacement
      ~ id                       = "staging-push" -> (known after apply)
      ~ revision                 = 189 -> (known after apply)
      - tags                     = {} -> null
        # (11 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

Plan: 1 to add, 1 to change, 1 to destroy.
::debug::Terraform exited with code 0.
::debug::stdout: %0ATerraform used the selected providers to generate the following execution%0Aplan. Resource actions are indicated with the following symbols:%0A  ~ update in-place%0A-/+ destroy and then create replacement%0A%0ATerraform will perform the following actions:%0A%0A  # module.ecs.aws_ecs_service.app_service will be updated in-place%0A  ~ resource "aws_ecs_service" "app_service" {%0A        id                                 = "arn:aws:ecs:eu-central-1:898587786287:service/staging-push/staging-push-service"%0A        name                               = "staging-push-service"%0A        tags                               = {}%0A      ~ task_definition                    = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push:189" -> (known after apply)%0A        # (15 unchanged attributes hidden)%0A%0A        # (4 unchanged blocks hidden)%0A    }%0A%0A  # module.ecs.aws_ecs_task_definition.app_task_definition must be replaced%0A-/+ resource "aws_ecs_task_definition" "app_task_definition" {%0A      ~ arn                      = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push:189" -> (known after apply)%0A      ~ arn_without_revision     = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push" -> (known after apply)%0A      ~ container_definitions    = (sensitive value) # forces replacement%0A      ~ id                       = "staging-push" -> (known after apply)%0A      ~ revision                 = 189 -> (known after apply)%0A      - tags                     = {} -> null%0A        # (11 unchanged attributes hidden)%0A%0A        # (1 unchanged block hidden)%0A    }%0A%0APlan: 1 to add, 1 to change, 1 to destroy.%0A
::debug::stderr: 
::debug::exitcode: 0

Action: pull_request

Copy link
Contributor

Show Plan

[command]/home/runner/work/_temp/c010f646-4003-47e9-a3e2-0291fef7d351/terraform-bin -chdir=terraform show -no-color /tmp/plan.tfplan

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # module.ecs.aws_ecs_service.app_service will be updated in-place
  ~ resource "aws_ecs_service" "app_service" {
        id                                 = "arn:aws:ecs:eu-central-1:898587786287:service/staging-push/staging-push-service"
        name                               = "staging-push-service"
        tags                               = {}
      ~ task_definition                    = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push:189" -> (known after apply)
        # (15 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

  # module.ecs.aws_ecs_task_definition.app_task_definition must be replaced
-/+ resource "aws_ecs_task_definition" "app_task_definition" {
      ~ arn                      = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push:189" -> (known after apply)
      ~ arn_without_revision     = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push" -> (known after apply)
      ~ container_definitions    = (sensitive value) # forces replacement
      ~ id                       = "staging-push" -> (known after apply)
      ~ revision                 = 189 -> (known after apply)
      - tags                     = {} -> null
        # (11 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.monitoring.grafana_dashboard.at_a_glance will be updated in-place
  ~ resource "grafana_dashboard" "at_a_glance" {
      ~ config_json  = jsonencode(
          ~ {
              + __inputs             = []
              + __requires           = []
              ~ annotations          = {
                  ~ list = [
                      ~ {
                            name       = "Annotations & Alerts"
                          + showIn     = 0
                            # (7 unchanged attributes hidden)
                        },
                    ]
                }
              - fiscalYearStartMonth = 0
              + gnetId               = null
              ~ graphTooltip         = 0 -> 1
              + hideControls         = false
              ~ panels               = [
                  ~ {
                      + collapsed     = false
                      ~ datasource    = {
                          ~ type = "prometheus" -> "datasource"
                          ~ uid  = "LAMKK7wVz" -> "grafana"
                        }
                      - description   = ""
                      - fieldConfig   = {
                          - defaults  = {
                              - color      = {
                                  - mode = "thresholds"
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      ~ gridPos       = {
                          ~ h = 8 -> 1
                          ~ w = 11 -> 24
                            # (2 unchanged attributes hidden)
                        }
                      - options       = {
                          - colorMode     = "value"
                          - graphMode     = "area"
                          - justifyMode   = "auto"
                          - orientation   = "auto"
                          - reduceOptions = {
                              - calcs  = [
                                  - "lastNotNull",
                                ]
                              - fields = ""
                              - values = false
                            }
                          - text          = {}
                          - textMode      = "auto"
                        }
                      - pluginVersion = "8.4.7"
                      ~ targets       = [
                          ~ {
                              ~ datasource   = {
                                  ~ type = "prometheus" -> "datasource"
                                  ~ uid  = "LAMKK7wVz" -> "grafana"
                                }
                              - expr         = "sum(rate(received_notifications{}[1h]))"
                              - legendFormat = "__auto"
                              - refId        = "Received"
                            },
                          - {
                              - datasource   = {
                                  - type = "prometheus"
                                  - uid  = "LAMKK7wVz"
                                }
                              - editorMode   = "code"
                              - expr         = "sum(increase(sent_apns_notifications{}[1h]))"
                              - hide         = false
                              - legendFormat = "__auto"
                              - range        = true
                              - refId        = "SentAPNS"
                            },
                          - {
                              - datasource   = {
                                  - type = "prometheus"
                                  - uid  = "LAMKK7wVz"
                                }
                              - editorMode   = "code"
                              - expr         = "sum(increase(sent_fcm_notifications{}[1h]))"
                              - hide         = false
                              - legendFormat = "__auto"
                              - range        = true
                              - refId        = "SentFCM"
                            },
                        ]
                      ~ title         = "Notifications per Hour" -> "Application"
                      ~ type          = "stat" -> "row"
                    },
                  ~ {
                      + datasource    = {
                          + type = "prometheus"
                          + uid  = "LAMKK7wVz"
                        }
                      + description   = null
                      + fieldConfig   = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + thresholdsStyle   = {
                                      + mode = "off"
                                    }
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                    ]
                                }
                              + unit       = "cps"
                            }
                          + overrides = []
                        }
                      ~ gridPos       = {
                          ~ h = 1 -> 8
                          ~ w = 24 -> 4
                          ~ y = 8 -> 1
                            # (1 unchanged attribute hidden)
                        }
                      + links         = []
                      + options       = {
                          + legend  = {
                              + calcs       = []
                              + displayMode = "list"
                              + placement   = "bottom"
                            }
                          + tooltip = {
                              + mode = "multi"
                              + sort = "none"
                            }
                        }
                      + pluginVersion = "8.4.7"
                      + targets       = [
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + exemplar     = true
                              + expr         = "sum by (aws_ecs_task_revision, name) (rate(postgres_queries_total[$__rate_interval]))"
                              + legendFormat = "{{name}} r{{aws_ecs_task_revision}}"
                              + refId        = "PostgresQueryRate"
                            },
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + exemplar     = true
                              + expr         = "sum(rate(postgres_queries_total[$__rate_interval]))"
                              + legendFormat = "r{{aws_ecs_task_revision}}"
                              + refId        = "PostgresQueryRateTotal"
                            },
                        ]
                      ~ title         = "Graphs" -> "Postgres Query Rate"
                      ~ type          = "row" -> "timeseries"
                    },
                  ~ {
                      + description   = null
                      ~ fieldConfig   = {
                          ~ defaults  = {
                              ~ custom     = {
                                  - axisLabel         = "Notifications"
                                  - hideFrom          = {
                                      - legend  = false
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                    # (11 unchanged attributes hidden)
                                }
                              ~ unit       = "none" -> "ms"
                                # (3 unchanged attributes hidden)
                            }
                          ~ overrides = [
                              - {
                                  - __systemRef = "hideSeriesFrom"
                                  - matcher     = {
                                      - id      = "byNames"
                                      - options = {
                                          - mode     = "exclude"
                                          - names    = [
                                              - "sum(received_notifications{})",
                                            ]
                                          - prefix   = "All except:"
                                          - readOnly = true
                                        }
                                    }
                                  - properties  = [
                                      - {
                                          - id    = "custom.hideFrom"
                                          - value = {
                                              - legend  = false
                                              - tooltip = false
                                              - viz     = true
                                            }
                                        },
                                    ]
                                },
                            ]
                        }
                      ~ gridPos       = {
                          ~ w = 11 -> 4
                          ~ x = 0 -> 4
                          ~ y = 9 -> 1
                            # (1 unchanged attribute hidden)
                        }
                      + links         = []
                      ~ options       = {
                          ~ tooltip = {
                              ~ mode = "single" -> "multi"
                                # (1 unchanged attribute hidden)
                            }
                            # (1 unchanged attribute hidden)
                        }
                      + pluginVersion = "8.4.7"
                      ~ targets       = [
                          ~ {
                              ~ exemplar     = true -> false
                              ~ expr         = "sum(increase(received_notifications{}[1h]))" -> "sum by (aws_ecs_task_revision, name) (rate(postgres_query_latency_sum[$__rate_interval])) / sum by (aws_ecs_task_revision, name) (rate(postgres_query_latency_count[$__rate_interval]))"
                              - format       = "time_series"
                              - interval     = ""
                              ~ legendFormat = "" -> "{{name}} r{{aws_ecs_task_revision}}"
                              ~ refId        = "Notifications" -> "PostgresQueryLatency"
                                # (1 unchanged attribute hidden)
                            },
                        ]
                      ~ title         = "Received Notifications" -> "Postgres Query Latency"
                        # (2 unchanged attributes hidden)
                    },
                  - {
                      - datasource  = {
                          - type = "prometheus"
                          - uid  = "LAMKK7wVz"
                        }
                      - fieldConfig = {
                          - defaults  = {
                              - color      = {
                                  - mode = "palette-classic"
                                }
                              - custom     = {
                                  - axisLabel         = "Clients"
                                  - axisPlacement     = "auto"
                                  - barAlignment      = 0
                                  - drawStyle         = "line"
                                  - fillOpacity       = 0
                                  - gradientMode      = "none"
                                  - hideFrom          = {
                                      - legend  = false
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - lineWidth         = 1
                                  - pointSize         = 5
                                  - scaleDistribution = {
                                      - type = "linear"
                                    }
                                  - showPoints        = "auto"
                                  - spanNulls         = false
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                  - thresholdsStyle   = {
                                      - mode = "off"
                                    }
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                      - {
                                          - color = "red"
                                          - value = 80
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      - gridPos     = {
                          - h = 8
                          - w = 10
                          - x = 11
                          - y = 9
                        }
                      - options     = {
                          - legend  = {
                              - calcs       = []
                              - displayMode = "list"
                              - placement   = "bottom"
                            }
                          - tooltip = {
                              - mode = "single"
                              - sort = "none"
                            }
                        }
                      - targets     = [
                          - {
                              - datasource   = {
                                  - type = "prometheus"
                                  - uid  = "LAMKK7wVz"
                                }
                              - exemplar     = true
                              - expr         = "sum(rate(registered_clients{}[$__rate_interval]))"
                              - interval     = ""
                              - legendFormat = ""
                              - refId        = "Clients"
                            },
                        ]
                      - title       = "Client registration rate"
                      - type        = "timeseries"
                    },
                  - {
                      - collapsed = false
                      - gridPos   = {
                          - h = 1
                          - w = 24
                          - x = 0
                          - y = 17
                        }
                      - panels    = []
                      - title     = "AWS Load Balancer"
                      - type      = "row"
                    },
                  - {
                      - datasource  = {
                          - type = "cloudwatch"
                          - uid  = "XnMFKnQVk"
                        }
                      - fieldConfig = {
                          - defaults  = {
                              - color      = {
                                  - mode = "palette-classic"
                                }
                              - custom     = {
                                  - axisLabel         = ""
                                  - axisPlacement     = "auto"
                                  - barAlignment      = 0
                                  - drawStyle         = "line"
                                  - fillOpacity       = 0
                                  - gradientMode      = "none"
                                  - hideFrom          = {
                                      - legend  = false
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - lineWidth         = 1
                                  - pointSize         = 5
                                  - scaleDistribution = {
                                      - type = "linear"
                                    }
                                  - showPoints        = "auto"
                                  - spanNulls         = false
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                  - thresholdsStyle   = {
                                      - mode = "off"
                                    }
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                      - {
                                          - color = "red"
                                          - value = 80
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      - gridPos     = {
                          - h = 9
                          - w = 7
                          - x = 0
                          - y = 18
                        }
                      - options     = {
                          - legend  = {
                              - calcs       = []
                              - displayMode = "list"
                              - placement   = "bottom"
                            }
                          - tooltip = {
                              - mode = "single"
                              - sort = "none"
                            }
                        }
                      - targets     = [
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "RequestCount"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "A"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                        ]
                      - title       = "Requests"
                      - type        = "timeseries"
                    },
                  - {
                      - alert       = {
                          - alertRuleTags       = {}
                          - conditions          = [
                              - {
                                  - evaluator = {
                                      - params = [
                                          - 15,
                                        ]
                                      - type   = "gt"
                                    }
                                  - operator  = {
                                      - type = "and"
                                    }
                                  - query     = {
                                      - params = [
                                          - "A",
                                          - "5m",
                                          - "now",
                                        ]
                                    }
                                  - reducer   = {
                                      - params = []
                                      - type   = "sum"
                                    }
                                  - type      = "query"
                                },
                              - {
                                  - evaluator = {
                                      - params = [
                                          - 15,
                                        ]
                                      - type   = "gt"
                                    }
                                  - operator  = {
                                      - type = "or"
                                    }
                                  - query     = {
                                      - params = [
                                          - "B",
                                          - "5m",
                                          - "now",
                                        ]
                                    }
                                  - reducer   = {
                                      - params = []
                                      - type   = "sum"
                                    }
                                  - type      = "query"
                                },
                            ]
                          - executionErrorState = "alerting"
                          - for                 = ""
                          - frequency           = "1m"
                          - handler             = 1
                          - message             = "Echo server - Prod - 5XX error"
                          - name                = "staging Echo Server 5XX alert"
                          - noDataState         = "no_data"
                          - notifications       = []
                        }
                      - datasource  = {
                          - type = "cloudwatch"
                          - uid  = "XnMFKnQVk"
                        }
                      - fieldConfig = {
                          - defaults  = {
                              - color      = {
                                  - mode = "palette-classic"
                                }
                              - custom     = {
                                  - axisLabel         = ""
                                  - axisPlacement     = "auto"
                                  - barAlignment      = 0
                                  - drawStyle         = "line"
                                  - fillOpacity       = 0
                                  - gradientMode      = "none"
                                  - hideFrom          = {
                                      - legend  = false
                                      - mode    = "dashed"
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - lineWidth         = 1
                                  - pointSize         = 5
                                  - scaleDistribution = {
                                      - type = "linear"
                                    }
                                  - showPoints        = "auto"
                                  - spanNulls         = false
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                  - thresholdsStyle   = {
                                      - mode = "off"
                                    }
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                      - {
                                          - color = "red"
                                          - value = 80
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      - gridPos     = {
                          - h = 9
                          - w = 7
                          - x = 7
                          - y = 18
                        }
                      - options     = {
                          - legend  = {
                              - calcs       = []
                              - displayMode = "list"
                              - placement   = "bottom"
                            }
                          - tooltip = {
                              - mode = "single"
                              - sort = "none"
                            }
                        }
                      - targets     = [
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "HTTPCode_ELB_5XX_Count"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "A"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "HTTPCode_Target_5XX_Count"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "B"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                        ]
                      - thresholds  = [
                          - {
                              - colorMode = "critical"
                              - op        = "gt"
                              - value     = 1
                              - visible   = true
                            },
                        ]
                      - title       = "5XX"
                      - type        = "timeseries"
                    },
                  - {
                      - datasource  = {
                          - type = "cloudwatch"
                          - uid  = "XnMFKnQVk"
                        }
                      - fieldConfig = {
                          - defaults  = {
                              - color      = {
                                  - mode = "palette-classic"
                                }
                              - custom     = {
                                  - axisLabel         = ""
                                  - axisPlacement     = "auto"
                                  - barAlignment      = 0
                                  - drawStyle         = "line"
                                  - fillOpacity       = 0
                                  - gradientMode      = "none"
                                  - hideFrom          = {
                                      - legend  = false
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - lineWidth         = 1
                                  - pointSize         = 5
                                  - scaleDistribution = {
                                      - type = "linear"
                                    }
                                  - showPoints        = "auto"
                                  - spanNulls         = false
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                  - thresholdsStyle   = {
                                      - mode = "off"
                                    }
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                      - {
                                          - color = "red"
                                          - value = 80
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      - gridPos     = {
                          - h = 9
                          - w = 7
                          - x = 14
                          - y = 18
                        }
                      - options     = {
                          - legend  = {
                              - calcs       = []
                              - displayMode = "list"
                              - placement   = "bottom"
                            }
                          - tooltip = {
                              - mode = "single"
                              - sort = "none"
                            }
                        }
                      - targets     = [
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "HTTPCode_ELB_4XX_Count"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "A"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "HTTPCode_Target_4XX_Count"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "B"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                        ]
                      - title       = "4XX"
                      - type        = "timeseries"
                    },
                ]
              + refresh              = ""
              + rows                 = []
                tags                 = []
              ~ timepicker           = {
                  + refresh_intervals = [
                      + "5s",
                      + "10s",
                      + "30s",
                      + "1m",
                      + "5m",
                      + "15m",
                      + "30m",
                      + "1h",
                      + "2h",
                      + "1d",
                    ]
                  + time_options      = [
                      + "5m",
                      + "15m",
                      + "1h",
                      + "6h",
                      + "12h",
                      + "24h",
                      + "2d",
                      + "7d",
                      + "30d",
                    ]
                }
              ~ timezone             = "" -> "utc"
              ~ title                = "staging-push" -> "Push Server - Staging"
              ~ uid                  = "staging-push" -> "push-staging"
              - weekStart            = ""
                # (7 unchanged attributes hidden)
            }
        )
        id           = "0:staging-push"
        # (8 unchanged attributes hidden)
    }

  # module.monitoring.grafana_dashboard.at_a_glance_old will be created
  + resource "grafana_dashboard" "at_a_glance_old" {
      + config_json  = jsonencode(
            {
              + annotations          = {
                  + list = [
                      + {
                          + builtIn    = 1
                          + datasource = "-- Grafana --"
                          + enable     = true
                          + hide       = true
                          + iconColor  = "rgba(0, 211, 255, 1)"
                          + name       = "Annotations & Alerts"
                          + target     = {
                              + limit    = 100
                              + matchAny = false
                              + tags     = []
                              + type     = "dashboard"
                            }
                          + type       = "dashboard"
                        },
                    ]
                }
              + editable             = true
              + fiscalYearStartMonth = 0
              + graphTooltip         = 0
              + links                = []
              + liveNow              = false
              + panels               = [
                  + {
                      + datasource    = {
                          + type = "prometheus"
                          + uid  = "LAMKK7wVz"
                        }
                      + description   = ""
                      + fieldConfig   = {
                          + defaults  = {
                              + color      = {
                                  + mode = "thresholds"
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                    ]
                                }
                            }
                          + overrides = []
                        }
                      + gridPos       = {
                          + h = 8
                          + w = 11
                          + x = 0
                          + y = 0
                        }
                      + options       = {
                          + colorMode     = "value"
                          + graphMode     = "area"
                          + justifyMode   = "auto"
                          + orientation   = "auto"
                          + reduceOptions = {
                              + calcs  = [
                                  + "lastNotNull",
                                ]
                              + fields = ""
                              + values = false
                            }
                          + text          = {}
                          + textMode      = "auto"
                        }
                      + pluginVersion = "8.4.7"
                      + targets       = [
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + expr         = "sum(rate(received_notifications{}[1h]))"
                              + legendFormat = "__auto"
                              + refId        = "Received"
                            },
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + editorMode   = "code"
                              + expr         = "sum(increase(sent_apns_notifications{}[1h]))"
                              + hide         = false
                              + legendFormat = "__auto"
                              + range        = true
                              + refId        = "SentAPNS"
                            },
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + editorMode   = "code"
                              + expr         = "sum(increase(sent_fcm_notifications{}[1h]))"
                              + hide         = false
                              + legendFormat = "__auto"
                              + range        = true
                              + refId        = "SentFCM"
                            },
                        ]
                      + title         = "Notifications per Hour"
                      + type          = "stat"
                    },
                  + {
                      + gridPos = {
                          + h = 1
                          + w = 24
                          + x = 0
                          + y = 8
                        }
                      + title   = "Graphs"
                      + type    = "row"
                    },
                  + {
                      + datasource  = {
                          + type = "prometheus"
                          + uid  = "LAMKK7wVz"
                        }
                      + fieldConfig = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisLabel         = "Notifications"
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + hideFrom          = {
                                      + legend  = false
                                      + tooltip = false
                                      + viz     = false
                                    }
                                  + lineInterpolation = "linear"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + stacking          = {
                                      + group = "A"
                                      + mode  = "none"
                                    }
                                  + thresholdsStyle   = {
                                      + mode = "off"
                                    }
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                    ]
                                }
                              + unit       = "none"
                            }
                          + overrides = [
                              + {
                                  + __systemRef = "hideSeriesFrom"
                                  + matcher     = {
                                      + id      = "byNames"
                                      + options = {
                                          + mode     = "exclude"
                                          + names    = [
                                              + "sum(received_notifications{})",
                                            ]
                                          + prefix   = "All except:"
                                          + readOnly = true
                                        }
                                    }
                                  + properties  = [
                                      + {
                                          + id    = "custom.hideFrom"
                                          + value = {
                                              + legend  = false
                                              + tooltip = false
                                              + viz     = true
                                            }
                                        },
                                    ]
                                },
                            ]
                        }
                      + gridPos     = {
                          + h = 8
                          + w = 11
                          + x = 0
                          + y = 9
                        }
                      + options     = {
                          + legend  = {
                              + calcs       = []
                              + displayMode = "list"
                              + placement   = "bottom"
                            }
                          + tooltip = {
                              + mode = "single"
                              + sort = "none"
                            }
                        }
                      + targets     = [
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + exemplar     = true
                              + expr         = "sum(increase(received_notifications{}[1h]))"
                              + format       = "time_series"
                              + interval     = ""
                              + legendFormat = ""
                              + refId        = "Notifications"
                            },
                        ]
                      + title       = "Received Notifications"
                      + type        = "timeseries"
                    },
                  + {
                      + datasource  = {
                          + type = "prometheus"
                          + uid  = "LAMKK7wVz"
                        }
                      + fieldConfig = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisLabel         = "Clients"
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + hideFrom          = {
                                      + legend  = false
                                      + tooltip = false
                                      + viz     = false
                                    }
                                  + lineInterpolation = "linear"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + stacking          = {
                                      + group = "A"
                                      + mode  = "none"
                                    }
                                  + thresholdsStyle   = {
                                      + mode = "off"
                                    }
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                      + {
                                          + color = "red"
                                          + value = 80
                                        },
                                    ]
                                }
                            }
                          + overrides = []
                        }
                      + gridPos     = {
                          + h = 8
                          + w = 10
                          + x = 11
                          + y = 9
                        }
                      + options     = {
                          + legend  = {
                              + calcs       = []
                              + displayMode = "list"
                              + placement   = "bottom"
                            }
                          + tooltip = {
                              + mode = "single"
                              + sort = "none"
                            }
                        }
                      + targets     = [
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + exemplar     = true
                              + expr         = "sum(rate(registered_clients{}[$__rate_interval]))"
                              + interval     = ""
                              + legendFormat = ""
                              + refId        = "Clients"
                            },
                        ]
                      + title       = "Client registration rate"
                      + type        = "timeseries"
                    },
                  + {
                      + collapsed = false
                      + gridPos   = {
                          + h = 1
                          + w = 24
                          + x = 0
                          + y = 17
                        }
                      + panels    = []
                      + title     = "AWS Load Balancer"
                      + type      = "row"
                    },
                  + {
                      + datasource  = {
                          + type = "cloudwatch"
                          + uid  = "XnMFKnQVk"
                        }
                      + fieldConfig = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisLabel         = ""
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + hideFrom          = {
                                      + legend  = false
                                      + tooltip = false
                                      + viz     = false
                                    }
                                  + lineInterpolation = "linear"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + stacking          = {
                                      + group = "A"
                                      + mode  = "none"
                                    }
                                  + thresholdsStyle   = {
                                      + mode = "off"
                                    }
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                      + {
                                          + color = "red"
                                          + value = 80
                                        },
                                    ]
                                }
                            }
                          + overrides = []
                        }
                      + gridPos     = {
                          + h = 9
                          + w = 7
                          + x = 0
                          + y = 18
                        }
                      + options     = {
                          + legend  = {
                              + calcs       = []
                              + displayMode = "list"
                              + placement   = "bottom"
                            }
                          + tooltip = {
                              + mode = "single"
                              + sort = "none"
                            }
                        }
                      + targets     = [
                          + {
                              + alias            = ""
                              + datasource       = {
                                  + type = "cloudwatch"
                                  + uid  = "XnMFKnQVk"
                                }
                              + dimensions       = {
                                  + LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              + expression       = ""
                              + id               = ""
                              + matchExact       = true
                              + metricEditorMode = 0
                              + metricName       = "RequestCount"
                              + metricQueryType  = 0
                              + namespace        = "AWS/ApplicationELB"
                              + period           = ""
                              + queryMode        = "Metrics"
                              + refId            = "A"
                              + region           = "default"
                              + sqlExpression    = ""
                              + statistic        = "Sum"
                            },
                        ]
                      + title       = "Requests"
                      + type        = "timeseries"
                    },
                  + {
                      + alert       = {
                          + alertRuleTags       = {}
                          + conditions          = [
                              + {
                                  + evaluator = {
                                      + params = [
                                          + 15,
                                        ]
                                      + type   = "gt"
                                    }
                                  + operator  = {
                                      + type = "and"
                                    }
                                  + query     = {
                                      + params = [
                                          + "A",
                                          + "5m",
                                          + "now",
                                        ]
                                    }
                                  + reducer   = {
                                      + params = []
                                      + type   = "sum"
                                    }
                                  + type      = "query"
                                },
                              + {
                                  + evaluator = {
                                      + params = [
                                          + 15,
                                        ]
                                      + type   = "gt"
                                    }
                                  + operator  = {
                                      + type = "or"
                                    }
                                  + query     = {
                                      + params = [
                                          + "B",
                                          + "5m",
                                          + "now",
                                        ]
                                    }
                                  + reducer   = {
                                      + params = []
                                      + type   = "sum"
                                    }
                                  + type      = "query"
                                },
                            ]
                          + executionErrorState = "alerting"
                          + for                 = ""
                          + frequency           = "1m"
                          + handler             = 1
                          + message             = "Echo server - Prod - 5XX error"
                          + name                = "staging Echo Server 5XX alert"
                          + noDataState         = "no_data"
                          + notifications       = []
                        }
                      + datasource  = {
                          + type = "cloudwatch"
                          + uid  = "XnMFKnQVk"
                        }
                      + fieldConfig = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisLabel         = ""
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + hideFrom          = {
                                      + legend  = false
                                      + mode    = "dashed"
                                      + tooltip = false
                                      + viz     = false
                                    }
                                  + lineInterpolation = "linear"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + stacking          
...

Action: pull_request

Base automatically changed from fix/upgrade-otel to main May 20, 2024 17:58
Copy link
Contributor

Show Plan

[command]/home/runner/work/_temp/27cab3a1-dff5-4583-851b-46be5a541edd/terraform-bin -chdir=terraform show -no-color /tmp/plan.tfplan

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # module.ecs.aws_ecs_service.app_service will be updated in-place
  ~ resource "aws_ecs_service" "app_service" {
        id                                 = "arn:aws:ecs:eu-central-1:898587786287:service/staging-push/staging-push-service"
        name                               = "staging-push-service"
        tags                               = {}
      ~ task_definition                    = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push:189" -> (known after apply)
        # (15 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

  # module.ecs.aws_ecs_task_definition.app_task_definition must be replaced
-/+ resource "aws_ecs_task_definition" "app_task_definition" {
      ~ arn                      = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push:189" -> (known after apply)
      ~ arn_without_revision     = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push" -> (known after apply)
      ~ container_definitions    = (sensitive value) # forces replacement
      ~ id                       = "staging-push" -> (known after apply)
      ~ revision                 = 189 -> (known after apply)
      - tags                     = {} -> null
        # (11 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.monitoring.grafana_dashboard.at_a_glance will be updated in-place
  ~ resource "grafana_dashboard" "at_a_glance" {
      ~ config_json  = jsonencode(
          ~ {
              + __inputs             = []
              + __requires           = []
              ~ annotations          = {
                  ~ list = [
                      ~ {
                            name       = "Annotations & Alerts"
                          + showIn     = 0
                            # (7 unchanged attributes hidden)
                        },
                    ]
                }
              - fiscalYearStartMonth = 0
              + gnetId               = null
              ~ graphTooltip         = 0 -> 1
              + hideControls         = false
              ~ panels               = [
                  ~ {
                      + collapsed     = false
                      ~ datasource    = {
                          ~ type = "prometheus" -> "datasource"
                          ~ uid  = "LAMKK7wVz" -> "grafana"
                        }
                      - description   = ""
                      - fieldConfig   = {
                          - defaults  = {
                              - color      = {
                                  - mode = "thresholds"
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      ~ gridPos       = {
                          ~ h = 8 -> 1
                          ~ w = 11 -> 24
                            # (2 unchanged attributes hidden)
                        }
                      - options       = {
                          - colorMode     = "value"
                          - graphMode     = "area"
                          - justifyMode   = "auto"
                          - orientation   = "auto"
                          - reduceOptions = {
                              - calcs  = [
                                  - "lastNotNull",
                                ]
                              - fields = ""
                              - values = false
                            }
                          - text          = {}
                          - textMode      = "auto"
                        }
                      - pluginVersion = "8.4.7"
                      ~ targets       = [
                          ~ {
                              ~ datasource   = {
                                  ~ type = "prometheus" -> "datasource"
                                  ~ uid  = "LAMKK7wVz" -> "grafana"
                                }
                              - expr         = "sum(rate(received_notifications{}[1h]))"
                              - legendFormat = "__auto"
                              - refId        = "Received"
                            },
                          - {
                              - datasource   = {
                                  - type = "prometheus"
                                  - uid  = "LAMKK7wVz"
                                }
                              - editorMode   = "code"
                              - expr         = "sum(increase(sent_apns_notifications{}[1h]))"
                              - hide         = false
                              - legendFormat = "__auto"
                              - range        = true
                              - refId        = "SentAPNS"
                            },
                          - {
                              - datasource   = {
                                  - type = "prometheus"
                                  - uid  = "LAMKK7wVz"
                                }
                              - editorMode   = "code"
                              - expr         = "sum(increase(sent_fcm_notifications{}[1h]))"
                              - hide         = false
                              - legendFormat = "__auto"
                              - range        = true
                              - refId        = "SentFCM"
                            },
                        ]
                      ~ title         = "Notifications per Hour" -> "Application"
                      ~ type          = "stat" -> "row"
                    },
                  ~ {
                      + datasource    = {
                          + type = "prometheus"
                          + uid  = "LAMKK7wVz"
                        }
                      + description   = null
                      + fieldConfig   = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + thresholdsStyle   = {
                                      + mode = "off"
                                    }
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                    ]
                                }
                              + unit       = "cps"
                            }
                          + overrides = []
                        }
                      ~ gridPos       = {
                          ~ h = 1 -> 8
                          ~ w = 24 -> 4
                          ~ y = 8 -> 1
                            # (1 unchanged attribute hidden)
                        }
                      + links         = []
                      + options       = {
                          + legend  = {
                              + calcs       = []
                              + displayMode = "list"
                              + placement   = "bottom"
                            }
                          + tooltip = {
                              + mode = "multi"
                              + sort = "none"
                            }
                        }
                      + pluginVersion = "8.4.7"
                      + targets       = [
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + exemplar     = true
                              + expr         = "sum by (aws_ecs_task_revision, name) (rate(postgres_queries_total[$__rate_interval]))"
                              + legendFormat = "{{name}} r{{aws_ecs_task_revision}}"
                              + refId        = "PostgresQueryRate"
                            },
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + exemplar     = true
                              + expr         = "sum(rate(postgres_queries_total[$__rate_interval]))"
                              + legendFormat = "r{{aws_ecs_task_revision}}"
                              + refId        = "PostgresQueryRateTotal"
                            },
                        ]
                      ~ title         = "Graphs" -> "Postgres Query Rate"
                      ~ type          = "row" -> "timeseries"
                    },
                  ~ {
                      + description   = null
                      ~ fieldConfig   = {
                          ~ defaults  = {
                              ~ custom     = {
                                  - axisLabel         = "Notifications"
                                  - hideFrom          = {
                                      - legend  = false
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                    # (11 unchanged attributes hidden)
                                }
                              ~ unit       = "none" -> "ms"
                                # (3 unchanged attributes hidden)
                            }
                          ~ overrides = [
                              - {
                                  - __systemRef = "hideSeriesFrom"
                                  - matcher     = {
                                      - id      = "byNames"
                                      - options = {
                                          - mode     = "exclude"
                                          - names    = [
                                              - "sum(received_notifications{})",
                                            ]
                                          - prefix   = "All except:"
                                          - readOnly = true
                                        }
                                    }
                                  - properties  = [
                                      - {
                                          - id    = "custom.hideFrom"
                                          - value = {
                                              - legend  = false
                                              - tooltip = false
                                              - viz     = true
                                            }
                                        },
                                    ]
                                },
                            ]
                        }
                      ~ gridPos       = {
                          ~ w = 11 -> 4
                          ~ x = 0 -> 4
                          ~ y = 9 -> 1
                            # (1 unchanged attribute hidden)
                        }
                      + links         = []
                      ~ options       = {
                          ~ tooltip = {
                              ~ mode = "single" -> "multi"
                                # (1 unchanged attribute hidden)
                            }
                            # (1 unchanged attribute hidden)
                        }
                      + pluginVersion = "8.4.7"
                      ~ targets       = [
                          ~ {
                              ~ exemplar     = true -> false
                              ~ expr         = "sum(increase(received_notifications{}[1h]))" -> "sum by (aws_ecs_task_revision, name) (rate(postgres_query_latency_sum[$__rate_interval])) / sum by (aws_ecs_task_revision, name) (rate(postgres_query_latency_count[$__rate_interval]))"
                              - format       = "time_series"
                              - interval     = ""
                              ~ legendFormat = "" -> "{{name}} r{{aws_ecs_task_revision}}"
                              ~ refId        = "Notifications" -> "PostgresQueryLatency"
                                # (1 unchanged attribute hidden)
                            },
                        ]
                      ~ title         = "Received Notifications" -> "Postgres Query Latency"
                        # (2 unchanged attributes hidden)
                    },
                  - {
                      - datasource  = {
                          - type = "prometheus"
                          - uid  = "LAMKK7wVz"
                        }
                      - fieldConfig = {
                          - defaults  = {
                              - color      = {
                                  - mode = "palette-classic"
                                }
                              - custom     = {
                                  - axisLabel         = "Clients"
                                  - axisPlacement     = "auto"
                                  - barAlignment      = 0
                                  - drawStyle         = "line"
                                  - fillOpacity       = 0
                                  - gradientMode      = "none"
                                  - hideFrom          = {
                                      - legend  = false
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - lineWidth         = 1
                                  - pointSize         = 5
                                  - scaleDistribution = {
                                      - type = "linear"
                                    }
                                  - showPoints        = "auto"
                                  - spanNulls         = false
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                  - thresholdsStyle   = {
                                      - mode = "off"
                                    }
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                      - {
                                          - color = "red"
                                          - value = 80
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      - gridPos     = {
                          - h = 8
                          - w = 10
                          - x = 11
                          - y = 9
                        }
                      - options     = {
                          - legend  = {
                              - calcs       = []
                              - displayMode = "list"
                              - placement   = "bottom"
                            }
                          - tooltip = {
                              - mode = "single"
                              - sort = "none"
                            }
                        }
                      - targets     = [
                          - {
                              - datasource   = {
                                  - type = "prometheus"
                                  - uid  = "LAMKK7wVz"
                                }
                              - exemplar     = true
                              - expr         = "sum(rate(registered_clients{}[$__rate_interval]))"
                              - interval     = ""
                              - legendFormat = ""
                              - refId        = "Clients"
                            },
                        ]
                      - title       = "Client registration rate"
                      - type        = "timeseries"
                    },
                  - {
                      - collapsed = false
                      - gridPos   = {
                          - h = 1
                          - w = 24
                          - x = 0
                          - y = 17
                        }
                      - panels    = []
                      - title     = "AWS Load Balancer"
                      - type      = "row"
                    },
                  - {
                      - datasource  = {
                          - type = "cloudwatch"
                          - uid  = "XnMFKnQVk"
                        }
                      - fieldConfig = {
                          - defaults  = {
                              - color      = {
                                  - mode = "palette-classic"
                                }
                              - custom     = {
                                  - axisLabel         = ""
                                  - axisPlacement     = "auto"
                                  - barAlignment      = 0
                                  - drawStyle         = "line"
                                  - fillOpacity       = 0
                                  - gradientMode      = "none"
                                  - hideFrom          = {
                                      - legend  = false
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - lineWidth         = 1
                                  - pointSize         = 5
                                  - scaleDistribution = {
                                      - type = "linear"
                                    }
                                  - showPoints        = "auto"
                                  - spanNulls         = false
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                  - thresholdsStyle   = {
                                      - mode = "off"
                                    }
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                      - {
                                          - color = "red"
                                          - value = 80
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      - gridPos     = {
                          - h = 9
                          - w = 7
                          - x = 0
                          - y = 18
                        }
                      - options     = {
                          - legend  = {
                              - calcs       = []
                              - displayMode = "list"
                              - placement   = "bottom"
                            }
                          - tooltip = {
                              - mode = "single"
                              - sort = "none"
                            }
                        }
                      - targets     = [
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "RequestCount"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "A"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                        ]
                      - title       = "Requests"
                      - type        = "timeseries"
                    },
                  - {
                      - alert       = {
                          - alertRuleTags       = {}
                          - conditions          = [
                              - {
                                  - evaluator = {
                                      - params = [
                                          - 15,
                                        ]
                                      - type   = "gt"
                                    }
                                  - operator  = {
                                      - type = "and"
                                    }
                                  - query     = {
                                      - params = [
                                          - "A",
                                          - "5m",
                                          - "now",
                                        ]
                                    }
                                  - reducer   = {
                                      - params = []
                                      - type   = "sum"
                                    }
                                  - type      = "query"
                                },
                              - {
                                  - evaluator = {
                                      - params = [
                                          - 15,
                                        ]
                                      - type   = "gt"
                                    }
                                  - operator  = {
                                      - type = "or"
                                    }
                                  - query     = {
                                      - params = [
                                          - "B",
                                          - "5m",
                                          - "now",
                                        ]
                                    }
                                  - reducer   = {
                                      - params = []
                                      - type   = "sum"
                                    }
                                  - type      = "query"
                                },
                            ]
                          - executionErrorState = "alerting"
                          - for                 = ""
                          - frequency           = "1m"
                          - handler             = 1
                          - message             = "Echo server - Prod - 5XX error"
                          - name                = "staging Echo Server 5XX alert"
                          - noDataState         = "no_data"
                          - notifications       = []
                        }
                      - datasource  = {
                          - type = "cloudwatch"
                          - uid  = "XnMFKnQVk"
                        }
                      - fieldConfig = {
                          - defaults  = {
                              - color      = {
                                  - mode = "palette-classic"
                                }
                              - custom     = {
                                  - axisLabel         = ""
                                  - axisPlacement     = "auto"
                                  - barAlignment      = 0
                                  - drawStyle         = "line"
                                  - fillOpacity       = 0
                                  - gradientMode      = "none"
                                  - hideFrom          = {
                                      - legend  = false
                                      - mode    = "dashed"
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - lineWidth         = 1
                                  - pointSize         = 5
                                  - scaleDistribution = {
                                      - type = "linear"
                                    }
                                  - showPoints        = "auto"
                                  - spanNulls         = false
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                  - thresholdsStyle   = {
                                      - mode = "off"
                                    }
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                      - {
                                          - color = "red"
                                          - value = 80
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      - gridPos     = {
                          - h = 9
                          - w = 7
                          - x = 7
                          - y = 18
                        }
                      - options     = {
                          - legend  = {
                              - calcs       = []
                              - displayMode = "list"
                              - placement   = "bottom"
                            }
                          - tooltip = {
                              - mode = "single"
                              - sort = "none"
                            }
                        }
                      - targets     = [
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "HTTPCode_ELB_5XX_Count"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "A"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "HTTPCode_Target_5XX_Count"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "B"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                        ]
                      - thresholds  = [
                          - {
                              - colorMode = "critical"
                              - op        = "gt"
                              - value     = 1
                              - visible   = true
                            },
                        ]
                      - title       = "5XX"
                      - type        = "timeseries"
                    },
                  - {
                      - datasource  = {
                          - type = "cloudwatch"
                          - uid  = "XnMFKnQVk"
                        }
                      - fieldConfig = {
                          - defaults  = {
                              - color      = {
                                  - mode = "palette-classic"
                                }
                              - custom     = {
                                  - axisLabel         = ""
                                  - axisPlacement     = "auto"
                                  - barAlignment      = 0
                                  - drawStyle         = "line"
                                  - fillOpacity       = 0
                                  - gradientMode      = "none"
                                  - hideFrom          = {
                                      - legend  = false
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - lineWidth         = 1
                                  - pointSize         = 5
                                  - scaleDistribution = {
                                      - type = "linear"
                                    }
                                  - showPoints        = "auto"
                                  - spanNulls         = false
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                  - thresholdsStyle   = {
                                      - mode = "off"
                                    }
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                      - {
                                          - color = "red"
                                          - value = 80
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      - gridPos     = {
                          - h = 9
                          - w = 7
                          - x = 14
                          - y = 18
                        }
                      - options     = {
                          - legend  = {
                              - calcs       = []
                              - displayMode = "list"
                              - placement   = "bottom"
                            }
                          - tooltip = {
                              - mode = "single"
                              - sort = "none"
                            }
                        }
                      - targets     = [
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "HTTPCode_ELB_4XX_Count"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "A"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "HTTPCode_Target_4XX_Count"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "B"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                        ]
                      - title       = "4XX"
                      - type        = "timeseries"
                    },
                ]
              + refresh              = ""
              + rows                 = []
                tags                 = []
              ~ timepicker           = {
                  + refresh_intervals = [
                      + "5s",
                      + "10s",
                      + "30s",
                      + "1m",
                      + "5m",
                      + "15m",
                      + "30m",
                      + "1h",
                      + "2h",
                      + "1d",
                    ]
                  + time_options      = [
                      + "5m",
                      + "15m",
                      + "1h",
                      + "6h",
                      + "12h",
                      + "24h",
                      + "2d",
                      + "7d",
                      + "30d",
                    ]
                }
              ~ timezone             = "" -> "utc"
              ~ title                = "staging-push" -> "Push Server - Staging"
              ~ uid                  = "staging-push" -> "push-staging"
              - weekStart            = ""
                # (7 unchanged attributes hidden)
            }
        )
        id           = "0:staging-push"
        # (8 unchanged attributes hidden)
    }

  # module.monitoring.grafana_dashboard.at_a_glance_old will be created
  + resource "grafana_dashboard" "at_a_glance_old" {
      + config_json  = jsonencode(
            {
              + annotations          = {
                  + list = [
                      + {
                          + builtIn    = 1
                          + datasource = "-- Grafana --"
                          + enable     = true
                          + hide       = true
                          + iconColor  = "rgba(0, 211, 255, 1)"
                          + name       = "Annotations & Alerts"
                          + target     = {
                              + limit    = 100
                              + matchAny = false
                              + tags     = []
                              + type     = "dashboard"
                            }
                          + type       = "dashboard"
                        },
                    ]
                }
              + editable             = true
              + fiscalYearStartMonth = 0
              + graphTooltip         = 0
              + links                = []
              + liveNow              = false
              + panels               = [
                  + {
                      + datasource    = {
                          + type = "prometheus"
                          + uid  = "LAMKK7wVz"
                        }
                      + description   = ""
                      + fieldConfig   = {
                          + defaults  = {
                              + color      = {
                                  + mode = "thresholds"
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                    ]
                                }
                            }
                          + overrides = []
                        }
                      + gridPos       = {
                          + h = 8
                          + w = 11
                          + x = 0
                          + y = 0
                        }
                      + options       = {
                          + colorMode     = "value"
                          + graphMode     = "area"
                          + justifyMode   = "auto"
                          + orientation   = "auto"
                          + reduceOptions = {
                              + calcs  = [
                                  + "lastNotNull",
                                ]
                              + fields = ""
                              + values = false
                            }
                          + text          = {}
                          + textMode      = "auto"
                        }
                      + pluginVersion = "8.4.7"
                      + targets       = [
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + expr         = "sum(rate(received_notifications{}[1h]))"
                              + legendFormat = "__auto"
                              + refId        = "Received"
                            },
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + editorMode   = "code"
                              + expr         = "sum(increase(sent_apns_notifications{}[1h]))"
                              + hide         = false
                              + legendFormat = "__auto"
                              + range        = true
                              + refId        = "SentAPNS"
                            },
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + editorMode   = "code"
                              + expr         = "sum(increase(sent_fcm_notifications{}[1h]))"
                              + hide         = false
                              + legendFormat = "__auto"
                              + range        = true
                              + refId        = "SentFCM"
                            },
                        ]
                      + title         = "Notifications per Hour"
                      + type          = "stat"
                    },
                  + {
                      + gridPos = {
                          + h = 1
                          + w = 24
                          + x = 0
                          + y = 8
                        }
                      + title   = "Graphs"
                      + type    = "row"
                    },
                  + {
                      + datasource  = {
                          + type = "prometheus"
                          + uid  = "LAMKK7wVz"
                        }
                      + fieldConfig = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisLabel         = "Notifications"
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + hideFrom          = {
                                      + legend  = false
                                      + tooltip = false
                                      + viz     = false
                                    }
                                  + lineInterpolation = "linear"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + stacking          = {
                                      + group = "A"
                                      + mode  = "none"
                                    }
                                  + thresholdsStyle   = {
                                      + mode = "off"
                                    }
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                    ]
                                }
                              + unit       = "none"
                            }
                          + overrides = [
                              + {
                                  + __systemRef = "hideSeriesFrom"
                                  + matcher     = {
                                      + id      = "byNames"
                                      + options = {
                                          + mode     = "exclude"
                                          + names    = [
                                              + "sum(received_notifications{})",
                                            ]
                                          + prefix   = "All except:"
                                          + readOnly = true
                                        }
                                    }
                                  + properties  = [
                                      + {
                                          + id    = "custom.hideFrom"
                                          + value = {
                                              + legend  = false
                                              + tooltip = false
                                              + viz     = true
                                            }
                                        },
                                    ]
                                },
                            ]
                        }
                      + gridPos     = {
                          + h = 8
                          + w = 11
                          + x = 0
                          + y = 9
                        }
                      + options     = {
                          + legend  = {
                              + calcs       = []
                              + displayMode = "list"
                              + placement   = "bottom"
                            }
                          + tooltip = {
                              + mode = "single"
                              + sort = "none"
                            }
                        }
                      + targets     = [
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + exemplar     = true
                              + expr         = "sum(increase(received_notifications{}[1h]))"
                              + format       = "time_series"
                              + interval     = ""
                              + legendFormat = ""
                              + refId        = "Notifications"
                            },
                        ]
                      + title       = "Received Notifications"
                      + type        = "timeseries"
                    },
                  + {
                      + datasource  = {
                          + type = "prometheus"
                          + uid  = "LAMKK7wVz"
                        }
                      + fieldConfig = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisLabel         = "Clients"
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + hideFrom          = {
                                      + legend  = false
                                      + tooltip = false
                                      + viz     = false
                                    }
                                  + lineInterpolation = "linear"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + stacking          = {
                                      + group = "A"
                                      + mode  = "none"
                                    }
                                  + thresholdsStyle   = {
                                      + mode = "off"
                                    }
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                      + {
                                          + color = "red"
                                          + value = 80
                                        },
                                    ]
                                }
                            }
                          + overrides = []
                        }
                      + gridPos     = {
                          + h = 8
                          + w = 10
                          + x = 11
                          + y = 9
                        }
                      + options     = {
                          + legend  = {
                              + calcs       = []
                              + displayMode = "list"
                              + placement   = "bottom"
                            }
                          + tooltip = {
                              + mode = "single"
                              + sort = "none"
                            }
                        }
                      + targets     = [
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + exemplar     = true
                              + expr         = "sum(rate(registered_clients{}[$__rate_interval]))"
                              + interval     = ""
                              + legendFormat = ""
                              + refId        = "Clients"
                            },
                        ]
                      + title       = "Client registration rate"
                      + type        = "timeseries"
                    },
                  + {
                      + collapsed = false
                      + gridPos   = {
                          + h = 1
                          + w = 24
                          + x = 0
                          + y = 17
                        }
                      + panels    = []
                      + title     = "AWS Load Balancer"
                      + type      = "row"
                    },
                  + {
                      + datasource  = {
                          + type = "cloudwatch"
                          + uid  = "XnMFKnQVk"
                        }
                      + fieldConfig = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisLabel         = ""
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + hideFrom          = {
                                      + legend  = false
                                      + tooltip = false
                                      + viz     = false
                                    }
                                  + lineInterpolation = "linear"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + stacking          = {
                                      + group = "A"
                                      + mode  = "none"
                                    }
                                  + thresholdsStyle   = {
                                      + mode = "off"
                                    }
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                      + {
                                          + color = "red"
                                          + value = 80
                                        },
                                    ]
                                }
                            }
                          + overrides = []
                        }
                      + gridPos     = {
                          + h = 9
                          + w = 7
                          + x = 0
                          + y = 18
                        }
                      + options     = {
                          + legend  = {
                              + calcs       = []
                              + displayMode = "list"
                              + placement   = "bottom"
                            }
                          + tooltip = {
                              + mode = "single"
                              + sort = "none"
                            }
                        }
                      + targets     = [
                          + {
                              + alias            = ""
                              + datasource       = {
                                  + type = "cloudwatch"
                                  + uid  = "XnMFKnQVk"
                                }
                              + dimensions       = {
                                  + LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              + expression       = ""
                              + id               = ""
                              + matchExact       = true
                              + metricEditorMode = 0
                              + metricName       = "RequestCount"
                              + metricQueryType  = 0
                              + namespace        = "AWS/ApplicationELB"
                              + period           = ""
                              + queryMode        = "Metrics"
                              + refId            = "A"
                              + region           = "default"
                              + sqlExpression    = ""
                              + statistic        = "Sum"
                            },
                        ]
                      + title       = "Requests"
                      + type        = "timeseries"
                    },
                  + {
                      + alert       = {
                          + alertRuleTags       = {}
                          + conditions          = [
                              + {
                                  + evaluator = {
                                      + params = [
                                          + 15,
                                        ]
                                      + type   = "gt"
                                    }
                                  + operator  = {
                                      + type = "and"
                                    }
                                  + query     = {
                                      + params = [
                                          + "A",
                                          + "5m",
                                          + "now",
                                        ]
                                    }
                                  + reducer   = {
                                      + params = []
                                      + type   = "sum"
                                    }
                                  + type      = "query"
                                },
                              + {
                                  + evaluator = {
                                      + params = [
                                          + 15,
                                        ]
                                      + type   = "gt"
                                    }
                                  + operator  = {
                                      + type = "or"
                                    }
                                  + query     = {
                                      + params = [
                                          + "B",
                                          + "5m",
                                          + "now",
                                        ]
                                    }
                                  + reducer   = {
                                      + params = []
                                      + type   = "sum"
                                    }
                                  + type      = "query"
                                },
                            ]
                          + executionErrorState = "alerting"
                          + for                 = ""
                          + frequency           = "1m"
                          + handler             = 1
                          + message             = "Echo server - Prod - 5XX error"
                          + name                = "staging Echo Server 5XX alert"
                          + noDataState         = "no_data"
                          + notifications       = []
                        }
                      + datasource  = {
                          + type = "cloudwatch"
                          + uid  = "XnMFKnQVk"
                        }
                      + fieldConfig = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisLabel         = ""
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + hideFrom          = {
                                      + legend  = false
                                      + mode    = "dashed"
                                      + tooltip = false
                                      + viz     = false
                                    }
                                  + lineInterpolation = "linear"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + stacking          
...

Action: pull_request

@@ -352,7 +368,7 @@ resource "grafana_dashboard" "at_a_glance" {
"uid" : grafana_data_source.prometheus.uid
},
"exemplar" : true,
"expr" : "sum(rate(registered_clients{}[$__rate_interval]))",
"expr" : "sum(rate(registered_clients_total{}[$__rate_interval]))",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After #334 this name has changed

image

@chris13524 chris13524 requested a review from nopestack May 20, 2024 20:16
Copy link
Contributor

Show Plan

[command]/home/runner/work/_temp/e07a3f65-4344-4686-b819-b465b3e82207/terraform-bin -chdir=terraform show -no-color /tmp/plan.tfplan

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create
  ~ update in-place
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # module.ecs.aws_ecs_service.app_service will be updated in-place
  ~ resource "aws_ecs_service" "app_service" {
        id                                 = "arn:aws:ecs:eu-central-1:898587786287:service/staging-push/staging-push-service"
        name                               = "staging-push-service"
        tags                               = {}
      ~ task_definition                    = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push:193" -> (known after apply)
        # (15 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

  # module.ecs.aws_ecs_task_definition.app_task_definition must be replaced
-/+ resource "aws_ecs_task_definition" "app_task_definition" {
      ~ arn                      = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push:193" -> (known after apply)
      ~ arn_without_revision     = "arn:aws:ecs:eu-central-1:898587786287:task-definition/staging-push" -> (known after apply)
      ~ container_definitions    = (sensitive value) # forces replacement
      ~ id                       = "staging-push" -> (known after apply)
      ~ revision                 = 193 -> (known after apply)
      - tags                     = {} -> null
        # (11 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.monitoring.grafana_dashboard.at_a_glance will be updated in-place
  ~ resource "grafana_dashboard" "at_a_glance" {
      ~ config_json  = jsonencode(
          ~ {
              + __inputs             = []
              + __requires           = []
              ~ annotations          = {
                  ~ list = [
                      ~ {
                            name       = "Annotations & Alerts"
                          + showIn     = 0
                            # (7 unchanged attributes hidden)
                        },
                    ]
                }
              - fiscalYearStartMonth = 0
              + gnetId               = null
              ~ graphTooltip         = 0 -> 1
              + hideControls         = false
              ~ panels               = [
                  ~ {
                      + collapsed     = false
                      ~ datasource    = {
                          ~ type = "prometheus" -> "datasource"
                          ~ uid  = "LAMKK7wVz" -> "grafana"
                        }
                      - description   = ""
                      - fieldConfig   = {
                          - defaults  = {
                              - color      = {
                                  - mode = "thresholds"
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      ~ gridPos       = {
                          ~ h = 8 -> 1
                          ~ w = 11 -> 24
                            # (2 unchanged attributes hidden)
                        }
                      - options       = {
                          - colorMode     = "value"
                          - graphMode     = "area"
                          - justifyMode   = "auto"
                          - orientation   = "auto"
                          - reduceOptions = {
                              - calcs  = [
                                  - "lastNotNull",
                                ]
                              - fields = ""
                              - values = false
                            }
                          - text          = {}
                          - textMode      = "auto"
                        }
                      - pluginVersion = "8.4.7"
                      ~ targets       = [
                          ~ {
                              ~ datasource   = {
                                  ~ type = "prometheus" -> "datasource"
                                  ~ uid  = "LAMKK7wVz" -> "grafana"
                                }
                              - expr         = "sum(rate(received_notifications{}[1h]))"
                              - legendFormat = "__auto"
                              - refId        = "Received"
                            },
                          - {
                              - datasource   = {
                                  - type = "prometheus"
                                  - uid  = "LAMKK7wVz"
                                }
                              - editorMode   = "code"
                              - expr         = "sum(increase(sent_apns_notifications{}[1h]))"
                              - hide         = false
                              - legendFormat = "__auto"
                              - range        = true
                              - refId        = "SentAPNS"
                            },
                          - {
                              - datasource   = {
                                  - type = "prometheus"
                                  - uid  = "LAMKK7wVz"
                                }
                              - editorMode   = "code"
                              - expr         = "sum(increase(sent_fcm_notifications{}[1h]))"
                              - hide         = false
                              - legendFormat = "__auto"
                              - range        = true
                              - refId        = "SentFCM"
                            },
                        ]
                      ~ title         = "Notifications per Hour" -> "Application"
                      ~ type          = "stat" -> "row"
                    },
                  ~ {
                      + datasource    = {
                          + type = "prometheus"
                          + uid  = "LAMKK7wVz"
                        }
                      + description   = null
                      + fieldConfig   = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + thresholdsStyle   = {
                                      + mode = "off"
                                    }
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                    ]
                                }
                              + unit       = "cps"
                            }
                          + overrides = []
                        }
                      ~ gridPos       = {
                          ~ h = 1 -> 8
                          ~ w = 24 -> 4
                          ~ y = 8 -> 1
                            # (1 unchanged attribute hidden)
                        }
                      + links         = []
                      + options       = {
                          + legend  = {
                              + calcs       = []
                              + displayMode = "list"
                              + placement   = "bottom"
                            }
                          + tooltip = {
                              + mode = "multi"
                              + sort = "none"
                            }
                        }
                      + pluginVersion = "8.4.7"
                      + targets       = [
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + exemplar     = true
                              + expr         = "sum by (aws_ecs_task_revision, name) (rate(postgres_queries_total[$__rate_interval]))"
                              + legendFormat = "{{name}} r{{aws_ecs_task_revision}}"
                              + refId        = "PostgresQueryRate"
                            },
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + exemplar     = true
                              + expr         = "sum(rate(postgres_queries_total[$__rate_interval]))"
                              + legendFormat = "r{{aws_ecs_task_revision}}"
                              + refId        = "PostgresQueryRateTotal"
                            },
                        ]
                      ~ title         = "Graphs" -> "Postgres Query Rate"
                      ~ type          = "row" -> "timeseries"
                    },
                  ~ {
                      + description   = null
                      ~ fieldConfig   = {
                          ~ defaults  = {
                              ~ custom     = {
                                  - axisLabel         = "Notifications"
                                  - hideFrom          = {
                                      - legend  = false
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                    # (11 unchanged attributes hidden)
                                }
                              ~ unit       = "none" -> "ms"
                                # (3 unchanged attributes hidden)
                            }
                          ~ overrides = [
                              - {
                                  - __systemRef = "hideSeriesFrom"
                                  - matcher     = {
                                      - id      = "byNames"
                                      - options = {
                                          - mode     = "exclude"
                                          - names    = [
                                              - "sum(received_notifications{})",
                                            ]
                                          - prefix   = "All except:"
                                          - readOnly = true
                                        }
                                    }
                                  - properties  = [
                                      - {
                                          - id    = "custom.hideFrom"
                                          - value = {
                                              - legend  = false
                                              - tooltip = false
                                              - viz     = true
                                            }
                                        },
                                    ]
                                },
                            ]
                        }
                      ~ gridPos       = {
                          ~ w = 11 -> 4
                          ~ x = 0 -> 4
                          ~ y = 9 -> 1
                            # (1 unchanged attribute hidden)
                        }
                      + links         = []
                      ~ options       = {
                          ~ tooltip = {
                              ~ mode = "single" -> "multi"
                                # (1 unchanged attribute hidden)
                            }
                            # (1 unchanged attribute hidden)
                        }
                      + pluginVersion = "8.4.7"
                      ~ targets       = [
                          ~ {
                              ~ exemplar     = true -> false
                              ~ expr         = "sum(increase(received_notifications{}[1h]))" -> "sum by (aws_ecs_task_revision, name) (rate(postgres_query_latency_sum[$__rate_interval])) / sum by (aws_ecs_task_revision, name) (rate(postgres_query_latency_count[$__rate_interval]))"
                              - format       = "time_series"
                              - interval     = ""
                              ~ legendFormat = "" -> "{{name}} r{{aws_ecs_task_revision}}"
                              ~ refId        = "Notifications" -> "PostgresQueryLatency"
                                # (1 unchanged attribute hidden)
                            },
                        ]
                      ~ title         = "Received Notifications" -> "Postgres Query Latency"
                        # (2 unchanged attributes hidden)
                    },
                  - {
                      - datasource  = {
                          - type = "prometheus"
                          - uid  = "LAMKK7wVz"
                        }
                      - fieldConfig = {
                          - defaults  = {
                              - color      = {
                                  - mode = "palette-classic"
                                }
                              - custom     = {
                                  - axisLabel         = "Clients"
                                  - axisPlacement     = "auto"
                                  - barAlignment      = 0
                                  - drawStyle         = "line"
                                  - fillOpacity       = 0
                                  - gradientMode      = "none"
                                  - hideFrom          = {
                                      - legend  = false
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - lineWidth         = 1
                                  - pointSize         = 5
                                  - scaleDistribution = {
                                      - type = "linear"
                                    }
                                  - showPoints        = "auto"
                                  - spanNulls         = false
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                  - thresholdsStyle   = {
                                      - mode = "off"
                                    }
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                      - {
                                          - color = "red"
                                          - value = 80
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      - gridPos     = {
                          - h = 8
                          - w = 10
                          - x = 11
                          - y = 9
                        }
                      - options     = {
                          - legend  = {
                              - calcs       = []
                              - displayMode = "list"
                              - placement   = "bottom"
                            }
                          - tooltip = {
                              - mode = "single"
                              - sort = "none"
                            }
                        }
                      - targets     = [
                          - {
                              - datasource   = {
                                  - type = "prometheus"
                                  - uid  = "LAMKK7wVz"
                                }
                              - exemplar     = true
                              - expr         = "sum(rate(registered_clients{}[$__rate_interval]))"
                              - interval     = ""
                              - legendFormat = ""
                              - refId        = "Clients"
                            },
                        ]
                      - title       = "Client registration rate"
                      - type        = "timeseries"
                    },
                  - {
                      - collapsed = false
                      - gridPos   = {
                          - h = 1
                          - w = 24
                          - x = 0
                          - y = 17
                        }
                      - panels    = []
                      - title     = "AWS Load Balancer"
                      - type      = "row"
                    },
                  - {
                      - datasource  = {
                          - type = "cloudwatch"
                          - uid  = "XnMFKnQVk"
                        }
                      - fieldConfig = {
                          - defaults  = {
                              - color      = {
                                  - mode = "palette-classic"
                                }
                              - custom     = {
                                  - axisLabel         = ""
                                  - axisPlacement     = "auto"
                                  - barAlignment      = 0
                                  - drawStyle         = "line"
                                  - fillOpacity       = 0
                                  - gradientMode      = "none"
                                  - hideFrom          = {
                                      - legend  = false
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - lineWidth         = 1
                                  - pointSize         = 5
                                  - scaleDistribution = {
                                      - type = "linear"
                                    }
                                  - showPoints        = "auto"
                                  - spanNulls         = false
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                  - thresholdsStyle   = {
                                      - mode = "off"
                                    }
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                      - {
                                          - color = "red"
                                          - value = 80
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      - gridPos     = {
                          - h = 9
                          - w = 7
                          - x = 0
                          - y = 18
                        }
                      - options     = {
                          - legend  = {
                              - calcs       = []
                              - displayMode = "list"
                              - placement   = "bottom"
                            }
                          - tooltip = {
                              - mode = "single"
                              - sort = "none"
                            }
                        }
                      - targets     = [
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "RequestCount"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "A"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                        ]
                      - title       = "Requests"
                      - type        = "timeseries"
                    },
                  - {
                      - alert       = {
                          - alertRuleTags       = {}
                          - conditions          = [
                              - {
                                  - evaluator = {
                                      - params = [
                                          - 15,
                                        ]
                                      - type   = "gt"
                                    }
                                  - operator  = {
                                      - type = "and"
                                    }
                                  - query     = {
                                      - params = [
                                          - "A",
                                          - "5m",
                                          - "now",
                                        ]
                                    }
                                  - reducer   = {
                                      - params = []
                                      - type   = "sum"
                                    }
                                  - type      = "query"
                                },
                              - {
                                  - evaluator = {
                                      - params = [
                                          - 15,
                                        ]
                                      - type   = "gt"
                                    }
                                  - operator  = {
                                      - type = "or"
                                    }
                                  - query     = {
                                      - params = [
                                          - "B",
                                          - "5m",
                                          - "now",
                                        ]
                                    }
                                  - reducer   = {
                                      - params = []
                                      - type   = "sum"
                                    }
                                  - type      = "query"
                                },
                            ]
                          - executionErrorState = "alerting"
                          - for                 = ""
                          - frequency           = "1m"
                          - handler             = 1
                          - message             = "Echo server - Prod - 5XX error"
                          - name                = "staging Echo Server 5XX alert"
                          - noDataState         = "no_data"
                          - notifications       = []
                        }
                      - datasource  = {
                          - type = "cloudwatch"
                          - uid  = "XnMFKnQVk"
                        }
                      - fieldConfig = {
                          - defaults  = {
                              - color      = {
                                  - mode = "palette-classic"
                                }
                              - custom     = {
                                  - axisLabel         = ""
                                  - axisPlacement     = "auto"
                                  - barAlignment      = 0
                                  - drawStyle         = "line"
                                  - fillOpacity       = 0
                                  - gradientMode      = "none"
                                  - hideFrom          = {
                                      - legend  = false
                                      - mode    = "dashed"
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - lineWidth         = 1
                                  - pointSize         = 5
                                  - scaleDistribution = {
                                      - type = "linear"
                                    }
                                  - showPoints        = "auto"
                                  - spanNulls         = false
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                  - thresholdsStyle   = {
                                      - mode = "off"
                                    }
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                      - {
                                          - color = "red"
                                          - value = 80
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      - gridPos     = {
                          - h = 9
                          - w = 7
                          - x = 7
                          - y = 18
                        }
                      - options     = {
                          - legend  = {
                              - calcs       = []
                              - displayMode = "list"
                              - placement   = "bottom"
                            }
                          - tooltip = {
                              - mode = "single"
                              - sort = "none"
                            }
                        }
                      - targets     = [
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "HTTPCode_ELB_5XX_Count"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "A"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "HTTPCode_Target_5XX_Count"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "B"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                        ]
                      - thresholds  = [
                          - {
                              - colorMode = "critical"
                              - op        = "gt"
                              - value     = 1
                              - visible   = true
                            },
                        ]
                      - title       = "5XX"
                      - type        = "timeseries"
                    },
                  - {
                      - datasource  = {
                          - type = "cloudwatch"
                          - uid  = "XnMFKnQVk"
                        }
                      - fieldConfig = {
                          - defaults  = {
                              - color      = {
                                  - mode = "palette-classic"
                                }
                              - custom     = {
                                  - axisLabel         = ""
                                  - axisPlacement     = "auto"
                                  - barAlignment      = 0
                                  - drawStyle         = "line"
                                  - fillOpacity       = 0
                                  - gradientMode      = "none"
                                  - hideFrom          = {
                                      - legend  = false
                                      - tooltip = false
                                      - viz     = false
                                    }
                                  - lineInterpolation = "linear"
                                  - lineWidth         = 1
                                  - pointSize         = 5
                                  - scaleDistribution = {
                                      - type = "linear"
                                    }
                                  - showPoints        = "auto"
                                  - spanNulls         = false
                                  - stacking          = {
                                      - group = "A"
                                      - mode  = "none"
                                    }
                                  - thresholdsStyle   = {
                                      - mode = "off"
                                    }
                                }
                              - mappings   = []
                              - thresholds = {
                                  - mode  = "absolute"
                                  - steps = [
                                      - {
                                          - color = "green"
                                          - value = null
                                        },
                                      - {
                                          - color = "red"
                                          - value = 80
                                        },
                                    ]
                                }
                            }
                          - overrides = []
                        }
                      - gridPos     = {
                          - h = 9
                          - w = 7
                          - x = 14
                          - y = 18
                        }
                      - options     = {
                          - legend  = {
                              - calcs       = []
                              - displayMode = "list"
                              - placement   = "bottom"
                            }
                          - tooltip = {
                              - mode = "single"
                              - sort = "none"
                            }
                        }
                      - targets     = [
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "HTTPCode_ELB_4XX_Count"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "A"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                          - {
                              - alias            = ""
                              - datasource       = {
                                  - type = "cloudwatch"
                                  - uid  = "XnMFKnQVk"
                                }
                              - dimensions       = {
                                  - LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              - expression       = ""
                              - id               = ""
                              - matchExact       = true
                              - metricEditorMode = 0
                              - metricName       = "HTTPCode_Target_4XX_Count"
                              - metricQueryType  = 0
                              - namespace        = "AWS/ApplicationELB"
                              - period           = ""
                              - queryMode        = "Metrics"
                              - refId            = "B"
                              - region           = "default"
                              - sqlExpression    = ""
                              - statistic        = "Sum"
                            },
                        ]
                      - title       = "4XX"
                      - type        = "timeseries"
                    },
                ]
              + refresh              = ""
              + rows                 = []
                tags                 = []
              ~ timepicker           = {
                  + refresh_intervals = [
                      + "5s",
                      + "10s",
                      + "30s",
                      + "1m",
                      + "5m",
                      + "15m",
                      + "30m",
                      + "1h",
                      + "2h",
                      + "1d",
                    ]
                  + time_options      = [
                      + "5m",
                      + "15m",
                      + "1h",
                      + "6h",
                      + "12h",
                      + "24h",
                      + "2d",
                      + "7d",
                      + "30d",
                    ]
                }
              ~ timezone             = "" -> "utc"
              ~ title                = "staging-push" -> "Push Server - Staging"
              ~ uid                  = "staging-push" -> "push-staging"
              - weekStart            = ""
                # (7 unchanged attributes hidden)
            }
        )
        id           = "0:staging-push"
        # (8 unchanged attributes hidden)
    }

  # module.monitoring.grafana_dashboard.at_a_glance_old will be created
  + resource "grafana_dashboard" "at_a_glance_old" {
      + config_json  = jsonencode(
            {
              + annotations          = {
                  + list = [
                      + {
                          + builtIn    = 1
                          + datasource = "-- Grafana --"
                          + enable     = true
                          + hide       = true
                          + iconColor  = "rgba(0, 211, 255, 1)"
                          + name       = "Annotations & Alerts"
                          + target     = {
                              + limit    = 100
                              + matchAny = false
                              + tags     = []
                              + type     = "dashboard"
                            }
                          + type       = "dashboard"
                        },
                    ]
                }
              + editable             = true
              + fiscalYearStartMonth = 0
              + graphTooltip         = 0
              + links                = []
              + liveNow              = false
              + panels               = [
                  + {
                      + datasource    = {
                          + type = "prometheus"
                          + uid  = "LAMKK7wVz"
                        }
                      + description   = ""
                      + fieldConfig   = {
                          + defaults  = {
                              + color      = {
                                  + mode = "thresholds"
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                    ]
                                }
                            }
                          + overrides = []
                        }
                      + gridPos       = {
                          + h = 8
                          + w = 11
                          + x = 0
                          + y = 0
                        }
                      + options       = {
                          + colorMode     = "value"
                          + graphMode     = "area"
                          + justifyMode   = "auto"
                          + orientation   = "auto"
                          + reduceOptions = {
                              + calcs  = [
                                  + "lastNotNull",
                                ]
                              + fields = ""
                              + values = false
                            }
                          + text          = {}
                          + textMode      = "auto"
                        }
                      + pluginVersion = "8.4.7"
                      + targets       = [
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + expr         = "sum(rate(received_notifications{}[1h]))"
                              + legendFormat = "__auto"
                              + refId        = "Received"
                            },
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + editorMode   = "code"
                              + expr         = "sum(increase(sent_apns_notifications{}[1h]))"
                              + hide         = false
                              + legendFormat = "__auto"
                              + range        = true
                              + refId        = "SentAPNS"
                            },
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + editorMode   = "code"
                              + expr         = "sum(increase(sent_fcm_notifications{}[1h]))"
                              + hide         = false
                              + legendFormat = "__auto"
                              + range        = true
                              + refId        = "SentFCM"
                            },
                        ]
                      + title         = "Notifications per Hour"
                      + type          = "stat"
                    },
                  + {
                      + gridPos = {
                          + h = 1
                          + w = 24
                          + x = 0
                          + y = 8
                        }
                      + title   = "Graphs"
                      + type    = "row"
                    },
                  + {
                      + datasource  = {
                          + type = "prometheus"
                          + uid  = "LAMKK7wVz"
                        }
                      + fieldConfig = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisLabel         = "Notifications"
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + hideFrom          = {
                                      + legend  = false
                                      + tooltip = false
                                      + viz     = false
                                    }
                                  + lineInterpolation = "linear"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + stacking          = {
                                      + group = "A"
                                      + mode  = "none"
                                    }
                                  + thresholdsStyle   = {
                                      + mode = "off"
                                    }
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                    ]
                                }
                              + unit       = "none"
                            }
                          + overrides = [
                              + {
                                  + __systemRef = "hideSeriesFrom"
                                  + matcher     = {
                                      + id      = "byNames"
                                      + options = {
                                          + mode     = "exclude"
                                          + names    = [
                                              + "sum(received_notifications{})",
                                            ]
                                          + prefix   = "All except:"
                                          + readOnly = true
                                        }
                                    }
                                  + properties  = [
                                      + {
                                          + id    = "custom.hideFrom"
                                          + value = {
                                              + legend  = false
                                              + tooltip = false
                                              + viz     = true
                                            }
                                        },
                                    ]
                                },
                            ]
                        }
                      + gridPos     = {
                          + h = 8
                          + w = 11
                          + x = 0
                          + y = 9
                        }
                      + options     = {
                          + legend  = {
                              + calcs       = []
                              + displayMode = "list"
                              + placement   = "bottom"
                            }
                          + tooltip = {
                              + mode = "single"
                              + sort = "none"
                            }
                        }
                      + targets     = [
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + exemplar     = true
                              + expr         = "sum(increase(received_notifications{}[1h]))"
                              + format       = "time_series"
                              + interval     = ""
                              + legendFormat = ""
                              + refId        = "Notifications"
                            },
                        ]
                      + title       = "Received Notifications"
                      + type        = "timeseries"
                    },
                  + {
                      + datasource  = {
                          + type = "prometheus"
                          + uid  = "LAMKK7wVz"
                        }
                      + fieldConfig = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisLabel         = "Clients"
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + hideFrom          = {
                                      + legend  = false
                                      + tooltip = false
                                      + viz     = false
                                    }
                                  + lineInterpolation = "linear"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + stacking          = {
                                      + group = "A"
                                      + mode  = "none"
                                    }
                                  + thresholdsStyle   = {
                                      + mode = "off"
                                    }
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                      + {
                                          + color = "red"
                                          + value = 80
                                        },
                                    ]
                                }
                            }
                          + overrides = []
                        }
                      + gridPos     = {
                          + h = 8
                          + w = 10
                          + x = 11
                          + y = 9
                        }
                      + options     = {
                          + legend  = {
                              + calcs       = []
                              + displayMode = "list"
                              + placement   = "bottom"
                            }
                          + tooltip = {
                              + mode = "single"
                              + sort = "none"
                            }
                        }
                      + targets     = [
                          + {
                              + datasource   = {
                                  + type = "prometheus"
                                  + uid  = "LAMKK7wVz"
                                }
                              + exemplar     = true
                              + expr         = "sum(rate(registered_clients_total{}[$__rate_interval]))"
                              + interval     = ""
                              + legendFormat = ""
                              + refId        = "Clients"
                            },
                        ]
                      + title       = "Client registration rate"
                      + type        = "timeseries"
                    },
                  + {
                      + collapsed = false
                      + gridPos   = {
                          + h = 1
                          + w = 24
                          + x = 0
                          + y = 17
                        }
                      + panels    = []
                      + title     = "AWS Load Balancer"
                      + type      = "row"
                    },
                  + {
                      + datasource  = {
                          + type = "cloudwatch"
                          + uid  = "XnMFKnQVk"
                        }
                      + fieldConfig = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisLabel         = ""
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + hideFrom          = {
                                      + legend  = false
                                      + tooltip = false
                                      + viz     = false
                                    }
                                  + lineInterpolation = "linear"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + stacking          = {
                                      + group = "A"
                                      + mode  = "none"
                                    }
                                  + thresholdsStyle   = {
                                      + mode = "off"
                                    }
                                }
                              + mappings   = []
                              + thresholds = {
                                  + mode  = "absolute"
                                  + steps = [
                                      + {
                                          + color = "green"
                                          + value = null
                                        },
                                      + {
                                          + color = "red"
                                          + value = 80
                                        },
                                    ]
                                }
                            }
                          + overrides = []
                        }
                      + gridPos     = {
                          + h = 9
                          + w = 7
                          + x = 0
                          + y = 18
                        }
                      + options     = {
                          + legend  = {
                              + calcs       = []
                              + displayMode = "list"
                              + placement   = "bottom"
                            }
                          + tooltip = {
                              + mode = "single"
                              + sort = "none"
                            }
                        }
                      + targets     = [
                          + {
                              + alias            = ""
                              + datasource       = {
                                  + type = "cloudwatch"
                                  + uid  = "XnMFKnQVk"
                                }
                              + dimensions       = {
                                  + LoadBalancer = "app/staging-push-load-balancer/aea5ef9d0a34453a"
                                }
                              + expression       = ""
                              + id               = ""
                              + matchExact       = true
                              + metricEditorMode = 0
                              + metricName       = "RequestCount"
                              + metricQueryType  = 0
                              + namespace        = "AWS/ApplicationELB"
                              + period           = ""
                              + queryMode        = "Metrics"
                              + refId            = "A"
                              + region           = "default"
                              + sqlExpression    = ""
                              + statistic        = "Sum"
                            },
                        ]
                      + title       = "Requests"
                      + type        = "timeseries"
                    },
                  + {
                      + alert       = {
                          + alertRuleTags       = {}
                          + conditions          = [
                              + {
                                  + evaluator = {
                                      + params = [
                                          + 15,
                                        ]
                                      + type   = "gt"
                                    }
                                  + operator  = {
                                      + type = "and"
                                    }
                                  + query     = {
                                      + params = [
                                          + "A",
                                          + "5m",
                                          + "now",
                                        ]
                                    }
                                  + reducer   = {
                                      + params = []
                                      + type   = "sum"
                                    }
                                  + type      = "query"
                                },
                              + {
                                  + evaluator = {
                                      + params = [
                                          + 15,
                                        ]
                                      + type   = "gt"
                                    }
                                  + operator  = {
                                      + type = "or"
                                    }
                                  + query     = {
                                      + params = [
                                          + "B",
                                          + "5m",
                                          + "now",
                                        ]
                                    }
                                  + reducer   = {
                                      + params = []
                                      + type   = "sum"
                                    }
                                  + type      = "query"
                                },
                            ]
                          + executionErrorState = "alerting"
                          + for                 = ""
                          + frequency           = "1m"
                          + handler             = 1
                          + message             = "Echo server - Prod - 5XX error"
                          + name                = "staging Echo Server 5XX alert"
                          + noDataState         = "no_data"
                          + notifications       = []
                        }
                      + datasource  = {
                          + type = "cloudwatch"
                          + uid  = "XnMFKnQVk"
                        }
                      + fieldConfig = {
                          + defaults  = {
                              + color      = {
                                  + mode = "palette-classic"
                                }
                              + custom     = {
                                  + axisLabel         = ""
                                  + axisPlacement     = "auto"
                                  + barAlignment      = 0
                                  + drawStyle         = "line"
                                  + fillOpacity       = 0
                                  + gradientMode      = "none"
                                  + hideFrom          = {
                                      + legend  = false
                                      + mode    = "dashed"
                                      + tooltip = false
                                      + viz     = false
                                    }
                                  + lineInterpolation = "linear"
                                  + lineWidth         = 1
                                  + pointSize         = 5
                                  + scaleDistribution = {
                                      + type = "linear"
                                    }
                                  + showPoints        = "auto"
                                  + spanNulls         = false
                                  + stacking    
...

Action: pull_request

@chris13524 chris13524 merged commit 4d687a7 into main May 20, 2024
9 checks passed
@chris13524 chris13524 deleted the fix/postgres-query-metrics branch May 20, 2024 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted The issue has been accepted into the project
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants