[Metricbeat] Add missing snapshot metrics to pubsub metricset in gcp

[kaiyan-sheng]

Right now, pubsub metricset is only collecting metrics from API endpoints below:

- "pubsub.googleapis.com/subscription/ack_message_count"
- "pubsub.googleapis.com/subscription/backlog_bytes"
- "pubsub.googleapis.com/subscription/num_outstanding_messages"
- "pubsub.googleapis.com/subscription/num_undelivered_messages"
- "pubsub.googleapis.com/subscription/oldest_unacked_message_age"
- "pubsub.googleapis.com/subscription/pull_ack_message_operation_count"
- "pubsub.googleapis.com/subscription/pull_ack_request_count"
- "pubsub.googleapis.com/subscription/pull_message_operation_count"
- "pubsub.googleapis.com/subscription/pull_request_count"
- "pubsub.googleapis.com/subscription/push_request_count"
- "pubsub.googleapis.com/subscription/push_request_latencies"
- "pubsub.googleapis.com/subscription/sent_message_count"
- "pubsub.googleapis.com/subscription/streaming_pull_ack_message_operation_count"
- "pubsub.googleapis.com/subscription/streaming_pull_ack_request_count"
- "pubsub.googleapis.com/subscription/streaming_pull_message_operation_count"
- "pubsub.googleapis.com/subscription/streaming_pull_response_count"
- "pubsub.googleapis.com/topic/message_sizes"
- "pubsub.googleapis.com/topic/send_message_operation_count"
- "pubsub.googleapis.com/topic/send_request_count"

We are missing all snapshot related metrics for pubsub monitoring as well as part of the topic related. This list needs to be revisited and enriched later as well as https://www.elastic.co/guide/en/beats/metricbeat/master/metricbeat-metricset-googlecloud-pubsub.html.

Dashboard needs to be improved as well after more metrics are added.

[elasticmachine]

Pinging @elastic/integrations-platforms (Team:Platforms)

[sayden]

I'm pretty sure that the metrics that we are missing were in beta when I collected them at first sight so, yes, we must add them if they are now GA.

https://cloud.google.com/monitoring/api/metrics_gcp#gcp-pubsub

[sayden]

Missing metrics are the following:

        - "pubsub.googleapis.com/snapshot/backlog_bytes"
        - "pubsub.googleapis.com/snapshot/backlog_bytes_by_region"
        - "pubsub.googleapis.com/snapshot/config_updates_count"
        - "pubsub.googleapis.com/snapshot/num_messages"
        - "pubsub.googleapis.com/snapshot/num_messages_by_region"
        - "pubsub.googleapis.com/snapshot/oldest_message_age"
        - "pubsub.googleapis.com/subscription/byte_cost"
        - "pubsub.googleapis.com/subscription/config_updates_count"
        - "pubsub.googleapis.com/subscription/dead_letter_message_count"
        - "pubsub.googleapis.com/subscription/mod_ack_deadline_message_count"
        - "pubsub.googleapis.com/subscription/mod_ack_deadline_message_operation_count"
        - "pubsub.googleapis.com/subscription/mod_ack_deadline_request_count"
        - "pubsub.googleapis.com/subscription/oldest_retained_acked_message_age"
        - "pubsub.googleapis.com/subscription/oldest_retained_acked_message_age_by_region"
        - "pubsub.googleapis.com/subscription/oldest_unacked_message_age_by_region"
        - "pubsub.googleapis.com/subscription/retained_acked_bytes"
        - "pubsub.googleapis.com/subscription/retained_acked_bytes_by_region"
        - "pubsub.googleapis.com/subscription/seek_request_count"
        - "pubsub.googleapis.com/subscription/streaming_pull_mod_ack_deadline_message_operation_count"
        - "pubsub.googleapis.com/subscription/streaming_pull_mod_ack_deadline_request_count"
        - "pubsub.googleapis.com/subscription/unacked_bytes_by_region"
        - "pubsub.googleapis.com/topic/byte_cost"
        - "pubsub.googleapis.com/topic/config_updates_count"
        - "pubsub.googleapis.com/topic/oldest_retained_acked_message_age_by_region"
        - "pubsub.googleapis.com/topic/oldest_unacked_message_age_by_region"
        - "pubsub.googleapis.com/topic/retained_acked_bytes_by_region"
        - "pubsub.googleapis.com/topic/unacked_bytes_by_region"

I suggest to curate them before adding as each metric is a linear increase in the cost on API calls.

[kaiyan-sheng]

This is done. Thank you for your input @sayden 👍