Expose LSN and replication delay as metrics

[save-buffer]

Problem

We currently have no way to see what the current LSN of a compute its, and in case of read replicas, we don't know what the difference in LSNs is.

Summary of changes

Adds these metrics

[github-actions]

2886 tests run: 2759 passed, 0 failed, 127 skipped (full report)

---

Flaky tests (3)

Postgres 16

• test_gc_aggressive: debug
• test_vm_bit_clear_on_heap_lock: debug

Postgres 14

• test_lock_time_tracing: release

Code coverage* (full report)

• functions: 31.4% (6241 of 19881 functions)
• lines: 47.1% (46742 of 99293 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
057672c at 2024-05-06T22:59:58.713Z :recycle:}

[hlinnaka]

What is the difference between the neon_collector and neon_collector_autoscaling sections in the yaml file? Which metrics are supposed to go in which?

[hlinnaka]

What is the difference between the neon_collector and neon_collector_autoscaling sections in the yaml file? Which metrics are supposed to go in which?

Ok, found the explanation in the commit message of commit 4b55dad:

As discussed in neondatabase/autoscaling#895, we want to have a separate sql_exporter for simple metrics to avoid overload the database because the autoscaling agent needs to scrape at a higher interval. The new exporter is exposed at port 9499.

So if I understand correctly, the point of the "autoscaling" metrics is to expose metrics that will be used by autoscaling, to make scaling decisions. The replication lag metrics doesn't seem necessary for that. So I think these new metrics should only be added to the neon_collector metrics, not neon_collector_autoscaling.

[save-buffer]

So I think these new metrics should only be added to the neon_collector metrics, not neon_collector_autoscaling.

Got it, good catch! I just kind of assumed they were there because we for some reason had different specs for pods and vms. Just removed it.