Metrics missing after upgrading to 280.1.10

[fmoehler]

Describe the bug
Metrics stop getting forwarded to riemann for bosh deployed vms after upgrading to 280.1.10.

The issue seems to originate from this PR:
#2579

We created a release without this PR which fixed the issue.

It affects only metrics for vms that are deployed by bosh, not for bosh itself.
To Reproduce
Steps to reproduce the behavior (example):

1. Deploy a bosh director with 280.1.10
2. Deploy a vm that forwards metrics to riemann
3. Observe that the metrics are not coming in
4. When downgrading to 280.1.9 we observe the metrics flowing again

Expected behavior
The metrics should not be missing from 280.1.10

Versions (please complete the following information):

• Infrastructure: ALL
• BOSH version 280.1.10

Additional context
No error messages found as of now.

[beyhan]

@fmoehler are there any logs indicating an failed forwarding or there are no traces in the log indicating a failed forwarding?

[fmoehler]

We are investigating at the moment. So far we found these logs in the bosh director:

health_monitor.log:E, [2024-12-05T10:21:44.867088 #6] ERROR : Unable to send get /configs to director: Invalid URI: https://127.0.0.1:25555/info
health_monitor.log:E, [2024-12-05T10:21:44.867320 #6] ERROR : /var/vcap/data/packages/health_monitor/8eaa03bc6d09d3eb508485e697836cc0ea5fbdee/gem_home/ruby/3.2.0/gems/bosh-monitor-0.0.0/lib/bosh/monitor/director
.rb:69:in `rescue in perform_request'
health_monitor.stderr.log:/var/vcap/data/packages/health_monitor/8eaa03bc6d09d3eb508485e697836cc0ea5fbdee/gem_home/ruby/3.2.0/gems/async-2.20.0/lib/async/scheduler.rb:374:in `run_once!': undefined method `idle_d
uration' for nil:NilClass (NoMethodError)

[beyhan]

For me #2591 looks also related. In the trace here we have a different method which is reported as undefined idle_duration and not to_str but the behaviour is similar.

[fmoehler]

Yes we identified that part also. Will create a release with this change and retest. Thanks @beyhan !

[fmoehler]

Thanks @beyhan this fixes the issue. Will close this now!