Skip to content

quincy: mgr/telemetry: handle empty device report when "send" is triggered#45092

Merged
ljflores merged 1 commit intoceph:quincyfrom
ljflores:wip-54326-quincy
Feb 22, 2022
Merged

quincy: mgr/telemetry: handle empty device report when "send" is triggered#45092
ljflores merged 1 commit intoceph:quincyfrom
ljflores:wip-54326-quincy

Conversation

@ljflores
Copy link
Copy Markdown
Member

backport tracker: https://tracker.ceph.com/issues/54326


backport of #44994
parent tracker: https://tracker.ceph.com/issues/54250

this backport was staged using ceph-backport.sh version 16.0.0.6848
find the latest version at https://github.com/ceph/ceph/blob/master/src/script/ceph-backport.sh

On certain environments, such as the "ceph-dev-docker" environment
(https://github.com/ricardoasmarques/ceph-dev-docker), the mgr
module is unable to fetch device metrics. As a result, the device
report generated by "gather_device_report()" returns an empty dict.
This causes an AssertionError when the "send" function is triggered
(i.e. by running `ceph telemetry status` or `ceph telemetry send`),
and the module crashes.

The fix in this commit checks that the generated device report
contains metrics before trying to send it. If the device report
does not contain metrics (it returns an empty dict), the module
will log an appropriate message in the mgr log and not send the
device report.

If this scenario happens when running the `ceph telemetry send` command,
the user will additionally see this message:
```
Ceph report sent to https://telemetry.ceph.com/report
Unable to send device report: channel is on, but generated report was empty.
```

I also added a few more debug messages in gather_device_report() to make
future debugging easier.

Fixes: https://tracker.ceph.com/issues/54250
Signed-off-by: Laura Flores <lflores@redhat.com>
(cherry picked from commit 54e0e58)
@ljflores ljflores added this to the quincy milestone Feb 21, 2022
@ljflores ljflores added the core label Feb 21, 2022
@nizamial09
Copy link
Copy Markdown
Member

jenkins test api

@ljflores
Copy link
Copy Markdown
Member Author

http://pulpito.front.sepia.ceph.com/yuriw-2022-02-21_18:20:15-rados-wip-yuri11-testing-2022-02-21-0831-quincy-distro-default-smithi/

Failures, unrelated:
https://tracker.ceph.com/issues/52124
https://tracker.ceph.com/issues/50280
https://tracker.ceph.com/issues/54337

Details:
1. Invalid read of size 8 in handle_recovery_delete() - Ceph - RADOS
2. cephadm: RuntimeError: uid/gid not found - Ceph
3. Selinux denials seen on fs/rados teuthology runs - Infrastructure

@ljflores ljflores merged commit e023f92 into ceph:quincy Feb 22, 2022
@ljflores ljflores deleted the wip-54326-quincy branch February 22, 2022 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants