cephadm/nvmeof: scrape nvmeof prometheus endpoint

[avanthakkar]

Fixes: https://tracker.ceph.com/issues/64536

Result:

Prometheus Target:

Contribution Guidelines

• To sign and title your commits, please refer to Submitting Patches to Ceph.

• If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

• When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

• Tracker (select at least one)
  • References tracker ticket
  • Very recent bug; references commit where it was introduced
  • New feature (ticket optional)
  • Doc update (no ticket needed)
  • Code cleanup (no ticket needed)
• Component impact
  • Affects Dashboard, opened tracker ticket
  • Affects Orchestrator, opened tracker ticket
  • No impact that needs to be tracked
• Documentation (select at least one)
  • Updates relevant documentation
  • No doc update is appropriate
• Tests (select at least one)
  • Includes unit test(s)
  • Includes integration test(s)
  • Includes bug reproducer
  • No tests

Show available Jenkins commands

• jenkins retest this please
• jenkins test classic perf
• jenkins test crimson perf
• jenkins test signed
• jenkins test make check
• jenkins test make check arm64
• jenkins test submodules
• jenkins test dashboard
• jenkins test dashboard cephadm
• jenkins test api
• jenkins test docs
• jenkins render docs
• jenkins test ceph-volume all
• jenkins test ceph-volume tox
• jenkins test windows
• jenkins test rook e2e

[adk3798]

Code itself seems good to me beyond one nitpick I left a comment on. If this is working in your testing (which it seems to be from the screenshots) this is okay with me as long as it causes no regressions in a teuthology run.

[adk3798]

while currently we don't allow this to be changed, for future proofing reasons, we might want to have this be a variable passed to the context for generating this file rather than just a hardcoded value. Something like (in prepare_create for the NvmeofService class)

context = {
            'spec': spec,
            'name': name,
            'addr': host_ip,
            'port': spec.port,
            'log_level': 'WARN',
            'rpc_socket': '/var/tmp/spdk.sock',
            'transport_tcp_options': transport_tcp_options,
            'rados_id': rados_id,
            'prometheus_port' : self.PROMETHEUS_PORT      <--- THIS LINE
        }

and then we can populate the conf with that value here.

[avanthakkar]

Sounds good to me, probably we can create a tracker for configuring this & also I think adding ssl support?

[adk3798]

https://pulpito.ceph.com/adking-2024-02-27_05:08:20-orch:cephadm-wip-adk-testing-2024-02-26-1600-distro-default-smithi/

not going to go over the failures in the regular detail as we're still cleaning up the in cluster log stuff, but it was mostly just things we still need to add to ignorelists and the mds_upgrade_sequence test having issues. Only test that failed due to PRs in the run was https://pulpito.ceph.com/adking-2024-02-27_05:08:20-orch:cephadm-wip-adk-testing-2024-02-26-1600-distro-default-smithi/7574464 so we can't merge the PRs making changes to mgr/nfs but everything else seems to be okay.

[pcuzner]

LGTM - Thanks!