mgr: map injector

[pereman2]

Well, I've used the OSDMap methods to set the max of osds and check them as existent in order to show them.

Signed-off-by: Pere Diaz Bou pdiazbou@redhat.com

Checklist

• References tracker ticket
• Updates documentation if necessary
• Includes tests for new functionality or reproducer for bug

---

Show available Jenkins commands

• jenkins retest this please
• jenkins test classic perf
• jenkins test crimson perf
• jenkins test signed
• jenkins test make check
• jenkins test make check arm64
• jenkins test submodules
• jenkins test dashboard
• jenkins test api
• jenkins test docs
• jenkins render docs
• jenkins test ceph-volume all
• jenkins test ceph-volume tox

[jdurgin]

Another question is how do we enable/parameterize this?

One way is dev-level ceph options - at least one to enable/disable fake map injection. We could use more to determine e.g. number of osds, or other things we might want to vary.

[tchaikov]

if we want to inject osdmaps into mgr. i'd suggest just monkey patch the mgr module under test using #41591. the change in this PR is able to inject maps in some predefined ways but it's still not flexible enough for testing.

[epuertat]

if we want to inject osdmaps into mgr. i'd suggest just monkey patch the mgr module under test using #41591. the change in this PR is able to inject maps in some predefined ways but it's still not flexible enough for testing.

@tchaikov we need to inject them at C++ side so the performance overhead of the PyFormatter is counted in. Otherwise, we cannot properly estimate the caching benefits unless we deploy a cluster with 1000's OSDs.

[pereman2]

if we want to inject osdmaps into mgr. i'd suggest just monkey patch the mgr module under test using #41591. the change in this PR is able to inject maps in some predefined ways but it's still not flexible enough for testing.

@tchaikov we need to inject them at C++ side so the performance overhead of the PyFormatter is counted in. Otherwise, we cannot properly estimate the caching benefits unless we deploy a cluster with 1000's OSDs.

@epuertat if you look at the PyOSDMap.cc we have this:

static PyObject *osdmap_dump(BasePyOSDMap* self, PyObject *obj)
{
  PyFormatter f;
  self->osdmap->dump(&f);
  return f.get();
}

which transaltes to BasePyOSDMap._dump(self) in the ceph_module.pyi. So what kefu wants is find a way to override the osdmap from mgr._get_osdmap so we can dump with the function above and then this would include the PyFormatter overhead that we need. All of this being done in the python side.

[github-actions]

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

[epuertat]

@pereman2 @tchaikov understood! Thanks for pointing to that. I wasn't aware that we had a CPython wrapper for the OSDMap class. If we plan to have more of those, it might be useful to rely on some improved C++ Python binding, like pybind11, in order to reduce boilerplate code, improve maintainability and get safer code.

That said, I still don't see clearly an alternative way to inject a fake number of OSDs together with the real data. @pereman2 if you know how to continue, no problem from my end, but please let's do something about this one: there're no approvals nor change requests.

If we don't have this merged, there's no easy way to validate the TTL Cache improvements (unless you have a few hundred nodes under your desk).

[epuertat]

LGTM. Let's merge this in the feature branch for testing purposes, and drop it when merging this back to master.