mem-cache bug: only small fraction of memory allocated is actually used

[sssd-bot]

Cloned from Pagure issue: https://pagure.io/SSSD/sssd/issue/4160

• Created at 2020-02-24 16:11:16 by atikhonov
• Assigned to atikhonov
• Associated bugzillas
  • https://bugzilla.redhat.com/show_bug.cgi?id=1787067

---

sssd/src/responder/nss/nsssrv_mmap_cache.c

Line 1310 in f3e89aa

mc_ctx->dt_size = MC_DT_SIZE(n_elem, payload);

For the "data table" sss_mmap_cache_init() allocates n_elems*payload == SSS_MC_CACHE_ELEMENTS*{SSS_AVG_PASSWD_PAYLOAD|SSS_AVG_GROUP_PAYLOAD|SSS_AVG_INITGROUP_PAYLOAD} == 50000*MC_SLOT_SIZE*{4|3|5} bytes of memory.

But for the "free table" only n_elems/8 == SSS_MC_CACHE_ELEMENTS/8 bytes are allocated -- this corresponds to SSS_MC_CACHE_ELEMENTS.

The problems is, no other piece of code (besides sss_mmap_cache_init()) is aware of SSS_AVG_*_PAYLOAD payload multiplier -- merely MC_SLOT_SIZE is expected as payload size everywhere (see MC_SLOT_WITHIN_BOUNDS, MC_SIZE_TO_SLOTS, etc).

Thus only {1/4|1/3|1/5} of allocated memory is actually used.

This results in significant performance hit in environment with large amount of users / groups and heavy nss load.

Comments

---

Comment from atikhonov at 2020-02-24 16:11:43

Metadata Update from @atikhonov:

• Custom field rhbz adjusted to https://bugzilla.redhat.com/show_bug.cgi?id=1787067

---

Comment from atikhonov at 2020-03-05 11:27:55

PR: #999

---

Comment from atikhonov at 2020-03-05 11:29:52

Metadata Update from @atikhonov:

• Issue tagged with: PR

---

Comment from thalman at 2020-03-13 14:39:55

Metadata Update from @thalman:

• Issue tagged with: Next milestone

---

Comment from pbrezina at 2020-04-09 13:09:01

• sssd-1-16
  • 38d4eab - mem-cache: sizes of free and data tables were made consistent

---

Comment from pbrezina at 2020-04-09 13:09:01

Metadata Update from @pbrezina:

• Issue close_status updated to: Fixed
• Issue status updated to: Closed (was: Open)

---

Comment from atikhonov at 2020-04-09 14:07:23

Patches are not merged in master branch yet.

---

Comment from atikhonov at 2020-04-09 14:07:23

Metadata Update from @atikhonov:

• Issue status updated to: Open (was: Closed)

[pbrezina]

• master
  • 484507b - mem-cache: always cleanup old content
  • b96b05b - mem-cache: comment added
  • b7f3193 - NSS: make memcache size configurable in megabytes
  • 2ad4aa8 - mem-cache: added log message in case cache is full
  • be8052b - NSS: enhanced debug during mem-cache initialization
  • e12340e - NSS: avoid excessive log messages
  • 80e7163 - NSS: make memcache size configurable
  • 2d90e64 - mem-cache: sizes of free and data tables were made consistent