[DNM] rgw/lua: read lua script from bucket instead of sys obj

[yuvalif]

this is just used for testing. DO NOT MERGE

Signed-off-by: yuval Lifshitz ylifshit@redhat.com

to avoid failure during bucket retrieval use (assuming vstart):

aws --endpoint-url http://localhost:8000 s3 mb s3://lua-script-bucket

Contribution Guidelines

• To sign and title your commits, please refer to Submitting Patches to Ceph.

• If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

Checklist

• Tracker (select at least one)
  • References tracker ticket
  • Very recent bug; references commit where it was introduced
  • New feature (ticket optional)
  • Doc update (no ticket needed)
  • Code cleanup (no ticket needed)
• Component impact
  • Affects Dashboard, opened tracker ticket
  • Affects Orchestrator, opened tracker ticket
  • No impact that needs to be tracked
• Documentation (select at least one)
  • Updates relevant documentation
  • No doc update is appropriate
• Tests (select at least one)
  • Includes unit test(s)
  • Includes integration test(s)
  • Includes bug reproducer
  • No tests

Show available Jenkins commands

• jenkins retest this please
• jenkins test classic perf
• jenkins test crimson perf
• jenkins test signed
• jenkins test make check
• jenkins test make check arm64
• jenkins test submodules
• jenkins test dashboard
• jenkins test dashboard cephadm
• jenkins test api
• jenkins test docs
• jenkins render docs
• jenkins test ceph-volume all
• jenkins test ceph-volume tox
• jenkins test windows

[github-actions]

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

[yuvalif]

@mkogan1 do you still need this for your testing?

[mkogan1]

@mkogan1 do you still need this for your testing?

yes please, will check it as a time slot will become available to work on the memory issue again

[yuvalif]

seems like the memory issues was resolved regardless of system objects

[mattbenjamin]

seems like the memory issues was resolved regardless of system objects

what is the rca?

[yuvalif]

seems like the memory issues was resolved regardless of system objects

what is the rca?

@mkogan1 ?

[mkogan1]

Apologizing for previous comment, it was false-positive (because of a bug main recently stopped linking with tcmalloc)
the memory still grows on main branch

[yuvalif]

Apologizing for previous comment, it was false-positive (because of a bug main recently stopped linking with tcmalloc) the memory still grows on main branch

ok. please update on the results of running with non-system lua script object

[mkogan1]

updating that this branch run over 12 hours shows NO memory jump:

(Note that scale is in MB)

Verifying that tcmalloc is indeed in use here:

❯ git branch -vv
...
* wip-yuval-lua-in-bucket 77528bb9ace [yuvalif/wip-yuval-lua-in-bucket] rgw/lua: read lua script from bucket instead of sys obj

❯ ldd ./build/bin/radosgw | grep libtc
        libtcmalloc.so.4 => /lib64/libtcmalloc.so.4 (0x00007ffff57da000)

(The slow rise in memory reasons is in all probability because of reasons unrelated to this PR)

[yuvalif]

@mkogan1 thanks! this is interesting.
please note that the "fix" here is just for the purpose of investigation.
we should try and understand the issues with system objects (maybe caching?).
only as a last resort, we may consider storing the lua scripts as regular objects in a special, hidden bucket