Skip to content

fix memory leak in utils/jinja.py #62007

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 23, 2022
Merged

fix memory leak in utils/jinja.py #62007

merged 4 commits into from
Jun 23, 2022

Conversation

sakateka
Copy link
Contributor

@sakateka sakateka commented Apr 29, 2022

The SaltCacheLoader class was leaking through an external module atexit.
atexit.register has an internal list where it adds finalizers at each call,
LazyLoader at each re-import of jinja modules,
imported into them utils/jinja.py and thereby called atexit.register'. Since atexit` is executed at the end of the process life, it is better to
let the operating system terminate all connections and free up all resources.

What does this PR do?

fix memory leak

What issues does this PR fix or reference?

Fixes:
may be #61988

Merge requirements satisfied?

[NOTICE] Bug fixes or features added to Salt require tests.

Commits signed with GPG?

Yes

Please review Salt's Contributing Guide for best practices.

See GitHub's page on GPG signing for more information about signing commits with GPG.

The `SaltCacheLoader` class was leaking through an external module `atexit`.
`atexit.register` has an internal list where it adds finalizers at each call,
`LazyLoader` at each re-import of jinja modules,
imported into them `utils/jinja.py ` and thereby called `atexit.register'.
Since `atexit` is executed at the end of the process life, it is better to
let the operating system terminate all connections and free up all resources.
@sakateka sakateka requested a review from a team as a code owner April 29, 2022 18:55
@sakateka sakateka requested review from krionbsd and removed request for a team April 29, 2022 18:55
@sakateka sakateka mentioned this pull request May 18, 2022
3 tasks
@sakateka
Copy link
Contributor Author

@s0undt3ch
Hi, this is a very simple patch, can you help merge it?
After this patch and #62006 #62021, memory consumption on my master servers is very stable.
image

@github-actions
Copy link

Hi! I'm your friendly PR bot!

You might be wondering what I'm doing commenting here on your PR.

Yes, as a matter of fact, I am...

I'm just here to help us improve the documentation. I can't respond to
questions or anything, but what I can do, I do well!

Okay... so what do you do?

I detect modules that are missing docstrings or "CLI Example" on existing docstrings!
When I was created we had a lot of these. The documentation for these
modules need some love and attention to make Salt better for our users.

So what does that have to do with my PR?

I noticed that in this PR there are some files changed that have some of these
issues. So I'm leaving this comment to let you know your options.

Okay, what are they?

Well, my favorite, is that since you were making changes here I'm hoping that
you would be the most familiar with this module and be able to add some other
examples or fix any of the reported issues.

If I can, then what?

Well, you can either add them to this PR or add them to another PR. Either way is fine!

Well... what if I can't, or don't want to?

That's also fine! We appreciate all contributions to the Salt Project. If you
can't add those other examples, either because you're too busy, or unfamiliar,
or you just aren't interested, we still appreciate the contributions that
you've made already.

Whatever approach you decide to take, just drop a comment here letting us know!

Detected Issues (click me)
Check Known Missing Docstrings...........................................Failed
- hook id: invoke
- duration: 1.31s
- exit code: 1

/home/runner/.cache/pre-commit/repoz5fct3ox/py_env-python3/lib/python3.9/site-packages/_distutils_hack/init.py:30: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
The function 'jinja_raise' on 'salt/utils/jinja.py' does not have a docstring
The function 'regex_escape' on 'salt/utils/jinja.py' does not have a docstring
The function 'method_call' on 'salt/utils/jinja.py' does not have a docstring
The function 'show_full_context' on 'salt/utils/jinja.py' does not have a docstring
Found 4 errors


Thanks again!

@Ch3LL Ch3LL requested a review from dwoz June 22, 2022 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants