Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jinja2._stringdefs uses much RAM #666

Closed
methane opened this issue Jan 20, 2017 · 2 comments
Closed

jinja2._stringdefs uses much RAM #666

methane opened this issue Jan 20, 2017 · 2 comments

Comments

@methane
Copy link
Contributor

@methane methane commented Jan 20, 2017

jinja2._stringdefs makes large strings for each unicode category.
They eat 100KB+ RAM. But none of them are used.

Only xid_start and xid_continue are used by jinja2.lexer.
Would you remove category strings?

Your Environment

  • Python version: Python 3.6
  • Jinja version: 2.9.4
@ThiefMaster
Copy link
Member

@ThiefMaster ThiefMaster commented Jan 20, 2017

100kb is not really "much ram". However, the module could probably be removed from sys.modules and the lexer.py module scope after this code has been executed so it can be garbage-collected...

try:
    compile('föö', '<unknown>', 'eval')
except SyntaxError:
    name_re = re.compile(r'\b[a-zA-Z_][a-zA-Z0-9_]*\b')
else:
    from jinja2 import _stringdefs
    name_re = re.compile(r'[%s][%s]*' % (_stringdefs.xid_start,
                                         _stringdefs.xid_continue))

Either way, the regex probably uses not much less memory since it contains all these chars.


IMO using non-ascii chars in identifiers is an awful idea, but I guess you can always find someone who doesn't code in english and Jinja is too mature to remove something like this anyway...

Loading

@davidism
Copy link
Member

@davidism davidism commented Jul 2, 2017

Solved more by #731.

Loading

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 13, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants