Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jinja2._stringdefs uses much RAM #666

Closed
methane opened this issue Jan 20, 2017 · 2 comments

Comments

@methane
Copy link
Contributor

commented Jan 20, 2017

jinja2._stringdefs makes large strings for each unicode category.
They eat 100KB+ RAM. But none of them are used.

Only xid_start and xid_continue are used by jinja2.lexer.
Would you remove category strings?

Your Environment

  • Python version: Python 3.6
  • Jinja version: 2.9.4
@ThiefMaster

This comment has been minimized.

Copy link
Member

commented Jan 20, 2017

100kb is not really "much ram". However, the module could probably be removed from sys.modules and the lexer.py module scope after this code has been executed so it can be garbage-collected...

try:
    compile('föö', '<unknown>', 'eval')
except SyntaxError:
    name_re = re.compile(r'\b[a-zA-Z_][a-zA-Z0-9_]*\b')
else:
    from jinja2 import _stringdefs
    name_re = re.compile(r'[%s][%s]*' % (_stringdefs.xid_start,
                                         _stringdefs.xid_continue))

Either way, the regex probably uses not much less memory since it contains all these chars.


IMO using non-ascii chars in identifiers is an awful idea, but I guess you can always find someone who doesn't code in english and Jinja is too mature to remove something like this anyway...

@davidism

This comment has been minimized.

Copy link
Member

commented Jul 2, 2017

Solved more by #731.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.