Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid entity expansion attacks in Element Tree #68426

vadmium opened this issue May 19, 2015 · 2 comments

Avoid entity expansion attacks in Element Tree #68426

vadmium opened this issue May 19, 2015 · 2 comments


Copy link

vadmium commented May 19, 2015

BPO 24238
Nosy @tiran, @vadmium, @serhiy-storchaka
  • etree_20130519.patch
  • etree-entities.v2.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2015-05-19.11:17:59.200>
    labels = ['type-security', 'expert-XML']
    title = 'Avoid entity expansion attacks in Element Tree'
    updated_at = <Date 2016-06-04.06:48:58.706>
    user = '' fields:

    activity = <Date 2016-06-04.06:48:58.706>
    actor = 'martin.panter'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['XML']
    creation = <Date 2015-05-19.11:17:59.200>
    creator = 'martin.panter'
    dependencies = []
    files = ['39430', '43185']
    hgrepos = []
    issue_num = 24238
    keywords = ['patch']
    message_count = 2.0
    messages = ['243577', '267238']
    nosy_count = 3.0
    nosy_names = ['christian.heimes', 'martin.panter', 'serhiy.storchaka']
    pr_nums = []
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'security'
    url = ''
    versions = ['Python 3.6']

    Copy link
    Member Author

    vadmium commented May 19, 2015

    This patch could be the basis of an alternative to Christian Heimes’s patch in bpo-17239. It adds a parser flag to the Element Tree modules so that they will immediately raise an exception when an entity declaration is encountered. I believe this should be sufficient to avoid DOS vulnerabilities like the Billion Laughs attack, where a small XML entity reference expands into a large string, and/or involves a large number of entity expansions.

    I think the advantage of this patch over the patch in bpo-17239 is this one should work on the current Expat library (which I understand Python can load externally). The other patch modifies the Expat library itself, so would only be useful when Python’s internal Expat library is being used (or the external Expat library was also patched in a similar manner).

    The disadvantage of this patch is that it disables handling XML data as soon as an entity is declared, even if the entities are not actually used, or they are only used in a non-malicious way. The other patch allows a limited amount of entity expansion.

    I would like some feedback on:

    • What others think of the basic approach, compared with Christian’s approach in bpo-17239
    • If reject_entities=True should be switched on by default, which could break compatibility, but could be sensible for most cases of basic XML parsing
    • If my changes to the examples in the documentation are excessive
    • If other Element Tree APIs should be modified similarly to XMLParser

    So far I have only changed the XMLParser class. The following APIs accept a parser object, so can also avoid the vulnerability by passing a custom parser object:

    • fromstringlist()
    • iterparse(), though “parser” is listed as deprecated (by bpo-17741)
    • parse() (module-level function)
    • XML()
    • XMLID()
    • ElementTree.parse() (method of ElementTree class)

    These APIs don’t have a custom parser object, so they are still always vulnerable:

    • fromstring()
    • XMLPullParser

    @vadmium vadmium added expert-XML type-security A security issue labels May 19, 2015
    Copy link
    Member Author

    vadmium commented Jun 4, 2016

    Today I discovered that Christian’s defusedxml project already does the same sort of thing. The difference is he calls the parameter forbid_entities. So I have updated my patch and changed the name from reject_entities to forbid_entities for compatibility.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    None yet

    No branches or pull requests

    1 participant