KeysManager not freeing memory after use

[rouge8]

With xmlsec==1.0.9, this function uses ~1MB of memory for xmlsec.KeysManager() on each call, that is never freed. With xmlsec==0.6.1, the memory seems to be freed when exiting the function.

def _encrypt_usernametoken(self, envelope):
    """Encrypt the ``UsernameToken`` element in the ``envelope``."""
    header = envelope.find(ns(SOAP_NS, 'Header'))
    security = header.find(ns(WSSE_NS, 'Security'))

    # Create a keys manager and load the cert into it.
    manager = xmlsec.KeysManager()
    key = xmlsec.Key.from_file(self.recipient_cert, xmlsec.KeyFormat.CERT_PEM)
    manager.add_key(key)

    # Encrypt the UsernameToken
    target = security.find(ns(WSSE_NS, 'UsernameToken'))

    # Create the EncryptedData node we will replace the target node with,
    # and make sure it has the contents XMLSec expects (a CipherValue node,
    # a KeyInfo node, and an EncryptedKey node within the KeyInfo which
    # itself has a CipherValue).
    enc_data = xmlsec.template.encrypted_data_create(
        envelope,
        xmlsec.Transform.DES3,
        type=xmlsec.EncryptionType.ELEMENT,
        ns='xenc',
    )
    xmlsec.template.encrypted_data_ensure_cipher_value(enc_data)
    key_info = xmlsec.template.encrypted_data_ensure_key_info(
        enc_data, ns='dsig')
    enc_key = xmlsec.template.add_encrypted_key(
        key_info,
        # Service requires RSA 1.5 encryption, which is the same as RSA
        # PKCS1.
        xmlsec.Transform.RSA_PKCS1,
    )
    xmlsec.template.encrypted_data_ensure_cipher_value(enc_key)

    enc_ctx = xmlsec.EncryptionContext(manager)
    # Generate a per-session DES key (will be encrypted using the cert).
    enc_ctx.key = xmlsec.Key.generate(
        xmlsec.KeyData.DES, 192, xmlsec.KeyDataType.SESSION)
    # Ask XMLSec to actually do the encryption.
    enc_data = enc_ctx.encrypt_xml(enc_data, target)

    # XMLSec inserts the EncryptedKey node directly within EncryptedData,
    # but WSSE wants it in the Security header instead, and referencing the
    # EncryptedData as well as the actual cert in a BinarySecurityToken.

    # Move the EncryptedKey node up into the wsse:Security header.
    security.insert(0, enc_key)

    # Create a wsse:BinarySecurityToken node containing the cert and add it
    # to the Security header.
    cert_bst = create_binary_security_token(self.recipient_cert)
    security.insert(0, cert_bst)

    # Create a ds:KeyInfo node referencing the BinarySecurityToken we just
    # created, and insert it into the EncryptedKey node.
    enc_key.insert(1, create_key_info_bst(cert_bst))

    # Add a DataReference from the EncryptedKey node to the EncryptedData.
    add_data_reference(enc_key, enc_data)

    # Remove the now-empty KeyInfo node from EncryptedData (it used to
    # contain EncryptedKey, but we moved that up into the Security header).
    enc_data.remove(key_info)

It also looks like our signing function leaks ~0.1MB/call with xmlsec==1.0.9:

def _sign(self, envelope):
    """Sign the ``soap:Body`` and ``wsu:Timestamp`` of ``envelope``."""
    # Insert the Signature node in the wsse:Security header.
    header = envelope.find(ns(SOAP_NS, 'Header'))
    security = header.find(ns(WSSE_NS, 'Security'))

    cert_bst = create_binary_security_token(self.sender_cert)
    security.append(cert_bst)

    # Create the Signature node.
    signature = xmlsec.template.create(
        envelope,
        xmlsec.Transform.EXCL_C14N,
        xmlsec.Transform.RSA_SHA1,
    )
    key_info = create_key_info_bst(cert_bst)
    signature.append(key_info)
    security.append(signature)

    # Perform the actual signing.
    ctx = xmlsec.SignatureContext()
    ctx.key = xmlsec.Key.from_file(self.sender_key, xmlsec.KeyFormat.PEM)
    _sign_node(ctx, signature, envelope.find(ns(SOAP_NS, 'Body')))
    _sign_node(ctx, signature, security.find(ns(WSU_NS, 'Timestamp')))
    ctx.sign(signature)

[bgaifullin]

@rouge8 I prepared patch to fix this issue, could you check branch fix_memory_leaks. it should fix memory leak in first function.

[rouge8]

@bgaifullin this is definitely an improvement. it still seems to be leaking some memory, but my guess is that's in signing rather than encryption?

[bgaifullin]

@rouge8 I found and fixed yet another leak. Could you please check again ?

[rouge8]

Yeah, #64 seems to fix the leaks I was seeing! Thanks!

[bgaifullin]

@rouge8, thank you for help. I release new version as soon as http://pythonhosted.org will be recovered.

[rouge8]

@bgaifullin looks like it's back. It was down for several days which is a little worrying 😱

[bgaifullin]

Fix in version: 1.1.0