Support lxml v6.1.0

[quentin-alc]

LXML released the version 6.1.0 to fix a vulnerability GHSA-vfmq-68hx-4jfw

Running xmlsec 1.3.17 with lxml version 6.1.0 raise an InternalError:
xmlsec.InternalError: (-1, 'lxml & xmlsec libxml2 library version mismatch')

[mxamin]

@quentin-alc xmlsec 1.3.17 should be compatible with lxml 6.1.0. The error you see indicates that installed xmlsec and lxml use a different version of libxml2. I'm trying to remove this limitation, but it's not that easy to handle :)
If you use the pre-build wheels, you won't see such issue, it only happens when you build the library locally using the local libxml2 which probably doesn't match the version that lxml is build with.

If you give me the following information, I can investigate and help you with a solution:

• Python version?
• OS?
• The command you use to install lxml and xmlsec?

If you need more information, you can look into #356

[quentin-alc]

Thank you for the quick response

The python version is 3.12

For the OS we use Ubuntu 20.04.06 LTS, here are all the informations:

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

We use uv to manage our dependencies. We have a uv.lock file that is generated and then on the server uv sync --frozen to sync the libraries.

I will try to setup a small project that I can share with you that replicate the issue.

[mxamin]

That would be great if you can give me a sample to replicate the issue.
Based on your setup, you can easily use the pre-build wheels, and it should work right out of the box.
I recommend uninstalling both libs, clearing the cache and reinstall both lxml and xmlsec:

uv pip uninstall lxml xmlsec
uv cache clean
uv pip install lxml==6.1.0 xmlsec==1.3.17

[quentin-alc]

I created a small app to try to demonstrate it and it works on the same server, with same python version and dependencies manager.
It still doesn't work on my main project but since I can get it working on another example, the issue is probably on my side. I'll try to clean the caches and see how it goes.

Thanks for the support and sorry for the issue

[mxamin]

No problem :)

[quentin-alc]

For future reference, if anyone encounters the same issue: I managed to isolate the cause to a conflict between uWSGI and the xmlsec/lxml libraries.

The code runs perfectly when executed directly via the Python interpreter, but fails under uWSGI with the following error:

File ".../.venv/lib/python3.12/site-packages/onelogin/saml2/utils.py", line 23, in <module>
    import xmlsec
xmlsec.InternalError: (-1, 'lxml & xmlsec libxml2 library version mismatch')
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***

From what I understand, uWSGI loads the system's shared libxml2 library. When the Python application later tries to load lxml or xmlsec a version mismatch occurs because the libxml2 lib on the system might be different than the one in the wheels, leading to the InternalError