[>=2.4.5] Fix to CVE-2022-25236 breaks biboumi, ClairMeta, jxmlease, libwbxml, openleadr-python, rnv, xmltodict

[jamiegau]

It appears the recent security update for expat has broken many systems.

Under ubuntu 20.04 I was able to regress by doing
apt-get install libexpat1=2.2.9-1build1 libexpat1-dev=2.2.9-1build1

Under Alpine linux and all my docker installs.. I AM NOT ABLE TO REGRESS AS THE REPO REPLACED THE RELEASE APK WITH AN UPDATED BUT BROKEN VERSION. They did not add a newer apk with a different version number..
I cannot find an older version to cook into the dockerfile. HELP!

[hartwork]

Hi @jamiegau , I'm a bit scared by you shouting in all caps here but I'd be willing to see if I can help and understand the problem better. We can jump on a voice call soon-ish if you reach out to me via my profile e-mail, if you like. If that doesn't fit for some reason, I'd ask to provide move details here of what the regression symptom is. Thank you!

[sebageek]

I think what @jamiegau refers to is a problem with the latest libexpat1 update in Ubuntu from 2.2.5-3ubuntu0.2 to 2.2.5-3ubuntu0.4. The problem is also described in this[0] Ubuntu bug. I have been running into the same problem in my production landscape after installing the latest security updates into our doocker image:

xml.parsers.expat.ExpatError: out of memory: line 1, column 0

It can be reproduced with a recent Ubuntu 18.04 docker image:

$ docker run --rm -it ubuntu:18.04
root@container:/# (apt update && apt install libexpat1 python3 -y) &>/dev/null
root@container:/# dpkg -l|grep libexpat
ii  libexpat1:amd64            2.2.5-3ubuntu0.4          amd64        XML parsing C library - runtime library
root@container:/# python3
Python 3.6.9 (default, Dec  8 2021, 21:08:43) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from xml.parsers import expat
>>> parser = expat.ParserCreate(namespace_separator=':')
>>> parser.Parse('<foo></foo>', True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
xml.parsers.expat.ExpatError: out of memory: line 1, column 0

[0] https://bugs.launchpad.net/ubuntu/+source/python-xmltodict/+bug/1961800

[jamiegau]

I use a tool called clairmeta. It reads XML files that make up cinema DCPs to test them for accuracy.
Today my toolchain stopped working on my production system after a small update.
The dockerfile was processed targeting alpine 3.15

I also noticed it was broken in the same way under ubuntu 20.04. I had a system it was not broken, and to reproduce the issue, only after doing a apt full-upgrade did the problem occur.

This made me look at packages that were replaced and I tracked it down to expat.

There is a recent security update that replaced the expat release that came with 20.04. I had to regress to the previous version and the problem went away.

I then started working on my docker containers to implement the same regression...
and I went to CAPS, because Alpine actually didn't release a new version but replaced/over wrote the same current version file.
This can be see on the repo as the creation date of the expat lib updated to 21st of this month while all other files are far old.

I suppose I am shouting because it is amazing that such a silly behaviour by Alpine in that they didn't update the release number and add it to the repo as so if such a problem occurred, those using alpine could regress.. But it's impossible. I cannot even find the original version of expat to cook into my dockerfile.

I would have to compile it myself, which could result in unknown amount of work, or move to a ubuntu based container that I can regress on.

In any case, as the clairmeta is a python based library, and utilises lxml, it likely relies on expat as well.
The clairmeta tool fires an exception of
Clairmeta - ERROR - Error parsing XML TurnOffMobAndEnjoy_ADV_F_51_2K_20170918_OV/CPL_f3a1a2ac-d110-4b43-8a25-e638c5dc7e82.xml : out of memory: line 1, column 0

But really, I am very annoyed as I am unable to do any updates or fix the problem until I can somehow get Alpine to regress or a fixed version of the expat appears in their repo.

This is more of an issue with Alpine in terms of my issue, not expat (apart from expat needing to be fixed) But that's why we have version number and the ability to pin versions.

[hartwork]

Hi @sebageek and @jamiegau , thanks for your replies. This also hit Debian today at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1006317 . Some relation to PR #561 is likely but maybe more. I'll have a closer look today.

[hartwork]

Hi @sebageek and @jamiegau, I was able to get to the root of the problem now. You will probably not like what's coming, please take a seat. I'll do a brain dump as bullet points:

For the Expat half ot things:

• The issue is not specific to CPython or Python.

• The issue is the use of namespace separator ':'.

• Error code "out of memory" is mis-leading. It's an internal syntax error from the security fix from PR [CVE-2022-25236] lib: Protect against insertion of namesep characters into namespace URIs #561.

• This C code is enough to reproduce:

  #include <expat.h>
  #include <stdlib.h>
  #include <string.h>
  
  int
  main() {
    const char *const doc = "<foo></foo>";
    XML_Parser parser = XML_ParserCreateNS(NULL, /*bad idea*/ ':');
    const enum XML_Status status
        = XML_Parse(parser, doc, strlen(doc), /*isFinal=*/XML_TRUE);
    XML_ParserFree(parser);
    return (status == XML_STATUS_OK) ? EXIT_SUCCESS : EXIT_FAILURE;
  }

• Within XML_Parse, this call tree happens:

  XML_Parse
      startParsing
          setContext (with context="xml=http://www.w3.org/XML/1998/namespace")
              addBinding
              --> XML_ERROR_SYNTAX  (with >=2.4.5)
          --> XML_FALSE
      --> XML_FALSE
  --> XML_STATUS_ERROR
  

• The API docs of XML_ParserCreateNS say:

  This means that you should pick a character for sep that can't be part of an URI.

  Colon can appear in a URI (RFC 3986), so a colon cannot be used as a namespace separator or there will be false positives (like this one here). Use of " " (space) or "\n" (line feed) would work.

For the Alpine half of things:

• Alpine has only one version per package per branch, in general. So replacing a version of Expat with an update is the only way Alpine has for updates that I'm aware of. I'm trying to say: That seems normal and to be expected.

• Alpine is one of the fastest downstreams with regard to updates. While I do have empathy with your situation, Alpine did nothing wrong here in my book.

• @jamiegau if you need quick help building some flavor of Expat from source, please reach out via e-mail, and we can jump on a call.

Best, Sebastian

[jamiegau]

Thanks @hartwork ..
The a fix is in the current release already? (2.4.6) Over 2.4.5 that exposes the bug?

I have two options, wait for it to flow into Alpine. You mentioned quick but how quick exactly? A few days. A few weeks? I could wait a while.
OR
a snippet of code for a 3.15 Dockerfile that builds the lib would probably get the their. Otherwise I would be googling that out and working on that for half the day if I have to.

I must admit, I am very surprised Alpine does not support regression to older versions. This could break a LOT of containers and those using it will be stuck like I am.

[hartwork]

@jamiegau for a short reply, I think Ymagis/ClairMeta#213 has potential to be among the more quick and more cost-effective options for your environment.

---

For anyone interested, here's a summary of known affected upstreams and links to their status:

biboumi

• Issue: https://lab.louiz.org/louiz/biboumi/-/issues/3465
• Gentoo patch: https://github.com/gentoo/gentoo/blob/master/net-im/biboumi/files/biboumi-9.0-fix-namespace-separator.patch

ClairMeta

• Issue: Error parsing XML "out of memory: line 1, column 0" after updating libexpat Ymagis/ClairMeta#212
• Pull request: Avoid ':' as namespace separator when parsing XML Ymagis/ClairMeta#213

jxmlease

• (not upstream): expat: 2.4.5 -> 2.4.6 NixOS/nixpkgs#161090 (comment)

libwbxml

• Issue: WBXML_NAMESPACE_SEPARATOR ':' incompatible with Expat >=2.4.5 security fixes libwbxml/libwbxml#76
• Pull request: Change namespace separator libwbxml/libwbxml#78

openleadr-python

• Pull request: Expat changes causing issues OpenLEADR/openleadr-python#105

rnv

• (not upstream): build fails due to rnv out of memory TEIC/TEI#2239 (comment)

xmltodict

• Issue: Namespace separator ':' incompatible with Expat >=2.4.5 <2.4.7 security fixes martinblech/xmltodict#289
• Pull request: Switch namespace separator from colon to pipe (fixes #289) martinblech/xmltodict#290

[jamiegau]

Thanks for all the help @hartwork and @remia
The test branch at https://github.com/remia/ClairMeta/tree/fix-namespace-separator rectified my issues by implementing the use of a white space ' ' over a ':' while using the security fixed updated packages.
Hopefully @remia can push it into a 1.2.1 release over the coming days.

Good luck to other projects facing this challenge.

[sebageek]

Hey @hartwork,

Passing a different namespace_separator to the lib that then passes this down to libexpat solves the problem for me and I can now go fix my software. Luckily this parameter is exposed by xmltodict. Thanks for you analysis and help in this!

[hartwork]

@sebageek glad to hear, thanks for the update! 👍