Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CVE-2018-20843] 88k xml file uses >2G memory #186

Closed
caolanm opened this issue Jan 25, 2018 · 7 comments
Closed

[CVE-2018-20843] 88k xml file uses >2G memory #186

caolanm opened this issue Jan 25, 2018 · 7 comments
Assignees
@hartwork
Labels
bug help wanted security
Milestone
2.2.7

Comments

@caolanm
Copy link

caolanm commented Jan 25, 2018

valgrind --tool=massif xmlwf clusterfuzz-testcase-4543406568112128.txt
reports that xmlwf uses > 2G of memory to load this bogus xml document.

clusterfuzz-testcase-4543406568112128.txt

This was reported by oss-fuzz against LibreOffice (https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=5226) which uses expat and has the same memory use so I felt I should pass it on.
@hartwork
Copy link
Member

Nice! 😄 Thanks for taking this report here.

@RMJ10
Copy link
Contributor

RMJ10 commented Aug 4, 2018 •
edited

I haven't got all the way into this yet, but I suspect the culprit is setElementTypePrefix. It adds each prefix to the DTD string pool as it finds a colon; not each separate element, but the whole prefix from the start of the string to the colon it most recently found. That's the string "DIs", "DIS:BBBBBBB", "DIs:BBBBBBB:includeemBBBBBB" etc. That's going to add up to a lot of space given the long strings of colons in the attribute "name".

@MohammedKhajapasha
Copy link
Contributor

Hi Caolanm,
Are you observing this issue with latest libexpat also? we have tried to reproduce this issue with given command and dummy xml file (valgrind --tool=massif xmlwf clusterfuzz-testcase-4543406568112128.txt) but we didn't observe any error here, could you please let us know if any configuration needed for this ? else could you please close this ticket.

@hartwork
Copy link
Member

hartwork commented May 8, 2019

@MohammedKhajapasha the bug report is legit and still applies to 2.2.6:

# wget https://github.com/libexpat/libexpat/files/1664546/clusterfuzz-testcase-4543406568112128.txt
# time xmlwf clusterfuzz-testcase-4543406568112128.txt & while pgrep xmlwf >/dev/null; do echo $(grep -E '^(VmPeak|VmHWM)' /proc/$(pgrep xmlwf)/status); sleep 0.5 ; done | tail -n1
clusterfuzz-testcase-4543406568112128.txt:1:88403: no element found

real    0m18.761s
user    0m18.099s
sys     0m0.660s
VmPeak: 2301900 kB VmHWM: 2298436 kB

@hartwork hartwork added the bug label May 8, 2019
@MohammedKhajapasha
Copy link
Contributor

MohammedKhajapasha commented Jun 11, 2019 •
edited

As per our initial analysis & Rhodri James comments, the culprit was setElementTypePrefix() which adds all string from start when it founds colon, mBBBBBB:aBBBBQ:::::::::, for every continuous occurrence of colon it adds whole string from start to DTD pool repeatedly which leads increase in VmPeak & VmHWM for application process.

@hartwork hartwork self-assigned this Jun 12, 2019
@MohammedKhajapasha MohammedKhajapasha mentioned this issue Jun 13, 2019
Closed
hartwork added a commit that referenced this issue Jun 13, 2019
@hartwork
xmlparse.c: Fix extraction of namespace prefix from XML name (#186)
11f8838
@hartwork hartwork mentioned this issue Jun 13, 2019
Merged
@hartwork
Copy link
Member

Please note pull request #262 wishing for review. Thanks!

hartwork added a commit that referenced this issue Jun 16, 2019
@hartwork
Changes: Document #186 and #262
ed682ae
@hartwork hartwork added this to the 2.2.7 milestone Jun 16, 2019
@hartwork hartwork mentioned this issue Jun 19, 2019
Closed
23 tasks
@fanquake fanquake mentioned this issue Jun 23, 2019
Merged
hartwork added a commit that referenced this issue Jun 24, 2019
@hartwork
Changes: Attach CVE-2018-20843 to #186 and #262
9c90d3d
@hartwork hartwork changed the title 88k xml file uses >2G memory [CVE-2018-20843] 88k xml file uses >2G memory Jun 24, 2019
@hartwork
Copy link
Member

https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-20843

@jubalh jubalh mentioned this issue Jul 4, 2019
Closed
laanwj added a commit to bitcoin/bitcoin that referenced this issue Jul 10, 2019
@laanwj
Merge #16270: depends: expat 2.2.7
d1fc827 
0512f05 depends: expat 2.2.7 (fanquake)

Pull request description:

  Major changes in expat 2.2.7:

  * [#186](libexpat/libexpat#186) [#262](libexpat/libexpat#262)  Fix extraction of namespace prefixes from XML names;
                      XML names with multiple colons could end up in the
                      wrong namespace, and take a high amount of RAM and CPU
                      resources while processing, opening the door to use for denial-of-service attacks
  * [#227](libexpat/libexpat#227) Autotools: Add --without-examples and --without-tests

  Full changelog is available [here](https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes#L5).

ACKs for top commit:
  laanwj:
    ACK 0512f05

Tree-SHA512: 45162a9b0011107fd59a97dae7b5eb61989dafbec26b1ee497d1b11bf5c6a119971096899caa2998648b82a62db57c629a1560453557146c2496b39a7f3f8de9
@tuliom tuliom mentioned this issue Jul 15, 2019
Closed
Jehops pushed a commit to Jehops/freebsd-ports-legacy that referenced this issue Sep 16, 2019
@opsec
textproc/expat2: upgrade 2.2.6 -> 2.2.7
941fc2d 
- exp-run by antoine

PR:		238864
Submitted by:	Sergei Vyshenski <svysh.fbsd@gmail.com> (maintainer)
Reviewed by:	koobs
Relnotes:	https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes
Security:	libexpat/libexpat#186
		libexpat/libexpat#262


git-svn-id: svn+ssh://svn.freebsd.org/ports/head@512162 35697150-7ecd-e111-bb59-0022644237b5
uqs pushed a commit to freebsd/freebsd-ports that referenced this issue Sep 16, 2019
textproc/expat2: upgrade 2.2.6 -> 2.2.7
03d098b 
- exp-run by antoine

PR:		238864
Submitted by:	Sergei Vyshenski <svysh.fbsd@gmail.com> (maintainer)
Reviewed by:	koobs
Relnotes:	https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes
Security:	libexpat/libexpat#186
		libexpat/libexpat#262


git-svn-id: svn+ssh://svn.freebsd.org/ports/head@512162 35697150-7ecd-e111-bb59-0022644237b5
uqs pushed a commit to freebsd/freebsd-ports that referenced this issue Sep 16, 2019
@opsec
textproc/expat2: upgrade 2.2.6 -> 2.2.7
b3dabdc 
- exp-run by antoine

PR:		238864
Submitted by:	Sergei Vyshenski <svysh.fbsd@gmail.com> (maintainer)
Reviewed by:	koobs
Relnotes:	https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes
Security:	libexpat/libexpat#186
		libexpat/libexpat#262
uqs pushed a commit to freebsd/freebsd-ports that referenced this issue Sep 25, 2019
@delphij
MFH: r512162, r512335
07d7860 
textproc/expat2: upgrade 2.2.6 -> 2.2.7

- exp-run by antoine

PR:		238864
Submitted by:	Sergei Vyshenski <svysh.fbsd@gmail.com> (maintainer)
Reviewed by:	koobs
Relnotes:	https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes
Security:	libexpat/libexpat#186
		libexpat/libexpat#262

textproc/expat2: upgrade 2.2.7 -> 2.2.8

PR:		240613
Submitted by:	Sergei Vyshenski <svysh.fbsd@gmail.com> (maintainer)
Exp-Run by:	antoine
Relnotes:	https://github.com/libexpat/libexpat/blob/R_2_2_8/expat/Changes
Security:	CVE-2019-15903

Approved by:	ports-secteam
monstrobishi pushed a commit to DeFiCh/ain that referenced this issue Sep 6, 2020
@laanwj
Merge #16270: depends: expat 2.2.7
96f76b5 
0512f05 depends: expat 2.2.7 (fanquake)

Pull request description:

  Major changes in expat 2.2.7:

  * [#186](libexpat/libexpat#186) [#262](libexpat/libexpat#262)  Fix extraction of namespace prefixes from XML names;
                      XML names with multiple colons could end up in the
                      wrong namespace, and take a high amount of RAM and CPU
                      resources while processing, opening the door to use for denial-of-service attacks
  * [#227](libexpat/libexpat#227) Autotools: Add --without-examples and --without-tests

  Full changelog is available [here](https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes#L5).

ACKs for top commit:
  laanwj:
    ACK 0512f05

Tree-SHA512: 45162a9b0011107fd59a97dae7b5eb61989dafbec26b1ee497d1b11bf5c6a119971096899caa2998648b82a62db57c629a1560453557146c2496b39a7f3f8de9
uqs pushed a commit to freebsd/freebsd-ports that referenced this issue Apr 1, 2021
@delphij
MFH: r512162, r512335
e7b1563 
textproc/expat2: upgrade 2.2.6 -> 2.2.7

- exp-run by antoine

PR:		238864
Submitted by:	Sergei Vyshenski <svysh.fbsd@gmail.com> (maintainer)
Reviewed by:	koobs
Relnotes:	https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes
Security:	libexpat/libexpat#186
		libexpat/libexpat#262

textproc/expat2: upgrade 2.2.7 -> 2.2.8

PR:		240613
Submitted by:	Sergei Vyshenski <svysh.fbsd@gmail.com> (maintainer)
Exp-Run by:	antoine
Relnotes:	https://github.com/libexpat/libexpat/blob/R_2_2_8/expat/Changes
Security:	CVE-2019-15903

Approved by:	ports-secteam
@github-actions github-actions bot mentioned this issue Jun 3, 2021
Closed
@github-actions github-actions bot mentioned this issue Sep 25, 2021
Open
Munkybooty pushed a commit to Munkybooty/dash that referenced this issue Nov 4, 2021
@laanwj @Munkybooty
Merge bitcoin#16270: depends: expat 2.2.7
6a4af37 
0512f05 depends: expat 2.2.7 (fanquake)

Pull request description:

  Major changes in expat 2.2.7:

  * [dashpay#186](libexpat/libexpat#186) [dashpay#262](libexpat/libexpat#262)  Fix extraction of namespace prefixes from XML names;
                      XML names with multiple colons could end up in the
                      wrong namespace, and take a high amount of RAM and CPU
                      resources while processing, opening the door to use for denial-of-service attacks
  * [dashpay#227](libexpat/libexpat#227) Autotools: Add --without-examples and --without-tests

  Full changelog is available [here](https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes#L5).

ACKs for top commit:
  laanwj:
    ACK 0512f05

Tree-SHA512: 45162a9b0011107fd59a97dae7b5eb61989dafbec26b1ee497d1b11bf5c6a119971096899caa2998648b82a62db57c629a1560453557146c2496b39a7f3f8de9
Munkybooty pushed a commit to Munkybooty/dash that referenced this issue Nov 6, 2021
@laanwj @Munkybooty
Merge bitcoin#16270: depends: expat 2.2.7
51128fd 
0512f05 depends: expat 2.2.7 (fanquake)

Pull request description:

  Major changes in expat 2.2.7:

  * [dashpay#186](libexpat/libexpat#186) [dashpay#262](libexpat/libexpat#262)  Fix extraction of namespace prefixes from XML names;
                      XML names with multiple colons could end up in the
                      wrong namespace, and take a high amount of RAM and CPU
                      resources while processing, opening the door to use for denial-of-service attacks
  * [dashpay#227](libexpat/libexpat#227) Autotools: Add --without-examples and --without-tests

  Full changelog is available [here](https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes#L5).

ACKs for top commit:
  laanwj:
    ACK 0512f05

Tree-SHA512: 45162a9b0011107fd59a97dae7b5eb61989dafbec26b1ee497d1b11bf5c6a119971096899caa2998648b82a62db57c629a1560453557146c2496b39a7f3f8de9
Munkybooty pushed a commit to Munkybooty/dash that referenced this issue Nov 12, 2021
@laanwj @Munkybooty
Merge bitcoin#16270: depends: expat 2.2.7
09b4657 
0512f05 depends: expat 2.2.7 (fanquake)

Pull request description:

  Major changes in expat 2.2.7:

  * [dashpay#186](libexpat/libexpat#186) [dashpay#262](libexpat/libexpat#262)  Fix extraction of namespace prefixes from XML names;
                      XML names with multiple colons could end up in the
                      wrong namespace, and take a high amount of RAM and CPU
                      resources while processing, opening the door to use for denial-of-service attacks
  * [dashpay#227](libexpat/libexpat#227) Autotools: Add --without-examples and --without-tests

  Full changelog is available [here](https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes#L5).

ACKs for top commit:
  laanwj:
    ACK 0512f05

Tree-SHA512: 45162a9b0011107fd59a97dae7b5eb61989dafbec26b1ee497d1b11bf5c6a119971096899caa2998648b82a62db57c629a1560453557146c2496b39a7f3f8de9
Munkybooty pushed a commit to Munkybooty/dash that referenced this issue Nov 16, 2021
@laanwj @Munkybooty
Merge bitcoin#16270: depends: expat 2.2.7
ca3c8da 
0512f05 depends: expat 2.2.7 (fanquake)

Pull request description:

  Major changes in expat 2.2.7:

  * [dashpay#186](libexpat/libexpat#186) [dashpay#262](libexpat/libexpat#262)  Fix extraction of namespace prefixes from XML names;
                      XML names with multiple colons could end up in the
                      wrong namespace, and take a high amount of RAM and CPU
                      resources while processing, opening the door to use for denial-of-service attacks
  * [dashpay#227](libexpat/libexpat#227) Autotools: Add --without-examples and --without-tests

  Full changelog is available [here](https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes#L5).

ACKs for top commit:
  laanwj:
    ACK 0512f05

Tree-SHA512: 45162a9b0011107fd59a97dae7b5eb61989dafbec26b1ee497d1b11bf5c6a119971096899caa2998648b82a62db57c629a1560453557146c2496b39a7f3f8de9
Munkybooty pushed a commit to Munkybooty/dash that referenced this issue Nov 18, 2021
@laanwj @Munkybooty
Merge bitcoin#16270: depends: expat 2.2.7
1e5321a 
0512f05 depends: expat 2.2.7 (fanquake)

Pull request description:

  Major changes in expat 2.2.7:

  * [dashpay#186](libexpat/libexpat#186) [dashpay#262](libexpat/libexpat#262)  Fix extraction of namespace prefixes from XML names;
                      XML names with multiple colons could end up in the
                      wrong namespace, and take a high amount of RAM and CPU
                      resources while processing, opening the door to use for denial-of-service attacks
  * [dashpay#227](libexpat/libexpat#227) Autotools: Add --without-examples and --without-tests

  Full changelog is available [here](https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes#L5).

ACKs for top commit:
  laanwj:
    ACK 0512f05

Tree-SHA512: 45162a9b0011107fd59a97dae7b5eb61989dafbec26b1ee497d1b11bf5c6a119971096899caa2998648b82a62db57c629a1560453557146c2496b39a7f3f8de9
Munkybooty pushed a commit to Munkybooty/dash that referenced this issue Nov 24, 2021
@laanwj @Munkybooty
Merge bitcoin#16270: depends: expat 2.2.7
7feae0c 
0512f05 depends: expat 2.2.7 (fanquake)

Pull request description:

  Major changes in expat 2.2.7:

  * [dashpay#186](libexpat/libexpat#186) [dashpay#262](libexpat/libexpat#262)  Fix extraction of namespace prefixes from XML names;
                      XML names with multiple colons could end up in the
                      wrong namespace, and take a high amount of RAM and CPU
                      resources while processing, opening the door to use for denial-of-service attacks
  * [dashpay#227](libexpat/libexpat#227) Autotools: Add --without-examples and --without-tests

  Full changelog is available [here](https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes#L5).

ACKs for top commit:
  laanwj:
    ACK 0512f05

Tree-SHA512: 45162a9b0011107fd59a97dae7b5eb61989dafbec26b1ee497d1b11bf5c6a119971096899caa2998648b82a62db57c629a1560453557146c2496b39a7f3f8de9
Munkybooty pushed a commit to Munkybooty/dash that referenced this issue Nov 30, 2021
@laanwj @Munkybooty
Merge bitcoin#16270: depends: expat 2.2.7
748f43d 
0512f05 depends: expat 2.2.7 (fanquake)

Pull request description:

  Major changes in expat 2.2.7:

  * [dashpay#186](libexpat/libexpat#186) [dashpay#262](libexpat/libexpat#262)  Fix extraction of namespace prefixes from XML names;
                      XML names with multiple colons could end up in the
                      wrong namespace, and take a high amount of RAM and CPU
                      resources while processing, opening the door to use for denial-of-service attacks
  * [dashpay#227](libexpat/libexpat#227) Autotools: Add --without-examples and --without-tests

  Full changelog is available [here](https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes#L5).

ACKs for top commit:
  laanwj:
    ACK 0512f05

Tree-SHA512: 45162a9b0011107fd59a97dae7b5eb61989dafbec26b1ee497d1b11bf5c6a119971096899caa2998648b82a62db57c629a1560453557146c2496b39a7f3f8de9
Munkybooty pushed a commit to Munkybooty/dash that referenced this issue Nov 30, 2021
@laanwj @Munkybooty
Merge bitcoin#16270: depends: expat 2.2.7
35e8922 
0512f05 depends: expat 2.2.7 (fanquake)

Pull request description:

  Major changes in expat 2.2.7:

  * [dashpay#186](libexpat/libexpat#186) [dashpay#262](libexpat/libexpat#262)  Fix extraction of namespace prefixes from XML names;
                      XML names with multiple colons could end up in the
                      wrong namespace, and take a high amount of RAM and CPU
                      resources while processing, opening the door to use for denial-of-service attacks
  * [dashpay#227](libexpat/libexpat#227) Autotools: Add --without-examples and --without-tests

  Full changelog is available [here](https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes#L5).

ACKs for top commit:
  laanwj:
    ACK 0512f05

Tree-SHA512: 45162a9b0011107fd59a97dae7b5eb61989dafbec26b1ee497d1b11bf5c6a119971096899caa2998648b82a62db57c629a1560453557146c2496b39a7f3f8de9
gades pushed a commit to cosanta/cosanta-core that referenced this issue Apr 20, 2022
@laanwj @gades
Merge bitcoin#16270: depends: expat 2.2.7
5386320 
0512f05 depends: expat 2.2.7 (fanquake)

Pull request description:

  Major changes in expat 2.2.7:

  * [dashpay#186](libexpat/libexpat#186) [dashpay#262](libexpat/libexpat#262)  Fix extraction of namespace prefixes from XML names;
                      XML names with multiple colons could end up in the
                      wrong namespace, and take a high amount of RAM and CPU
                      resources while processing, opening the door to use for denial-of-service attacks
  * [dashpay#227](libexpat/libexpat#227) Autotools: Add --without-examples and --without-tests

  Full changelog is available [here](https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes#L5).

ACKs for top commit:
  laanwj:
    ACK 0512f05

Tree-SHA512: 45162a9b0011107fd59a97dae7b5eb61989dafbec26b1ee497d1b11bf5c6a119971096899caa2998648b82a62db57c629a1560453557146c2496b39a7f3f8de9
@lyricidal lyricidal mentioned this issue Mar 22, 2023
Merged
svmhdvn pushed a commit to svmhdvn/freebsd-ports that referenced this issue Jan 10, 2024
@opsec
textproc/expat2: upgrade 2.2.6 -> 2.2.7
8419628 
- exp-run by antoine

PR:		238864
Submitted by:	Sergei Vyshenski <svysh.fbsd@gmail.com> (maintainer)
Reviewed by:	koobs
Relnotes:	https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes
Security:	libexpat/libexpat#186
		libexpat/libexpat#262
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hartwork hartwork

Labels
bug help wanted security
Projects
None yet
Milestone
2.2.7
Development

No branches or pull requests
4 participants
@caolanm @hartwork @RMJ10 @MohammedKhajapasha