Poetry relies on tomlkit `0.11.{2/3}` which has a memory leak #6144

TNonet · 2022-08-09T16:09:21Z

I am on the latest Poetry version.
I have searched the issues of this repo and believe that this is not a duplicate.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

OS version and name: >>>cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Poetry version: Multiple
Link of a Gist with the contents of your pyproject.toml file:

Issue

TL;DR: poetry export now seems to use 5x+ more memory than it used to.

In a CircleCI workflow, we run poetry export. This has worked for months, but sometime between Friday (8/5/2022) and Monday (8/8/2022) morning, something changed and caused poetry export to use significantly more memory.

Using this cimg/python:3.9 docker image which has poetry 1.1.13 installed, we install poetry 1.1.14 using curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python3 - --version 1.1.14.

Then running poetry export produces the following allocations (using https://github.com/bloomberg/memray):

⠋ Calculating high watermark... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   Memray ERROR: Failed to process memory record

Allocation metadata
-------------------
Command line arguments: '/home/circleci/.local/share/pypoetry/venv/lib/python3.9/site-packages/memray/__main__.py run -o output.bin /home/circleci/.local/bin/poetry export'
Peak memory size: 1.878GB
Number of allocations: 55887

Biggest 10 allocations:
-----------------------
📂 1.078GB (100.00 %) <ROOT>
└── [[20 frames hidden in 14 file(s)]]
    └── 📂 1.078GB (100.00 %) <listcomp>  /home/circleci/.local/share/pypoetry/venv/lib/python3.9/site-packages/tomlkit/source.py:71
        ├── [[2 frames hidden in 2 file(s)]]
        │   └── 📄 86.750MB (7.86 %) update_wrapper  /home/circleci/.pyenv/versions/3.9.13/lib/python3.9/functools.py:56
        ├── 📂 219.125MB (19.86 %) __init__  /home/circleci/.local/share/pypoetry/venv/lib/python3.9/site-packages/tomlkit/toml_char.py:9
        │   ├── 📄 128.375MB (11.63 %) decorating_function  /home/circleci/.pyenv/versions/3.9.13/lib/python3.9/functools.py:519
        │   └── [[1 frames hidden in 1 file(s)]]
        │       └── 📄 90.750MB (8.22 %) update_wrapper  /home/circleci/.pyenv/versions/3.9.13/lib/python3.9/functools.py:56
        ├── 📂 215.750MB (19.55 %) __init__  /home/circleci/.local/share/pypoetry/venv/lib/python3.9/site-packages/tomlkit/toml_char.py:11
        │   ├── 📄 122.500MB (11.10 %) decorating_function  /home/circleci/.pyenv/versions/3.9.13/lib/python3.9/functools.py:519
        │   └── [[1 frames hidden in 1 file(s)]]
        │       └── 📄 93.250MB (8.45 %) update_wrapper  /home/circleci/.pyenv/versions/3.9.13/lib/python3.9/functools.py:56
        ├── 📂 212.250MB (19.24 %) __init__  /home/circleci/.local/share/pypoetry/venv/lib/python3.9/site-packages/tomlkit/toml_char.py:13
        │   ├── 📄 120.500MB (10.92 %) decorating_function  /home/circleci/.pyenv/versions/3.9.13/lib/python3.9/functools.py:519
        │   └── [[1 frames hidden in 1 file(s)]]
        │       └── 📄 91.750MB (8.32 %) update_wrapper  /home/circleci/.pyenv/versions/3.9.13/lib/python3.9/functools.py:56
        ├── [[2 frames hidden in 2 file(s)]]
        │   └── 📄 97.500MB (8.84 %) update_wrapper  /home/circleci/.pyenv/versions/3.9.13/lib/python3.9/functools.py:56
        └── [[2 frames hidden in 2 file(s)]]
            └── 📄 86.750MB (7.86 %) update_wrapper  /home/circleci/.pyenv/versions/3.9.13/lib/python3.9/functools.py:56

Workaround attempts:

Running the same workflow without installing poetry 1.1.13 runs fine but suffers from Poetry not passing through SHAs for certain package versions. #5967.
Updating poetry using poetry update self results in the same OOM.
Uninstalling poetry using the info from this issue, Cannot Uninstall Poetry on #644, does not work. Therefore, I can't test uninstalling 1.1.13 and installing 1.1.14.

Running the same workflow on this cimg/python:3.10 docker image which has poetry 1.1.14 runs which much smaller allocation:

Allocation metadata
-------------------
Command line arguments: '/home/circleci/.local/share/pypoetry/venv/lib/python3.10/site-packages/memray/__main__.py run -o output.bin /home/circleci/.local/bin/poetry export'
Peak memory size: 209.743MB
Number of allocations: 488785

Biggest 10 allocations:
-----------------------
📂 170.559MB (100.00 %) <ROOT>
└── [[3 frames hidden in 2 file(s)]]
    └── 📂 170.559MB (100.00 %) _run_code  /home/circleci/.pyenv/versions/3.10.6/lib/python3.10/runpy.py:86
        ├── [[5 frames hidden in 4 file(s)]]
        │   └── 📂 169.304MB (99.26 %) wrap_handle  /home/circleci/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/commands/command.py:92
        │       ├── [[2 frames hidden in 2 file(s)]]
        │       │   └── 📂 141.291MB (82.84 %) read  /home/circleci/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/core/toml/file.py:32
        │       │       ├── [[2 frames hidden in 2 file(s)]]
        │       │       │   └── 📂 140.117MB (82.15 %) parse  /home/circleci/.local/share/pypoetry/venv/lib/python3.10/site-packages/tomlkit/api.py:82
        │       │       │       ├── [[3 frames hidden in 2 file(s)]]
        │       │       │       │   └── 📄 36.000MB (21.11 %) __init__  /home/circleci/.local/share/pypoetry/venv/lib/python3.10/site-packages/tomlkit/toml_char.py:8
        │       │       │       └── [[10 frames hidden in 2 file(s)]]
        │       │       │           └── 📄 14.087MB (8.26 %) _to_linecol  /home/circleci/.local/share/pypoetry/venv/lib/python3.10/site-packages/tomlkit/source.py:179
        │       │       └── [[1 frames hidden in 1 file(s)]]
        │       │           └── 📄 1.173MB (0.69 %) decode  /home/circleci/.pyenv/versions/3.10.6/lib/python3.10/codecs.py:322
        │       ├── [[7 frames hidden in 5 file(s)]]
        │       │   └── 📂 25.014MB (14.67 %) parse  /home/circleci/.local/share/pypoetry/venv/lib/python3.10/site-packages/tomlkit/api.py:82
        │       │       ├── 📂 16.000MB (9.38 %) parse  /home/circleci/.local/share/pypoetry/venv/lib/python3.10/site-packages/tomlkit/parser.py:157
        │       │       │   ├── [[9 frames hidden in 2 file(s)]]
        │       │       │   │   └── 📄 14.000MB (8.21 %) _to_linecol  /home/circleci/.local/share/pypoetry/venv/lib/python3.10/site-packages/tomlkit/source.py:179
        │       │       │   └── [[6 frames hidden in 3 file(s)]]
        │       │       │       └── 📄 2.000MB (1.17 %) <genexpr>  /home/circleci/.local/share/pypoetry/venv/lib/python3.10/site-packages/tomlkit/items.py:412
        │       │       └── [[3 frames hidden in 2 file(s)]]
        │       │           └── 📄 5.000MB (2.93 %) __init__  /home/circleci/.local/share/pypoetry/venv/lib/python3.10/site-packages/tomlkit/toml_char.py:8
        │       └── [[13 frames hidden in 11 file(s)]]
        │           └── 📄 3.000MB (1.76 %) <listcomp>  /home/circleci/.local/share/pypoetry/venv/lib/python3.10/site-packages/tomlkit/source.py:71
        └── [[75 frames hidden in 14 file(s)]]
            └── 📄 1.255MB (0.74 %) _compile_bytecode  <frozen importlib._bootstrap_external>:672

The text was updated successfully, but these errors were encountered:

radoering · 2022-08-09T16:27:26Z

Can you check if it's related to the tomlkit version?

TNonet · 2022-08-09T20:20:45Z

Hi @radoering,

Thanks for the response! I will take a look at this soon.

dsalisbury · 2022-08-10T01:08:06Z

Hey hey!

Another data point: this is triggering in bitbucket pipelines with a step that has a 1GB memory limit for the docker service.

I can also recreate it reliably locally using plain docker commands. I made a repo here with a sample pyproject.toml, lockfile, and test harness to run things https://github.com/dsalisbury/poetry-6144

Watch along with docker stats and you'll see the container go up to liek 1023MiB then get killed

In my testing, using tomlkit==0.11.1 (the previous version) keeps the memory utilisation at a typical level

dsalisbury · 2022-08-10T02:11:11Z

I noticed that there's a new bugfix release of tomlkit (0.11.3) about 30 mins ago but it hasn't made any difference here

radoering · 2022-08-10T04:31:40Z

If it's a regression in tomlkit, you should probably open an issue at https://github.com/sdispater/tomlkit/issues.

TNonet · 2022-08-10T14:49:55Z

Downgrading tomlkit (0.11.3) to 0.11.1 seems to alleviate the issue! They should release a new version soon!

Can we exclude tomlkit 0.11.{2/3} from the valid tomlkit versions for poetry?

Secrus · 2022-08-11T07:09:56Z

Downgrading tomlkit (0.11.3) to 0.11.1 seems to alleviate the issue! They should release a new version soon!

Can we exclude tomlkit 0.11.{2/3} from the valid tomlkit versions for poetry?

Theoretically yes, but it would only work on the master branch version since I doubt this would be enough for another release to be made.

TNonet · 2022-08-13T14:25:40Z

This is now fixed with tomlkit https://github.com/sdispater/tomlkit/releases/tag/0.11.4.

Downgrading tomlkit (0.11.3) to 0.11.1 seems to alleviate the issue! They should release a new version soon!
Can we exclude tomlkit 0.11.{2/3} from the valid tomlkit versions for poetry?

Theoretically yes, but it would only work on the master branch version since I doubt this would be enough for another release to be made.

While it may not be enough for another release, it could go out with the next.

radoering · 2022-08-13T15:51:32Z

I agree that we should exclude tomlkit 0.11.2 and 0.11.3. Just created a PR.

github-actions · 2024-03-01T14:06:05Z

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

TNonet added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Aug 9, 2022

dsalisbury mentioned this issue Aug 10, 2022

Some sort of memory leak or just significant memory growth in 0.11.2 manifesting in Python Poetry python-poetry/tomlkit#225

Closed

TNonet changed the title ~~Possible memory leak on CircleCI~~ Poetry relies on tomlkit 0.11.{2/3} which has a memory leak Aug 10, 2022

radoering mentioned this issue Aug 13, 2022

avoid tomlkit versions with memory leak #6160

Merged

radoering closed this as completed in #6160 Aug 13, 2022

mkniewallner removed the status/triage This issue needs to be triaged label Aug 19, 2022

github-actions bot locked as resolved and limited conversation to collaborators Mar 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Poetry relies on tomlkit `0.11.{2/3}` which has a memory leak #6144

Poetry relies on tomlkit `0.11.{2/3}` which has a memory leak #6144

TNonet commented Aug 9, 2022 •

edited

radoering commented Aug 9, 2022

TNonet commented Aug 9, 2022

dsalisbury commented Aug 10, 2022 •

edited

dsalisbury commented Aug 10, 2022 •

edited

radoering commented Aug 10, 2022

TNonet commented Aug 10, 2022

Secrus commented Aug 11, 2022

TNonet commented Aug 13, 2022

radoering commented Aug 13, 2022

github-actions bot commented Mar 1, 2024

Poetry relies on tomlkit 0.11.{2/3} which has a memory leak #6144

Poetry relies on tomlkit 0.11.{2/3} which has a memory leak #6144

Comments

TNonet commented Aug 9, 2022 • edited

Issue

radoering commented Aug 9, 2022

TNonet commented Aug 9, 2022

dsalisbury commented Aug 10, 2022 • edited

dsalisbury commented Aug 10, 2022 • edited

radoering commented Aug 10, 2022

TNonet commented Aug 10, 2022

Secrus commented Aug 11, 2022

TNonet commented Aug 13, 2022

radoering commented Aug 13, 2022

github-actions bot commented Mar 1, 2024

Poetry relies on tomlkit `0.11.{2/3}` which has a memory leak #6144

Poetry relies on tomlkit `0.11.{2/3}` which has a memory leak #6144

TNonet commented Aug 9, 2022 •

edited

dsalisbury commented Aug 10, 2022 •

edited

dsalisbury commented Aug 10, 2022 •

edited