Incorrect table entry removal

[kivicode]

TOMLKit is failing to pop a table key in some very strange circumstances.

Below is a failing example:

import tomlkit

test_toml = """
[tool.poetry]
version = "0.3.0"

[tool.uv.sources]
torch = { index = "pytorch-gpu" }
unsloth = { git = "https://github.com/unslothai/unsloth.git" }

[tool.poetry.group.dev.dependencies]
pytest = "^7.2.2"
pytest-cov = "^4.0.0"
black = "^23.1.0"
mypy = "^1.2.0"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[[tool.poetry.source]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
priority = "explicit"
"""

parsed = tomlkit.loads(test_toml)
parsed["tool"].pop("poetry")

Traceback:

Traceback (most recent call last):
  File "test.py", line 28, in <module>
    parsed["tool"].pop("poetry")
  File "/[...]/python3.8/_collections_abc.py", line 801, in pop
    del self[key]
  File "/[...]/python3.8/site-packages/tomlkit/container.py", line 868, in __delitem__
    del table[key]
  File "/[...]/python3.8/site-packages/tomlkit/items.py", line 1502, in __delitem__
    self.remove(key)
  File "/[...]/python3.8/site-packages/tomlkit/items.py", line 1472, in remove
    self._value.remove(key)
  File "/[...]/python3.8/site-packages/tomlkit/container.py", line 352, in remove
    raise NonExistentKey(key)
tomlkit.exceptions.NonExistentKey: 'Key "poetry" does not exist.'

What's strange is that if I comment out any section from the above example - it works just fine.

---

Python: 3.8.20
tomlkit: 0.13.2

[aslowwriter]

We were also struggling with this issue and in presuit of it, we did some debugging and at the very least came up with a minimal example (toml and python)

[input]
a = "b"

[state.variables]
c = "d"

[input.foo]
bar = "baz"

import tomlkit

with open("MWE.toml") as fp:
    doc = tomlkit.load(fp)

inp = doc["input"]
static = inp.setdefault("static", {})

static["asdf"] = "slope"

# uncomment to get assertion error
# assert "asdf" in doc["input"]["static"]

static = doc["input"]["static"]["asdf"] = "slope"
assert "asdf" in doc["input"]["static"]

We suspect that it comes down to a mismatch between self._internal_containter and self._tables. Debugging the example above and stepping into line 8 gives this:

> /default/lib/python3.10/site-packages/tomlkit/container.py(834)__getitem__()
-> return self._internal_container[key]
(Pdb) ll
830         def __getitem__(self, key: Key | str) -> Any:
831             if key not in self._internal_container:
832                 raise NonExistentKey(key)
833
834  ->         return self._internal_container[key]
(Pdb) self._internal_container
{'a': 'b', 'foo': {'bar': 'baz'}, 'static': {}}
(Pdb) self._tables
[{'a': 'b', 'static': {}}, {'foo': {'bar': 'baz'}}]
(Pdb) id(self._tables[0]['static']) == id(self._internal_container['static'])
False
(Pdb) type(self._tables[0]['static']) == type(self._internal_container['static'])
True
(Pdb) type(self._tables[0]['static'])
<class 'tomlkit.items.Table'>

when later stepping into 12 gives this:
12 eventually gives this gives this:

> tomlkit/container.py(830)__getitem__()
-> def __getitem__(self, key: Key | str) -> Any:
(Pdb) self._internal_container
{'a': 'b', 'static': {}, 'foo': {'bar': 'baz'}}
(Pdb) self._tables
[{'a': 'b', 'static': {}}, {'foo': {'bar': 'baz'}}]
(Pdb) self._tables_map
{<Key a >: [0], None: [0], <Key static>: [0], <Key foo>: [1]}
(Pdb)

We susupect it is because in the __setitem__ of the table contianer a deepcopy is made, which would maybe then be returned later instead of a shallow copy. (

tomlkit/tomlkit/container.py

Line 250 in 1a3085c

current = copy.deepcopy(current)

)

Unfortunately I don't have the time to investigate this further and we have since found a work around for our case, so I'll be leaving it off here, but I wanted to add the context for the next person to look into it at least.

[frostming]

It's a duplicate of #383 and fixed by #420