Shallow copy not working correctly for DictConfig

[louismartin]

Describe the bug
It seems that DictConfig objects cannot be copied in python. I am not sure if this is expected behaviour or not.

To Reproduce

from omegaconf import OmegaConf
a = OmegaConf.create({"a": 1, "b": 2})
b = a.copy()
b["a"] = 42
print(b)
print(a)

Outputs:

{'a': 42, 'b': 2}
{'a': 42, 'b': 2}

Simple fix

A simple fix would be to first cast the object to a dict:

from omegaconf import OmegaConf
a = OmegaConf.create({"a": 1, "b": 2})
b = dict(a).copy()
b["a"] = 42
print(b)
print(a)

Outputs:

{'a': 42, 'b': 2}
{'a': 1, 'b': 2}

This however only works for first level attributes, nested DictConfigs won't be casted to dict and will therefore be modified inplace.
For instance:

from omegaconf import OmegaConf
a = OmegaConf.create({"a": {"aa": 1}, "b": 2})
b = dict(a).copy()
b["a"]["aa"] = 42
print(b)
print(a)

Outputs:

{'a': {'aa': 42}, 'b': 2}
{'a': {'aa': 42}, 'b': 2}

This caused a downstream bug in fairseq: facebookresearch/fairseq#3011

[omry]

Hi, this looks identical to the semantics of copy on the standard dict:

In [3]: a = {"a": {"aa": 1}, "b": 2}

In [4]: b = a.copy()

In [5]: b["a"]["aa"] = 42

In [6]: a
Out[6]: {'a': {'aa': 42}, 'b': 2}

In [7]: b
Out[7]: {'a': {'aa': 42}, 'b': 2}

You should probably use copy.deepcopy().

[omry]

Closing as I think it's not a bug, let me know if you disagree.

[louismartin]

Oh yes good point, I totally agree that for the nested DictConfigs it is the same behaviour as standard dictionaries.
Is this wanted, however, that non-nested DictConfigs cannot be copied using the copy operation? (unlike standard dictionaries this time)

[omry]

Can you create a minimal example where the behavior of the DictConfig deviates from that of the standard config?

[louismartin]

I meant that DictConfig does not behave like dictionaries of the standard python library.

Standard dictionaries:

a = {"a": 1, "b": 2}
b = a.copy()
b["a"] = 42
print(b)
print(a)
----- Output -----
{'a': 42, 'b': 2}
{'a': 1, 'b': 2}

DictConfig:

from omegaconf import OmegaConf
a = OmegaConf.create({"a": 1, "b": 2})
b = a.copy()
b["a"] = 42
print(b)
print(a)
----- Output -----
{'a': 42, 'b': 2}
{'a': 42, 'b': 2}

[omry]

Gotcha. definitely looks like a bug.
For now use deepcopy.

[omry]

Once this is fixed, we should consider backporting to the 2.0_branch.

[louismartin]

Perfect thanks!

[omry]

@odelalleau, @pereman2 can one if you take a look?

[pereman2]

@odelalleau, @pereman2 can one if you take a look?

Sure

[omry]

@pereman2, can you backport to 2.0?

[omry]

This is merged to the 2.0 branch. We will probably wait a bit before releasing 2.0.6 to see if there are any other small bugfixes it should include.

[omry]

@louismartin, I am guessing that you also want this in fbcode. Ping me on Workplace to discuss how to get this done (should be easy).