[Feature Request] Consider supporting singleton pattern in recursive instantiation

[omry]

The following config can in theory be used to instantiate one object

container:
  _target_: ...
  object1:
    _target_: ...
    _singleton_: true

  object2: ${.object1}

c = instantiate(cfg.container)
assert c.object1 is c.object2

This will will probably require changes in OmegaConf.

[rsokl]

This would be very powerful. It would effectively enable Hydra to be used for dependency injection (but orchestrated via configs).

[aliutkus]

As a note, I would like this to work:

misc:
  a:
     _target_: mymodule.A
     _singleton_: true
     foo: 1

b: 
   _target_: mymodule.B
   bar: 2
   a: ${misc.a}

c:
   _target_: mymodule.C
   baz: 3
   a: ${misc.a}

i.e., the reference would not need to happen inside the same object to be instantiated. This may be very useful for cases where the same object (say some datapipeline) is to be used by models or other things, while keeping some structure to the config

[Ubadub]

Hi, I noticed that this was initially part of the Hydra 1.3.0 milestone but was removed. It seems like a very useful feature to have. Are there any updates on when we can expect this feature?

[Jasha10]

I'll comment to record one potential issue I see with the proposed API.

It looks like calling resolve before instantiate will result in object1 and object2 being different objects.

from omegaconf import OmegaConf

cfgA = OmegaConf.create("""
object1:
 _target_: MyTarget
 _singleton_: true
object2: ${object1}
""")

cfgB = cfgA.copy()
OmegaConf.resolve(cfgB)

assert OmegaConf.to_yaml(cfgB) == """\
object1:
  _target_: MyTarget
  _singleton_: true
object2:
  _target_: MyTarget
  _singleton_: true
"""

Concretely, how would Hydra know that instantiate(cfgB) should have object1 and object2 be the same? Most operations in Hydra/OmegaConf respect the invariant that operating on a resolved config should give the same results as operating on an unresolved config.

[yuvalkirstain]

@Jasha10 thank you for the response. Just to stress out the issue, in most image generation/video generation projects we instantiate objects that use GPU and other expensive resources, and use them multiple times. Without this feature we need to manually pass these objects. This workaround breaks the clean flow that hydra usually enables 😢

[marios1861]

Is there any progress on this? The lack of this feature breaks the free config-based pattern that hydra enables for deep learning applications, as already stated by @yuvalkirstain.