Hierarchical instantiation

[le1nux]

Implements hierarchical instantiation of components via dependency injection.

Given a config defined in a YAML file, we first load the config into a dictionary within the factory. We then traverse the dictionary recursively in depth-first fashion. Once we reach a leave component, we first instantiate the respective component config via Pydantic. Since the component config is self-contained, we leverage the referenced class type in the type_hint to instantiate the component. We then return upwards to the parent item in the dictionary. If there are other child components in the parent component, then we also instantiate those. Once, all dependencies of the parent are instantiated, we can also instantiate the parent config and then the parent component.

We traverse the entire config in this fashion by first instantiation the component configs and then the components themselves. Once all the components have been built, we return the root components whose keys we passed to the factory.

This instantiation has a couple of advantages.

1. we can define arbitrary configs and component dependencies, since pedantic is only applied to single components in isolation. Before we were limited by the Appconfig definition.
2. Registry is not necessary anymore. We only need to pass the ConfigTypes to the Factory.
3. Components in the config specify the dependent components with their actual types now. Before we had dependencies as config types , e.g., DataloaderConfig would have had a dependency for DatasetConfig. Now, the dependency can be either a concrete class e.g., Dataset or even more generic an Interface DatasetIF. In fact, the possibility for interface type checks makes sure that that the dependencies always implement the required interface and would throw an error otherwise.

As of right now I don't see any disadvantage ... but would be great to discuss potential issues as well.

[fromm-m]

approved

[dertilo]

Hi modalities-people!
Can you help me understand the differences/advantages of your hierarchical instantiation approach compared to other open-source frameworks like hydra or pydantic?