Dynamically accessing attributes of provided objects.

[JarnoRFB]

Hi, I recently needed to access an attribute of a provided object for constructing a new object. A minimal example would look like this

import random

from dependency_injector import containers, providers


class B:
    def __init__(self):
        self.val = random.randint(0, 10)

    def __repr__(self):
        return f"B({self.val})"


class A:
    def __init__(self, b_val: int):
        self.b_val = b_val

    def __repr__(self):
        return f"A({self.b_val})"

naively I tried to do

class MyContainer(containers.DeclarativeContainer):
    b = providers.Singleton(B)
    a = providers.Singleton(A, b.val)


def main() -> None:
    container = MyContainer()
    print(container.b())
    print(container.a())


if __name__ == "__main__":
    main()

which does not work, because of AttributeError: 'dependency_injector.providers.Singleton' object has no attribute 'val'.

I was able to work around this, by using

class MyContainer(containers.DeclarativeContainer):
    b = providers.Singleton(B)
    b_val = providers.Callable(getattr, b, "val")
    a = providers.Singleton(A, b_val)

However, I thought that it would be nice if dependency_injector would provide this behavior out of the box.

My attempts at implementing a provider that has this behavior look like

class AttributeFactory(providers.Provider):

    __slots__ = ('_factory',)

    def __init__(self, *args, **kwargs):
        self._factory = providers.Factory(*args, **kwargs)
        super().__init__()

    def __getattr__(self, item):
        return providers.Callable(getattr, self._factory, item)

class MyContainer(containers.DeclarativeContainer):
    b = AttributeFactory(B)
    a = providers.Singleton(A, b.val)

However, this fails with TypeError: __init__() takes at least 1 positional argument (0 given) because of some copying, that I can not really interpret.

I would be interested in hearing thoughts on this or if there are other ways to achieve this kind of attribute access.

[rmk135]

Hi @JarnoRFB ,

There was a similar request about method calling some time ago #190 (comment)

As you see the last time it ended up with a client code redesigning.

You were very close with creating your own provider. You failed only because of the undocumented behaviour (this issue with copying).

I'm sending to you a patched version of your provider:

class AttributeFactory(providers.Provider):

    __slots__ = ('_factory',)

    def __init__(self, *args, **kwargs):
        self._factory = providers.Factory(*args, **kwargs)
        super().__init__()

    def __deepcopy__(self, memo):
        copy = providers.deepcopy(self._factory, memo)
        return self.__class__(
            copy.provides,
            *copy.args,
            **copy.kwargs,
        )

    def __getattr__(self, item):
        return providers.Callable(getattr, self._factory, item)

    def _provide(self, args, kwargs):
        return self._factory(*args, **kwargs)

There are two points:

1. The copying. When you create a declarative container, the new dynamic container returned instead. This new dynamic container is populated with a copy of all of the declarative container providers. This is done to keep the declarative container in the reference state. It makes impossible to change the declarative container class through the changes in its instances. All the providers need to implement the __deepcopy__() method. See also how factory copying is done. This is not documented anywhere. I'm sorry and apologies. I'll upstream it to the docs.

2. New provider must implement def _provide(self, args, kwargs). In current case it just points everything to the underlaying factory.

[rmk135]

Since this use case is raised second time I would like to get it covered. I've created a little sketch for three different options:

import random

from dependency_injector import containers, providers


class B:
    def __init__(self):
        self.val = random.randint(0, 10)

    def val_method(self):
        return random.randint(0, 10)

    def __repr__(self):
        return f"B({self.val})"


class A:
    def __init__(self, b_val: int, b2_val):
        self.b_val = b_val
        self.b2_val = b2_val

    def __repr__(self):
        return f"A({self.b_val}, {self.b2_val})"


class AttributeGetter(providers.Provider):

    def __init__(self, provider, attribute):
        self._provider = provider
        self._attribute = attribute
        super().__init__()

    @property
    def call(self):
        return MethodCaller(self._provider, self._attribute)

    def __deepcopy__(self, memo=None):
        return self.__class__(providers.deepcopy(self._provider), self._attribute)

    def _provide(self, args, kwargs):
        provided = self._provider(*args, **kwargs)
        return getattr(provided, self._attribute)


class MethodCaller(providers.Provider):

    def __init__(self, provider, method):
        self._provider = provider
        self.method = method
        super().__init__()

    def __deepcopy__(self, memo=None):
        return self.__class__(providers.deepcopy(self._provider), self.method)

    def _provide(self, args, kwargs):
        provided = self._provider(*args, **kwargs)
        method = getattr(provided, self.method)
        return method()


class MyFactory(providers.Factory):

    def __getattr__(self, item):
        return AttributeGetter(self, item)

    def attribute(self, item):
        return AttributeGetter(self, item)

    def method(self, item):
        return MethodCaller(self, item)

    get_attribute = attribute
    call_method = method


class MyContainer(containers.DeclarativeContainer):

    b = MyFactory(B)

    a = providers.Singleton(
        A,
        b.attribute('val'),
        b.method('val_method'),
    )

    a2 = providers.Singleton(
        A,
        AttributeGetter(b, 'val'),
        MethodCaller(b, 'val_method'),
    )

    a3 = providers.Singleton(
        A,
        b.val,
        b.val_method.call,
    )


if __name__ == '__main__':
    container = MyContainer()
    a = container.a()
    print(a)

    a2 = container.a2()
    print(a2)

    a3 = container.a2()
    print(a2)

[rmk135]

I don't yet know which one of this three options is a good one:

class MyContainer(containers.DeclarativeContainer):

    b = MyFactory(B)

    a = providers.Singleton(
        A,
        b.attribute('val'),
        b.method('val_method'),
    )

    a2 = providers.Singleton(
        A,
        AttributeGetter(b, 'val'),
        MethodCaller(b, 'val_method'),
    )

    a3 = providers.Singleton(
        A,
        b.val,
        b.val_method.call,
    )

Dependency Injector has never used any string identifier so # 1 and # 2 is out of common design. Option # 3 might be a solution. The only thing I worry about with # 3 is that there might be name conflicts between provided instance attributes and provider attributes (especially with provider attributes added later).

@JarnoRFB what do you think?

PS: if anybody watching this issue and has a thought, you're welcome to join the conversation

[JarnoRFB]

@rmk135 Thanks for the quick response and the nice illustration!

I agree with you that using strings is unhandy and breaks the look and feel. I also agree that naming conflicts will be problematic both because of evolving dependency_injector and possibly breaking client code.

As a fourth option, I could imaging adding an extra namespace to all providers that gives access to attributes, methods and items. For example pandas does something similar for grouping methods related to str or plot.

provided came to mind as name that makes it clear that we are referring to attributes of the provided instance. Here's a possible example.

import random

from dependency_injector import containers, providers


class B:
    def __init__(self):
        self.val = random.randint(0, 10)
        self.vals = [random.randint(0, 10)]

    def val_method(self):
        return random.randint(0, 10)

    def __repr__(self):
        return f"B({self.val, self.vals})"

    def __getitem__(self, item):
        return self.vals[item]


class A:
    def __init__(self, b_val: int, b2_val, b3_val, b4_val):
        self.b_val = b_val
        self.b2_val = b2_val
        self.b3_val = b3_val
        self.b4_val = b4_val

    def __repr__(self):
        return f"A({self.b_val}, {self.b2_val}, {self.b3_val}, {self.b4_val})"


class AttributeGetter(providers.Provider):

    def __init__(self, provider, attribute):
        self._provider = provider
        self._attribute = attribute
        super().__init__()

    @property
    def call(self):
        return MethodCaller(self._provider, self._attribute)

    def __deepcopy__(self, memo=None):
        return self.__class__(providers.deepcopy(self._provider), self._attribute)

    def _provide(self, args, kwargs):
        provided = self._provider(*args, **kwargs)
        return getattr(provided, self._attribute)

    def __getitem__(self, item):
        return MethodCaller(self, "__getitem__", item)

    def __repr__(self):
        return f"AttributeGetter({self._attribute})"


class MethodCaller(providers.Provider):

    def __init__(self, provider, method, *args, **kwargs):
        self._provider = provider
        self.method = method
        self._args = args
        self._kwargs = kwargs
        super().__init__()

    def __deepcopy__(self, memo=None):
        return self.__class__(providers.deepcopy(self._provider), self.method, *self._args, **self._kwargs)

    def _provide(self, args, kwargs):
        provided = self._provider(*args, **kwargs)
        method = getattr(provided, self.method)
        return method(*self._args, **self._kwargs)


class MySingleton(providers.Singleton):

    @property
    def provided(self):
        return ProvidedAttributes(self)


class ProvidedAttributes(providers.Provider):

    def __init__(self, provider):
        self._provider = provider
        super().__init__()

    def __getattr__(self, item):
        return AttributeGetter(self._provider, item)

    def __getitem__(self, item):
        return MethodCaller(self._provider, "__getitem__", item)


class MyContainer(containers.DeclarativeContainer):
    b = MySingleton(B)

    a4 = providers.Singleton(
        A,
        b.provided.val,
        b.provided.vals[0],
        b.provided.val_method.call(),
        b.provided[0],
    )


if __name__ == '__main__':
    container = MyContainer()
    b = container.b()

    a4 = container.a4()
    print(b)
    print(a4)

Note that I exchanged Factory with Singleton, because I wanted to check if the b instance is reused throughout accessing the different attributes. However, that does not seem to work like that.

[rmk135]

Hey @JarnoRFB ,

That a great design! I'll add this to the providers module.

Note that I exchanged Factory with Singleton, because I wanted to check if the b instance is reused throughout accessing the different attributes. However, that does not seem to work like that.

Yep, I also noticed that yesterday. I think about adding .provided attribute to the Callable, Factory, Singleton and Object providers. They will have the same functionality. The implementation is easy with your design,- that's great as well :)

PS: Some time ago I've finished reading one cool book. It's TRIZ by Genrich Altshuller. TRIZ (or TIPS) is the theory of inventive problem solving. It stands on finding and fighting different contradictions. The typical (or inherent) contradiction sounds like "Software should be complex (to have many features), but simple (to be easy to learn)". The solution you have proposed is free of negative part of the contradictions and includes the positive parts. TRIZ states the "ideal object" as - the object that does not exist, but its function is handled. Your design looks like an "ideal object" (following TRIZ). Very cool! Thank you

[rmk135]

I reproduced the problem with the improper work of the Singleton. Here is patched version of the code:

import random

from dependency_injector import containers, providers


class B:
    def __init__(self):
        self.val = random.randint(0, 10)
        self.vals = [self.val]

    def val_method(self):
        return self.val

    def __repr__(self):
        return f"B({self.val, self.vals})"

    def __getitem__(self, item):
        return self.vals[item]


class A:
    def __init__(self, b_val: int, b2_val, b3_val, b4_val, b):
        self.b_val = b_val
        self.b2_val = b2_val
        self.b3_val = b3_val
        self.b4_val = b4_val
        self.b = b

    def __repr__(self):
        return f"A({self.b_val}, {self.b2_val}, {self.b3_val}, {self.b4_val}, {self.b})"


class AttributeGetter(providers.Provider):

    def __init__(self, provider, attribute):
        self._provider = provider
        self._attribute = attribute
        super().__init__()

    def call(self):
        return MethodCaller(self._provider, self._attribute)

    def __deepcopy__(self, memo=None):
        return self.__class__(providers.deepcopy(self._provider, memo), self._attribute)

    def __getitem__(self, item):
        return MethodCaller(self, "__getitem__", item)

    def __repr__(self):
        return f"AttributeGetter({self._attribute})"

    def _provide(self, args, kwargs):
        provided = self._provider(*args, **kwargs)
        return getattr(provided, self._attribute)


class MethodCaller(providers.Provider):

    def __init__(self, provider, method, *args, **kwargs):
        self._provider = provider
        self.method = method
        self._args = args
        self._kwargs = kwargs
        super().__init__()

    def __deepcopy__(self, memo=None):
        return self.__class__(providers.deepcopy(self._provider, memo), self.method, *self._args, **self._kwargs)

    def _provide(self, args, kwargs):
        provided = self._provider(*args, **kwargs)
        method = getattr(provided, self.method)
        return method(*self._args, **self._kwargs)


class MySingleton(providers.Singleton):

    @property
    def provided(self):
        return ProvidedAttributes(self)


class ProvidedAttributes(providers.Provider):

    def __init__(self, provider):
        self._provider = provider
        super().__init__()

    def __deepcopy__(self, memo=None):
        return self.__class__(providers.deepcopy(self._provider, memo))

    def __getattr__(self, item):
        return AttributeGetter(self._provider, item)

    def __getitem__(self, item):
        return MethodCaller(self._provider, "__getitem__", item)

    def _provide(self, args, kwargs):
        return self._provider(*args, **kwargs)


class MyContainer(containers.DeclarativeContainer):
    b = MySingleton(B)

    a4 = providers.Singleton(
        A,
        b.provided.val,
        b.provided.val_method.call(),
        b.provided.vals[0],
        b.provided[0],
        b.provided,
    )


if __name__ == '__main__':
    container = MyContainer()
    b = container.b()
    print(b)

    a4 = container.a4()
    print(a4)

    assert a4.b is b

[rmk135]

Work is in progress.

I did some changes to make attribute getter, item getter and method caller to work in tandem. It's funny to play with it:

class MyContainer(containers.DeclarativeContainer):
    b = providers.Singleton(B)

    d = providers.Object(
        {
            'a': {
                'b': {
                    'c1': 10,
                    'c2': lambda arg: {'arg': arg}
                },
            },
        },
    )

    a4 = providers.Singleton(
        A,
        b.provided.val,
        b.provided.val_method.call(),
        b.provided.vals[0],
        b.provided[0],
        b.provided,
    )

    l = providers.List(
        d.provided['a']['b']['c1'],
        d.provided['a']['b']['c2'].call(22)['arg'],
        d.provided['a']['b']['c2'].call(a4)['arg'],
        d.provided['a']['b']['c2'].call(a4)['arg'].b_val,
        d.provided['a']['b']['c2'].call(a4)['arg'].get_b2_val.call(),
    )

I have it working. Productizing it will take some time. I would like to get all this well covered with tests.

[JarnoRFB]

@rmk135 Glad I could contribute some idea. I never heard about TRIZ, but it sounds like an interesting read!

I have it working. Productizing it will take some time. I would like to get all this well covered with tests.

Great, let me know if I can contribute something. I believe the feature will make dependency injector extremely flexible, lowering the barrier for people adopting it.

[rmk135]

The feature is published in the 3.31.0.

@JarnoRFB , I've added you to the list of contributors that is distributed with each copy of the Dependency Injector - https://github.com/ets-labs/python-dependency-injector/blob/master/CONTRIBUTORS.rst

I've also updated the changelog at http://python-dependency-injector.ets-labs.org/main/changelog.html stating your design contribution. That's great. Thank you.

There is a docs page for the new feature - http://python-dependency-injector.ets-labs.org/providers/provided_instance.html.

As of contribution, you're very welcome. Look at #282 as an example. Ignore the .c files. As of this feature, it seems complete to me. If you notice any issues, feel free to open a new PR or re-open this issue.

[JarnoRFB]

@rmk135 I am amazed by the speed you implemented this. Thanks a lot!