Cannot load pickle file when parameters are passed through Provide

[Balthus1989]

Hello,
I have a class that is in charge to load some pickle files for a "machine learning" purpose.

Specifically, I call my class, called Regressor, in this way, through the usage of a .ini file reporting all the desired settings:

from dependency_injector import containers, providers
from dependency_injector.wiring import inject, Provide
from src.service.costestimator.configuration.configuration import Container

class Regressor():
    def __init__(self,
                 regressors_list = Provide[Container.config.regressors_scalers.regressors_list.as_(str)],
                 regressors_path = Provide[Container.config.regressors_scalers.regressors_path.as_(str)],
                 scalers_path = Provide[Container.config.regressors_scalers.scalers_path.as_(str)],
                 file_type_regressors = Provide[Container.config.regressors_scalers.file_type_regressors.as_(str)],
                 file_type_scalers = Provide[Container.config.regressors_scalers.file_type_scalers.as_(str)]):

When it comes to the "core" part of the code, it fails to load the pickle file:

    def load_model(self):
        if self.regressors_out is None:
            resulting_reg = {}
            resulting_scal = {}
            
            for regressor in self.regressors_list:
                with open(self.regressors_path.joinpath(self.regressors_path, f'{regressor}.{self.file_type_regressors}'), 'rb') as reg:
                    resulting_reg[regressor] = pickle.load(reg)

The error I get is the following:
ImportError: cannot import name '_ccallback_c' from 'scipy._lib'

The funny part is that: if I instantiate my Regressor class using the "Factory approach", it perfectly works.

Any idea? Thank you in advance for any response

[rmk135]

Hi @Balthus1989! That's interesting. I'll take a look.

[rmk135]

I created a naive example and tried if it works with pickle:

import pickle
import sys

from dependency_injector import containers, providers
from dependency_injector.wiring import inject, Provide


class Service:
    ...


class Container(containers.DeclarativeContainer):

    service = providers.Factory(Service)


class Main:

    @inject
    def __init__(self, service: Service = Provide[Container.service]):
        self.service = service

    def main(self):
        print(self.service)


if __name__ == '__main__':
    container = Container()
    container.wire(modules=[sys.modules[__name__]])

    main = Main()
    main.main()
    # <__main__.Service object at 0x103ba21f0>

    pickled_main = pickle.dumps(main)
    print(pickled_main)
    # b'\x80\x04\x957\x00\x00\x00\x00\x00\x00\x00\x8c\x08__main__\x94\x8c\x04Main\x94\x93\x94)\x81\x94}\x94\x8c\x07service\x94h\x00\x8c\x07Service\x94\x93\x94)\x81\x94sb.'

    unpickled_main = pickle.loads(pickled_main)
    unpickled_main.main()
    # <__main__.Service object at 0x103d745e0>

I see no problems so far.

---

I google for this error ImportError: cannot import name '_ccallback_c' from 'scipy._lib'. People recommend to reinstall scipy: pip install --upgrade scipy.

Also if you have things working with factory, try to follow the difference in imports in working or non-working solutions.

[Balthus1989]

I googled the error, but reinstalling scipy didn't work for me.

A clarification: I use a Container which is dinamically imported and shared among several classes. My code was reported in a previous issues, more precisely here: #365

[Balthus1989]

Another important detail: the "working method" uses the following Container class

from dependency_injector import containers, providers
from src.service.costestimator.costestimator import costestimator
from src.models.regressors.regressor import Regressor

class Container(containers.DeclarativeContainer):
    config = providers.Configuration()

    costestimator_factory = providers.Factory(
        costestimator,
        customer_name = config.customer.customer_name.as_(str),
        customer_plant = config.customer.customer_plant.as_(str),
        quotations_key = config.customer.quotations_key.as_(str),
        machines_key = config.customer.machines_key.as_(str),
        list_of_features = config.regressors_scalers.list_of_features.as_(str),
        regressors_list = config.regressors_scalers.regressors_list.as_(str),
        machine_feature_column = config.scripts.machine_feature_column.as_(str),
        scripts_path = config.scripts.path.as_(str)
    )
    
    regressor_factory = providers.Factory(
        Regressor,
        regressors_list = config.regressors_scalers.regressors_list.as_(str),
        regressors_path = config.regressors_scalers.regressors_path.as_(str),
        scalers_path = config.regressors_scalers.scalers_path.as_(str),
        file_type_regressors = config.regressors_scalers.file_type_regressors.as_(str),
        file_type_scalers = config.regressors_scalers.file_type_scalers.as_(str)
    )

Thus, each time I invoke these classes in this way (e.g.):

regressor_factory = Container.regressor_factory()

[rmk135]

I can't reproduce it.

[rmk135]

Could you make a mini-project to reproduce an error?

[rmk135]

I think I've got something. In my case I have an error with numpy: ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'.

If I have a module that imports numpy and I use register_loader_containers(), importing of such module cases an error.

[Balthus1989]

I think I've got something. In my case I have an error with numpy: ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'.

If I have a module that imports numpy and I use register_loader_containers(), importing of such module cases an error.

Probably it is implied with the error that I've reported.

If you need a mini-project, just let me know. I will provide it to you tomorrow :-)

[rmk135]

I think I got it fixed. I'll publish a hotfix release in an hour. It will fix import errors for both numpy and scipy.

[rmk135]

I published the fix in version 4.29.2. Try to upgrade and and see if error goes away.

[Balthus1989]

Ehi, now everything works.

I'm much obliged, thank you!!!

[rmk135]

No problems, I'm glad everything is working! I'm much more obligated to you since your feedback helped to improve the dependency injector. Thank you!