'DSUNet' object has no attribute 'config'

[entangledloops]

I'm unable to get the example script working from here:
https://github.com/microsoft/DeepSpeed-MII/blob/main/examples/local/txt2img-example.py

When I run without arguments it loads the model and deploys okay.
But then using the --query produces this:

ERROR:grpc._server:Exception calling application: 'DSUNet' object has no attribute 'config'
Traceback (most recent call last):
  File "/home/snd/bin/miniconda3/envs/ldm/lib/python3.8/site-packages/grpc/_server.py", line 443, in _call_behavior
    response_or_iterator = behavior(argument, context)
  File "/home/snd/bin/miniconda3/envs/ldm/lib/python3.8/site-packages/mii/grpc_related/modelresponse_server.py", line 77, in Txt2ImgReply
    response = self.inference_pipeline(request, **query_kwargs)
  File "/home/snd/bin/miniconda3/envs/ldm/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/snd/bin/miniconda3/envs/ldm/lib/python3.8/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py", line 504, in __call__
    height = height or self.unet.config.sample_size * self.vae_scale_factor
  File "/home/snd/bin/miniconda3/envs/ldm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1265, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'DSUNet' object has no attribute 'config'
Traceback (most recent call last):
  File "deploy.py", line 52, in <module>
    result = generator.query({
  File "/home/snd/bin/miniconda3/envs/ldm/lib/python3.8/site-packages/mii/server_client.py", line 367, in query
    response = self.asyncio_loop.run_until_complete(
  File "/home/snd/bin/miniconda3/envs/ldm/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/snd/bin/miniconda3/envs/ldm/lib/python3.8/site-packages/mii/server_client.py", line 263, in _query_in_tensor_parallel
    await responses[0]
  File "/home/snd/bin/miniconda3/envs/ldm/lib/python3.8/site-packages/mii/server_client.py", line 313, in _request_async_response
    response = await self.stubs[stub_id].Txt2ImgReply(req)
  File "/home/snd/bin/miniconda3/envs/ldm/lib/python3.8/site-packages/grpc/aio/_call.py", line 290, in __await__
    raise _create_rpc_error(self._cython_call._initial_metadata,
grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with:
        status = StatusCode.UNKNOWN
        details = "Exception calling application: 'DSUNet' object has no attribute 'config'"
        debug_error_string = "UNKNOWN:Error received from peer ipv6:%5B::1%5D:50050 {created_time:"2022-11-28T15:19:01.639187607-08:00", grpc_status:2, grpc_message:"Exception calling application: \'DSUNet\' object has no attribute \'config\'"}"

Versions:

deepspeed                     0.7.5
deepspeed-mii                 0.0.3
transformers                  4.24.0

[eran-sefirot]

+1, I'm using EC2 g5.4xlarge (64GB RAM)
Deep learning AMI with pytorch 1.12.1 and CUDA 11.6

I am able to run the baseline script successfully

[mrwyattii]

This was resolved in microsoft/DeepSpeed#2536

Just tested and it's working. Please update DeepSpeed to 0.7.6 and installing deepspeed-mii from source or install version 0.0.4 when we release it later today.

[ghunkins]

Installing 0.7.6 and 0.0.4 did not work for me.

Required the above PR to fix, installing deepseed from source fixes this. Just waiting on PYPI release.

PR: microsoft/DeepSpeed#2589

pip install git+https://github.com/microsoft/DeepSpeed.git

Then via ds_report:

deepspeed info ................... 0.7.7+2076bf23, 2076bf23, master

After that, all worked!