FileNotFoundError: [Errno 2] No such file or directory: 'deepspeed'

[C00reNUT]

Hello,

When running the following code I get the FileNotFoundError Error.

Any idea why this happens? I follow the usual install through conda (pytorch+cuda) and pip install .

mii_configs = {"tensor_parallel": 1, "dtype": "fp16"}
mii.deploy(task="text-generation",
           model="gpt2",
           deployment_name="gpt2_deployment",
           mii_config=mii_configs)

[2022-08-25 12:41:19,489] [INFO] [deployment.py:74:deploy] *************DeepSpeed Optimizations: True*************
[2022-08-25 12:41:19,524] [INFO] [server_client.py:206:_initialize_service] multi-gpu deepspeed launch: ['deepspeed', '--num_gpus', '1', '--no_local_rank', '--no_python', '/mnt/2287294e-32c7-437b-84bd-452a29548b1a/conda_env/DeepSpeedInterface/bin/python', '-m', 'mii.launch.multi_gpu_server', '--task-name', 'text-generation', '--model', 'gpt2', '--model-path', '/tmp/mii_models', '--port', '50050', '--ds-optimize', '--provider', 'hugging-face', '--config', 'eyJ0ZW5zb3JfcGFyYWxsZWwiOiAxLCAicG9ydF9udW1iZXIiOiA1MDA1MCwgImR0eXBlIjogImZwMTYiLCAiZW5hYmxlX2N1ZGFfZ3JhcGgiOiBmYWxzZSwgImNoZWNrcG9pbnRfZGljdCI6IG51bGx9']

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Input In [2], in <cell line: 2>()
      1 mii_configs = {"tensor_parallel": 1, "dtype": "fp16"}
----> 2 mii.deploy(task="text-generation",
      3            model="gpt2",
      4            deployment_name="gpt2_deployment",
      5            mii_config=mii_configs)

File /mnt/2287294e-32c7-437b-84bd-452a29548b1a/NLP/Text_Generation/text_generator/DeepSpeed-MII/mii/deployment.py:94, in deploy(task, model, deployment_name, deployment_type, model_path, enable_deepspeed, enable_zero, ds_config, mii_config)
     92     print(f"Score file created at {generated_score_path(deployment_name)}")
     93 elif deployment_type == DeploymentType.LOCAL:
---> 94     return _deploy_local(deployment_name, model_path=model_path)
     95 else:
     96     raise Exception(f"Unknown deployment type: {deployment_type}")

File /mnt/2287294e-32c7-437b-84bd-452a29548b1a/NLP/Text_Generation/text_generator/DeepSpeed-MII/mii/deployment.py:100, in _deploy_local(deployment_name, model_path)
     99 def _deploy_local(deployment_name, model_path):
--> 100     mii.utils.import_score_file(deployment_name).init()

File /tmp/mii_cache/gpt2_deployment/score.py:29, in init()
     26 assert task is not None, "The task name should be set before calling init"
     28 global model
---> 29 model = mii.MIIServerClient(task,
     30                             model_name,
     31                             model_path,
     32                             ds_optimize=configs[mii.constants.ENABLE_DEEPSPEED_KEY],
     33                             ds_zero=configs[mii.constants.ENABLE_DEEPSPEED_ZERO_KEY],
     34                             ds_config=configs[mii.constants.DEEPSPEED_CONFIG_KEY],
     35                             mii_configs=configs[mii.constants.MII_CONFIGS_KEY],
     36                             use_grpc_server=use_grpc_server,
     37                             initialize_grpc_client=initialize_grpc_client)

File /mnt/2287294e-32c7-437b-84bd-452a29548b1a/NLP/Text_Generation/text_generator/DeepSpeed-MII/mii/server_client.py:83, in MIIServerClient.__init__(self, task_name, model_name, model_path, ds_optimize, ds_zero, ds_config, mii_configs, initialize_service, initialize_grpc_client, use_grpc_server)
     80     self.model = None
     82 if self.initialize_service:
---> 83     self.process = self._initialize_service(model_name,
     84                                             model_path,
     85                                             ds_optimize,
     86                                             ds_zero,
     87                                             ds_config,
     88                                             mii_configs)
     89     if self.use_grpc_server:
     90         self._wait_until_server_is_live()

File /mnt/2287294e-32c7-437b-84bd-452a29548b1a/NLP/Text_Generation/text_generator/DeepSpeed-MII/mii/server_client.py:209, in MIIServerClient._initialize_service(self, model_name, model_path, ds_optimize, ds_zero, ds_config, mii_configs)
    207     mii_env = os.environ.copy()
    208     mii_env["TRANSFORMERS_CACHE"] = model_path
--> 209     process = subprocess.Popen(cmd, env=mii_env)
    210 return process

File /mnt/2287294e-32c7-437b-84bd-452a29548b1a/conda_env/DeepSpeedInterface/lib/python3.9/subprocess.py:951, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask)
    947         if self.text_mode:
    948             self.stderr = io.TextIOWrapper(self.stderr,
    949                     encoding=encoding, errors=errors)
--> 951     self._execute_child(args, executable, preexec_fn, close_fds,
    952                         pass_fds, cwd, env,
    953                         startupinfo, creationflags, shell,
    954                         p2cread, p2cwrite,
    955                         c2pread, c2pwrite,
    956                         errread, errwrite,
    957                         restore_signals,
    958                         gid, gids, uid, umask,
    959                         start_new_session)
    960 except:
    961     # Cleanup if the child failed starting.
    962     for f in filter(None, (self.stdin, self.stdout, self.stderr)):

File /mnt/2287294e-32c7-437b-84bd-452a29548b1a/conda_env/DeepSpeedInterface/lib/python3.9/subprocess.py:1821, in Popen._execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, gid, gids, uid, umask, start_new_session)
   1819     if errno_num != 0:
   1820         err_msg = os.strerror(errno_num)
-> 1821     raise child_exception_type(errno_num, err_msg, err_filename)
   1822 raise child_exception_type(err_msg)

FileNotFoundError: [Errno 2] No such file or directory: 'deepspeed'

[mrwyattii]

Hi @C00reNUT, sorry for the late reply and thanks for checking out MII. Do you have DeepSpeed installed in your environment?

Could you show the output of ds_report?

[C00reNUT]

I tried it once more and when I run the code as a .py scripts it works, but when I run it from jupyter lab cells (which is what I did in the first place) it throws the FileNotFoundError error. The jupyter notebook is in the same location as the .py script and it uses the same conda env.

Anyway it works now. And I guess that the problem is probably some mismatch with PATHs on my machine...

ds_report

--------------------------------------------------
DeepSpeed C++/CUDA extension op report
--------------------------------------------------
NOTE: Ops not installed will be just-in-time (JIT) compiled at
      runtime if needed. Op compatibility means that your system
      meet the required dependencies to JIT install the op.
--------------------------------------------------
JIT compiled ops requires ninja
ninja .................. [OKAY]
--------------------------------------------------
op name ................ installed .. compatible
--------------------------------------------------
cpu_adam ............... [NO] ....... [OKAY]
cpu_adagrad ............ [NO] ....... [OKAY]
fused_adam ............. [NO] ....... [OKAY]
fused_lamb ............. [NO] ....... [OKAY]
 [WARNING]  please install triton==1.0.0 if you want to use sparse attention
sparse_attn ............ [NO] ....... [NO]
transformer ............ [NO] ....... [OKAY]
stochastic_transformer . [NO] ....... [OKAY]
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
async_io ............... [NO] ....... [NO]
utils .................. [NO] ....... [OKAY]
quantizer .............. [NO] ....... [OKAY]
transformer_inference .. [NO] ....... [OKAY]
--------------------------------------------------
DeepSpeed general environment info:
torch install path ............... ['/mnt/2287294e-32c7-437b-84bd-452a29548b1a/conda_env/DeepSpeedInterface/lib/python3.9/site-packages/torch']
torch version .................... 1.12.1
torch cuda version ............... 11.6
torch hip version ................ None
nvcc version ..................... 11.6
deepspeed install path ........... ['/mnt/2287294e-32c7-437b-84bd-452a29548b1a/conda_env/DeepSpeedInterface/lib/python3.9/site-packages/deepspeed']
deepspeed info ................... 0.7.2, unknown, unknown
deepspeed wheel compiled w. ...... torch 1.12, cuda 11.6

[TobyGE]

Same here! Thanks for sharing the solution @C00reNUT