Mlflow Model serve error

[Dekermanjian]

Hi,

I am learning how to use Kedro_mlflow using this tutorial. I was able to get through all of it until I got to the model serving section. I am running into a ResolvePackageNotFound error

2023/05/06 18:53:58 INFO mlflow.models.cli: Selected backend for flavor 'python_function'
2023/05/06 18:53:59 INFO mlflow.utils.conda: === Creating conda environment mlflow-1dbfb03dc532b412d12edb6b1cf9e8d68679bead ===

EnvironmentSectionNotValid: The following sections on '/Users//Desktop/kedro_mlflow2/kedro-mlflow-tutorial/mlruns/1/086c0b354cf142b987ebb4dccf32258f/artifacts/kedro_mlflow_tutorial/conda.yaml' are invalid and will be ignored:
 - build_dependencies
 - python

Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies.  Conda may not use the correct pip to install your packages, and they may end up in the wrong place.  Please add an explicit pip dependency.  I'm adding one for you, but still nagging you.
Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound: 
  - kedro_mlflow_tutorial==0.1

Traceback (most recent call last):
  File "/Users//opt/anaconda3/envs/kedro_mlflow_example/bin/mlflow", line 8, in <module>
    sys.exit(cli())
  File "/Users//opt/anaconda3/envs/kedro_mlflow_example/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users//opt/anaconda3/envs/kedro_mlflow_example/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users//opt/anaconda3/envs/kedro_mlflow_example/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users//opt/anaconda3/envs/kedro_mlflow_example/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users//opt/anaconda3/envs/kedro_mlflow_example/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users//opt/anaconda3/envs/kedro_mlflow_example/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users//opt/anaconda3/envs/kedro_mlflow_example/lib/python3.10/site-packages/mlflow/models/cli.py", line 68, in serve
    return _get_flavor_backend(
  File "/Users//opt/anaconda3/envs/kedro_mlflow_example/lib/python3.10/site-packages/mlflow/pyfunc/backend.py", line 210, in serve
    conda_env_name = get_or_create_conda_env(
  File "/Users//opt/anaconda3/envs/kedro_mlflow_example/lib/python3.10/site-packages/mlflow/utils/conda.py", line 214, in get_or_create_conda_env
    process._exec_cmd(
  File "/Users//opt/anaconda3/envs/kedro_mlflow_example/lib/python3.10/site-packages/mlflow/utils/process.py", line 94, in _exec_cmd
    raise ShellCommandException.from_completed_process(comp_process)
mlflow.utils.process.ShellCommandException: Non-zero exit code: 1
Command: ['/Users//opt/anaconda3/bin/conda', 'env', 'create', '-n', 'mlflow-1dbfb03dc532b412d12edb6b1cf9e8d68679bead', '--file', '/Users//Desktop/kedro_mlflow2/kedro-mlflow-tutorial/mlruns/1/086c0b354cf142b987ebb4dccf32258f/artifacts/kedro_mlflow_tutorial/conda.yaml']

Any help with this would be appreciated.

[Dekermanjian]

Just in case anyone else has this same issue.

The conda.yaml file generated in the mlruns is not correct. You need to fix it so that the environment is set up properly when you are serving with MlFlow.

Here is what a working conda.yaml file looks like:

dependencies:
- python=3.9.16
- pip
- pip:
  - -e /Path/kedro-mlflow-tutorial/src/.

[Galileo-Galilei]

Hi @Dekermanjian,

actually your solution will work only locally because anyone who wants to use your model will need face an error if he does not have the src folder at the same path on his computer.

The recommended way to use it is to keep the conda.yml from the tutorial, and to make the kedro_mlflow_tutorial==0.1 available for install. You have many way to do this:

• in an enterprise setup, you will likely publish the kedro_mlflow_tutorial==0.1 on your enterprise package manager (private pypi / nexus, gitlab or github package mangers...). conda will be able to fetch it and build the environment with no error
• In a local / development environment, you should likely create the conda environment manually, use -e /Path/kedro-mlflow-tutorial/src/ to install it, and then serve the model with the --no-conda flag, e.g. mlflow models serve -m [model/path] --no-conda
• In a public / open-source environment, I'd recommend to publish your package (here, kedro-mlflow-tutorial on public PyPI). I don't do this for this tutroial because it is likely than different people experimenting with this tutorial will end up install the one on PyPI accidentally

Does it make sense ?

[Dekermanjian]

@Galileo-Galilei Yes, this does make a lot of sense. Thank you very much for taking the time to explain the proper way to serve the model.