Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker container stopped working with: ModuleNotFoundError: No module named 'ruamel' #984

Closed
fernandocamargoai opened this issue Aug 11, 2020 · 21 comments
Assignees
Labels
bug Something isn't working

Comments

@fernandocamargoai
Copy link

Describe the bug
Without any change to my code, new Docker containers aren't working anymore. When I try to run it, I get:

Traceback (most recent call last):
  File "/opt/conda/bin/bentoml", line 5, in <module>
    from bentoml.cli import cli
  File "/opt/conda/lib/python3.6/site-packages/bentoml/__init__.py", line 27, in <module>
    from bentoml.saved_bundle import load, save_to_dir
  File "/opt/conda/lib/python3.6/site-packages/bentoml/saved_bundle/__init__.py", line 15, in <module>
    from bentoml.saved_bundle.bundler import save_to_dir
  File "/opt/conda/lib/python3.6/site-packages/bentoml/saved_bundle/bundler.py", line 31, in <module>
    from bentoml.utils.usage_stats import track_save
  File "/opt/conda/lib/python3.6/site-packages/bentoml/utils/usage_stats.py", line 22, in <module>
    from ruamel.yaml import YAML
ModuleNotFoundError: No module named 'ruamel'

To Reproduce
Steps to reproduce the behavior:

  1. Create a BentoService with:
@env(conda_channels=["conda-forge"], conda_dependencies=["libpq=12.3"],
     pip_dependencies=["mxnet==1.4.1", "gluonts==0.5", "numpy==1.16", "pandas==1.0.5", "holidays==0.9.12",
                       "python-dateutil==2.8", "convertdate==2.2", "pydantic==1.6", "luigi==2.8", "sqlalchemy==1.3",
                       "psycopg2==2.8"])
  1. Pack the model and build the Docker Container
  2. When the container is being built, we can see the following package being installed while updating the conda environment: ruamel_yaml-0.15.87
  3. When trying to run the container, ModuleNotFoundError: No module named 'ruamel' appears. If you open bash into the container, you can see that the ruamel is installed, but can't be imported.

Expected behavior
It was expected to run successfully.

Screenshots/Logs

Resulting environment.yml:

name: bentoml-DemandForecaster
channels:
- defaults
- conda-forge
dependencies:
- python=3.6.11
- pip
- libpq=12.3

Log of installed packages through conda:

Downloading and Extracting Packages
idna-2.10            | 50 KB     | ########## | 100% 
python-3.6.11        | 34.1 MB   | ########## | 100% 
yaml-0.2.5           | 75 KB     | ########## | 100% 
cryptography-2.9.2   | 556 KB    | ########## | 100% 
pysocks-1.7.1        | 30 KB     | ########## | 100% 
krb5-1.17.1          | 1.3 MB    | ########## | 100% 
pyopenssl-19.1.0     | 48 KB     | ########## | 100% 
pycparser-2.20       | 94 KB     | ########## | 100% 
cffi-1.14.0          | 223 KB    | ########## | 100% 
sqlite-3.32.3        | 1.4 MB    | ########## | 100% 
tk-8.6.10            | 3.0 MB    | ########## | 100% 
conda-4.8.3          | 2.8 MB    | ########## | 100% 
wheel-0.34.2         | 51 KB     | ########## | 100% 
pycosat-0.6.3        | 82 KB     | ########## | 100% 
pip-20.2.1           | 1.8 MB    | ########## | 100% 
openssl-1.1.1g       | 2.5 MB    | ########## | 100% 
xz-5.2.5             | 341 KB    | ########## | 100% 
tqdm-4.42.1          | 56 KB     | ########## | 100% 
python_abi-3.6       | 4 KB      | ########## | 100% 
urllib3-1.25.9       | 103 KB    | ########## | 100% 
ruamel_yaml-0.15.87  | 270 KB    | ########## | 100% 
conda-package-handli | 797 KB    | ########## | 100% 
libpq-12.3           | 2.6 MB    | ########## | 100% 
certifi-2020.6.20    | 156 KB    | ########## | 100% 
requests-2.24.0      | 56 KB     | ########## | 100% 
setuptools-49.2.1    | 736 KB    | ########## | 100% 
ca-certificates-2020 | 125 KB    | ########## | 100% 
readline-8.0         | 356 KB    | ########## | 100% 
chardet-3.0.4        | 180 KB    | ########## | 100% 
brotlipy-0.7.0       | 323 KB    | ########## | 100% 
six-1.15.0           | 13 KB     | ########## | 100% 
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done

Error when trying to import bentoml:

Traceback (most recent call last):
  File "/opt/conda/bin/bentoml", line 5, in <module>
    from bentoml.cli import cli
  File "/opt/conda/lib/python3.6/site-packages/bentoml/__init__.py", line 27, in <module>
    from bentoml.saved_bundle import load, save_to_dir
  File "/opt/conda/lib/python3.6/site-packages/bentoml/saved_bundle/__init__.py", line 15, in <module>
    from bentoml.saved_bundle.bundler import save_to_dir
  File "/opt/conda/lib/python3.6/site-packages/bentoml/saved_bundle/bundler.py", line 31, in <module>
    from bentoml.utils.usage_stats import track_save
  File "/opt/conda/lib/python3.6/site-packages/bentoml/utils/usage_stats.py", line 22, in <module>
    from ruamel.yaml import YAML
ModuleNotFoundError: No module named 'ruamel'

Environment:

  • OS: Linux Manjaro
  • Python/BentoML Version: Python 3.6.11, BentoML 0.8.5

Additional context
As a workaround, I added ruamel.yaml=0.16 in conda_dependencies.

@fernandocamargoai fernandocamargoai added the bug Something isn't working label Aug 11, 2020
@parano parano self-assigned this Aug 11, 2020
@aslom
Copy link

aslom commented Aug 14, 2020

I have the same problem. I do not know bentoml - where is conda_dependencies file located for workaround?

@parano
Copy link
Member

parano commented Aug 14, 2020

@aslom in your BentoService definition's @env annotation:

@env(conda_dependencies=["ruamel.yaml=0.16"], ...)

@yubozhao
Copy link
Contributor

@fernandocamargoti I can reproduce this issue. After I ssh into the container, I can see the installed package is installed as ruamel_yaml instead of ruamel.yaml. I am looking into why this happens and work on a fix for it

@yubozhao
Copy link
Contributor

Additional discussion conda/conda#6762

@parano
Copy link
Member

parano commented Aug 19, 2020

@yubozhao

I've tried the following with the base docker image in 0.8.5:

docker run -it bentoml/model-server:0.8.5 bash
(base) root@c4fd2962da58:/# python
Python 3.7.6 (default, Jan  8 2020, 19:59:22)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ruamel.yaml
>>> import ruamel_yaml
>>> ruamel_yaml.__path__
['/opt/conda/lib/python3.7/site-packages/ruamel_yaml']
>>> ruamel.yaml.__path__
['/opt/conda/lib/python3.7/site-packages/ruamel/yaml']

It seems both package names work. Any idea why the package name ruamel.yaml stopped working after building a Bento API server docker image?

@aslom
Copy link

aslom commented Aug 20, 2020

@parano about https://github.com/bentoml/BentoML/blob/1812d290466470f35ed8d0374491da9231ee5151/docs/source/concepts.rst#conda-packages as I am newcomer - if I understand it correctly to add @env workfoaround to BentoService I will need to modify model python in IrisClassifier/20200812101515_76F43B/IrisClassifier/iris_classifier.py ? Something along lines of https://github.com/bentoml/BentoML/blob/1812d290466470f35ed8d0374491da9231ee5151/docs/source/concepts.rst#conda-packages ?

I modified it but still getting error:

$  grep ruamel IrisClassifier/iris_classifier.py
@env(conda_dependencies=["ruamel.yaml=0.16"],auto_pip_dependencies=True)
$  docker build -t aslom/iris-classifier .
....
$ docker run aslom/iris-classifier:latest
Traceback (most recent call last):
  File "/opt/conda/bin/bentoml", line 5, in <module>
    from bentoml.cli import cli
  File "/opt/conda/lib/python3.8/site-packages/bentoml/__init__.py", line 27, in <module>
    from bentoml.saved_bundle import load, save_to_dir
  File "/opt/conda/lib/python3.8/site-packages/bentoml/saved_bundle/__init__.py", line 15, in <module>
    from bentoml.saved_bundle.bundler import save_to_dir
  File "/opt/conda/lib/python3.8/site-packages/bentoml/saved_bundle/bundler.py", line 31, in <module>
    from bentoml.utils.usage_stats import track_save
  File "/opt/conda/lib/python3.8/site-packages/bentoml/utils/usage_stats.py", line 22, in <module>
    from ruamel.yaml import YAML
ModuleNotFoundError: No module named 'ruamel'

@aslom
Copy link

aslom commented Aug 20, 2020

Here is what I get with docker image I built (see comment above):

 docker run -it aslom/iris-classifier:latest /bin/bash
(base) root@59a4d93c2935:/bento# python
Python 3.8.5 (default, Aug  5 2020, 08:36:46)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ruamel.yaml
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'ruamel'
>>> import ruamel_yaml
>>> ruamel_yaml.__path__
['/opt/conda/lib/python3.8/site-packages/ruamel_yaml']
>>> ruamel.yaml.__path__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'ruamel' is not defined
>>>

@parano
Copy link
Member

parano commented Aug 20, 2020

Hi @aslom, looks like you made a typo there, it should be ruamel.yaml==0.16 (two =s).

We are working on a fix now #997, it will come with the next release!

@aslom
Copy link

aslom commented Aug 20, 2020

Interesting 😱 that there was no error during compile or run :(

I am still getting the same problem - see that it ruamel_yaml-0.15.87 [1]

what am I missing?

$ grep ruamel IrisClassifier/iris_classifier.py
@env(conda_dependencies=["ruamel.yaml==0.16"], auto_pip_dependencies=True)
$  docker build -t aslom/iris-classifier:latest . > log1.txt
$  grep ruamel log1.txt
ruamel_yaml-0.15.87  | 259 KB    | ########## | 100%
Requirement already satisfied: ruamel.yaml>=0.15.0 in /opt/conda/lib/python3.8/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 3)) (0.15.87)
$ docker run aslom/iris-classifier:latest
Traceback (most recent call last):
  File "/opt/conda/bin/bentoml", line 5, in <module>
    from bentoml.cli import cli
  File "/opt/conda/lib/python3.8/site-packages/bentoml/__init__.py", line 27, in <module>
    from bentoml.saved_bundle import load, save_to_dir
  File "/opt/conda/lib/python3.8/site-packages/bentoml/saved_bundle/__init__.py", line 15, in <module>
    from bentoml.saved_bundle.bundler import save_to_dir
  File "/opt/conda/lib/python3.8/site-packages/bentoml/saved_bundle/bundler.py", line 31, in <module>
    from bentoml.utils.usage_stats import track_save
  File "/opt/conda/lib/python3.8/site-packages/bentoml/utils/usage_stats.py", line 22, in <module>
    from ruamel.yaml import YAML
ModuleNotFoundError: No module named 'ruamel'

[1] attached log1.txt
log1.txt
iris_classifier.py.txt

@parano
Copy link
Member

parano commented Aug 20, 2020

@aslom I see, l think you modify the IrisClassifier/iris_classifier.py file under the saved bundle directory directly?

We'd recommend change the BentoService class code and re-run the save to create a new saved bundle, and build docker image again.

It is also possible to modify the environment.yml file under an existing saved bundle directory to manually add the conda dependency line. Although in general we don't recommend modifying files in saved bundle manually.

@aslom
Copy link

aslom commented Aug 20, 2020

@parano before new release with https://github.com/bentoml/BentoML/pull/997/files is available how do I build my own docker image for aslom/bentoml-model-server:0.8.6 from github - what is the easiest way to do it? I could not find instructions in https://github.com/bentoml/BentoML/blob/master/DEVELOPMENT.md#install-bentoml-from-source (I had not problem to build bentoml)

Maybe using https://github.com/bentoml/BentoML/blob/master/docker/model-server/release.sh ?
Not sure if it is the best way:

cd BentoML/docker/model-server
docker build -t aslom/bentoml-model-server:latest .
...
Removing intermediate container 35553db2bb91
 ---> 0171cd793be2
Step 5/8 : RUN pip install bentoml==$BENTOML_VERSION
 ---> Running in 4b62b86a2089
ERROR: Could not find a version that satisfies the requirement bentoml== (from versions: 0.0.1, 0.0.2, 0.0.3, 0.0.5, 0.0.6a0, 0.0.7.dev0, 0.0.7, 0.0.8, 0.0.8.post1, 0.0.9, 0.1.1, 0.1.2, 0.2.0, 0.2.1, 0.2.2, 0.3.0, 0.3.1, 0.3.3, 0.3.4, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.4.5, 0.4.7, 0.4.8, 0.4.9, 0.5.0, 0.5.1, 0.5.2, 0.5.3, 0.5.4, 0.5.5, 0.5.6, 0.5.7, 0.5.8, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.7.7, 0.7.8, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.8.4, 0.8.5)
ERROR: No matching distribution found for bentoml==
The command '/bin/sh -c pip install bentoml==$BENTOML_VERSION' returned a non-zero code: 1

@parano
Copy link
Member

parano commented Aug 20, 2020

@aslom You can build the base image and upload to your own dockerhub registry, and then use it with the custom docker image parameter: https://docs.bentoml.org/en/latest/concepts.html#using-other-docker-base-images

Although the PR is still working in progress. You won't need to rebuild the image after the PR was updated. I'd recommend using the workaround for now.

@aslom
Copy link

aslom commented Aug 20, 2020

@parano yes I copied bundle to local directory as it is the only place I found code that uses BentoService. Is there some guide docs how to change the BentoService class code ? Modified

 cat environment.yml
name: bentoml-IrisClassifier
channels:
- defaults
dependencies:
- python=3.8.5
- pip
- ruamel.yaml==0.16

tha got error

docker build -t aslom/bentoml-model-server:latest .|tee log2.txt
Sending build context to Docker daemon  53.76kB
Step 1/15 : FROM bentoml/model-server:0.8.5
 ---> 6639eed59dc6
Step 2/15 : COPY . /bento
 ---> 7f8f35654c87
Step 3/15 : WORKDIR /bento
...
Solving environment: ...working... failed

ResolvePackageNotFound:
  - ruamel.yaml==0.16

conda command not found, ignoring environment.yml
+ echo 'conda command not found, ignoring environment.yml'
+ pip install -r ./requirements.txt --no-cache-dir
...
$ docker run aslom/iris-classifier:latest
Traceback (most recent call last):
  File "/opt/conda/bin/bentoml", line 5, in <module>
    from bentoml.cli import cli
  File "/opt/conda/lib/python3.8/site-packages/bentoml/__init__.py", line 27, in <module>
    from bentoml.saved_bundle import load, save_to_dir
  File "/opt/conda/lib/python3.8/site-packages/bentoml/saved_bundle/__init__.py", line 15, in <module>
    from bentoml.saved_bundle.bundler import save_to_dir
  File "/opt/conda/lib/python3.8/site-packages/bentoml/saved_bundle/bundler.py", line 31, in <module>
    from bentoml.utils.usage_stats import track_save
  File "/opt/conda/lib/python3.8/site-packages/bentoml/utils/usage_stats.py", line 22, in <module>
    from ruamel.yaml import YAML
ModuleNotFoundError: No module named 'ruamel' 

log2.txt

@parano
Copy link
Member

parano commented Aug 20, 2020

@aslom you should re-run the save call to generate a new saved bundle:

git clone http://github.com/bentoml/bentoml

cd bentoml

# modify guides/quick-start/iris_classifier.py to include the workaround

# re-run save
python guides/quick-start/main.py

# find the saved bundle path
cd $(bentoml get IrisClassifier:latest --print-location --quiet)

# build docker image:
docker build -t aslom/bentoml-model-server:latest .

@yubozhao
Copy link
Contributor

yubozhao commented Aug 20, 2020

@aslom I think the issue is ruamel.yaml is available in the conda-forge channel instead of the default conda channels
You can add this to your env decorator and rerun save to generate a new saved bundle

@env(conda_channels=["conda-forge"], conda_dependencies=["ruamel.yaml"], auto_pip_dependencies=True)
class ...

@aslom
Copy link

aslom commented Aug 20, 2020

@yubozhao tried but got syntax error:


$ grep ruamel IrisClassifier/iris_classifier.py
@env(conda_channels=["conda-forge"], conda_dependencies=["ruamel.yaml", auto_pip_dependencies=True)

$ cat environment.yml
name: bentoml-IrisClassifier
channels:
- defaults
- conda-forge
dependencies:
- python=3.8.5
- pip
- ruamel.yaml==0.16


$ docker build -t aslom/iris-classifier:latest . > log5.txt
$ docker run aslom/iris-classifier:latest
[2020-08-20 18:01:11,040] INFO - Starting BentoML API server in production mode..
[2020-08-20 18:01:11,328] INFO - get_gunicorn_num_of_workers: 5, calculated by cpu count
[2020-08-20 18:01:11 +0000] [1] [INFO] Starting gunicorn 20.0.4
[2020-08-20 18:01:11 +0000] [1] [INFO] Listening at: http://0.0.0.0:5000 (1)
[2020-08-20 18:01:11 +0000] [1] [INFO] Using worker: sync
[2020-08-20 18:01:11 +0000] [14] [INFO] Booting worker with pid: 14
[2020-08-20 18:01:11 +0000] [15] [INFO] Booting worker with pid: 15
[2020-08-20 18:01:11 +0000] [16] [INFO] Booting worker with pid: 16
[2020-08-20 18:01:11 +0000] [17] [INFO] Booting worker with pid: 17
[2020-08-20 18:01:11 +0000] [18] [INFO] Booting worker with pid: 18
[2020-08-20 18:01:11 +0000] [14] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/opt/conda/lib/python3.8/site-packages/gunicorn/workers/base.py", line 119, in init_process
    self.load_wsgi()
  File "/opt/conda/lib/python3.8/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/opt/conda/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/opt/conda/lib/python3.8/site-packages/bentoml/server/gunicorn_server.py", line 91, in load
    bento_service = load(self.bento_service_bundle_path)
  File "/opt/conda/lib/python3.8/site-packages/bentoml/saved_bundle/loader.py", line 199, in load
    svc_cls = load_bento_service_class(bundle_path)
  File "/opt/conda/lib/python3.8/site-packages/bentoml/saved_bundle/loader.py", line 138, in load_bento_service_class
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 779, in exec_module
  File "<frozen importlib._bootstrap_external>", line 916, in get_code
  File "<frozen importlib._bootstrap_external>", line 846, in source_to_code
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/bento/IrisClassifier/iris_classifier.py", line 5
    @env(conda_channels=["conda-forge"], conda_dependencies=["ruamel.yaml", auto_pip_dependencies=True)
                                                                                                 ^
SyntaxError: invalid syntax
[2020-08-20 18:01:11 +0000] [14] [INFO] Worker exiting (pid: 14)
...

log5.txt

@aslom
Copy link

aslom commented Aug 20, 2020

@parano how do you install all dependencies? Could not find it in DEVELOPMENT.md

$ git pull

Already up to date.

$  python guides/quick-start/main.py
Traceback (most recent call last):
  File "guides/quick-start/main.py", line 1, in <module>
    from sklearn import svm
ModuleNotFoundError: No module named 'sklearn'

@aslom
Copy link

aslom commented Aug 20, 2020

Fixed the typo missing ] and that finally made it work :)

$ grep ruamel IrisClassifier/iris_classifier.py
@env(conda_channels=["conda-forge"], conda_dependencies=["ruamel.yaml"], auto_pip_dependencies=True)
$ cat environment.yml
name: bentoml-IrisClassifier
channels:
- defaults
- conda-forge
dependencies:
- python=3.8.5
- pip
- ruamel.yaml==0.16

$ docker build -t aslom/iris-classifier:latest . |tee log6.txt
...

$ grep ruamel log6.txt
ruamel_yaml-0.15.87  | 259 KB    | ########## | 100%
ruamel.yaml-0.16.0   | 172 KB    | ########## | 100%
ruamel-1.0           | 7 KB      | ########## | 100%
Requirement already satisfied: ruamel.yaml>=0.15.0 in /opt/conda/lib/python3.8/site-packages (from bentoml==0.8.5->-r ./requirements.txt (line 3)) (0.15.87)
[log6.txt](https://github.com/bentoml/BentoML/files/5104909/log6.txt)


$ docker run aslom/iris-classifier:latest
[2020-08-20 18:46:49,506] INFO - Starting BentoML API server in production mode..
[2020-08-20 18:46:49,733] INFO - get_gunicorn_num_of_workers: 5, calculated by cpu count
[2020-08-20 18:46:49 +0000] [1] [INFO] Starting gunicorn 20.0.4
[2020-08-20 18:46:49 +0000] [1] [INFO] Listening at: http://0.0.0.0:5000 (1)
[2020-08-20 18:46:49 +0000] [1] [INFO] Using worker: sync
[2020-08-20 18:46:49 +0000] [14] [INFO] Booting worker with pid: 14
[2020-08-20 18:46:49 +0000] [15] [INFO] Booting worker with pid: 15
[2020-08-20 18:46:49 +0000] [16] [INFO] Booting worker with pid: 16
[2020-08-20 18:46:49 +0000] [17] [INFO] Booting worker with pid: 17
[2020-08-20 18:46:49 +0000] [18] [INFO] Booting worker with pid: 18
[2020-08-20 18:46:50,125] WARNING - bentoml.handlers.* will be deprecated after BentoML 1.0, use bentoml.adapters.* instead
[2020-08-20 18:46:50,126] WARNING - DataframeHandler will be deprecated after BentoML 1.0, use DataframeInput instead
[2020-08-20 18:46:50,216] WARNING - bentoml.handlers.* will be deprecated after BentoML 1.0, use bentoml.adapters.* instead
[2020-08-20 18:46:50,217] WARNING - DataframeHandler will be deprecated after BentoML 1.0, use DataframeInput instead
[2020-08-20 18:46:50,262] WARNING - bentoml.handlers.* will be deprecated after BentoML 1.0, use bentoml.adapters.* instead
[2020-08-20 18:46:50,271] WARNING - DataframeHandler will be deprecated after BentoML 1.0, use DataframeInput instead
[2020-08-20 18:46:50,354] WARNING - bentoml.handlers.* will be deprecated after BentoML 1.0, use bentoml.adapters.* instead
[2020-08-20 18:46:50,354] WARNING - DataframeHandler will be deprecated after BentoML 1.0, use DataframeInput instead
[2020-08-20 18:46:50,354] WARNING - bentoml.handlers.* will be deprecated after BentoML 1.0, use bentoml.adapters.* instead
[2020-08-20 18:46:50,355] WARNING - DataframeHandler will be deprecated after BentoML 1.0, use DataframeInput instead
^C[2020-08-20 18:47:22 +0000] [1] [INFO] Handling signal: int
[2020-08-20 18:47:22 +0000] [17] [INFO] Worker exiting (pid: 17)
[2020-08-20 18:47:22 +0000] [15] [INFO] Worker exiting (pid: 15)
[2020-08-20 18:47:22 +0000] [18] [INFO] Worker exiting (pid: 18)
[2020-08-20 18:47:22 +0000] [14] [INFO] Worker exiting (pid: 14)
[2020-08-20 18:47:22 +0000] [16] [INFO] Worker exiting (pid: 16)
[2020-08-20 18:47:23 +0000] [1] [INFO] Shutting down: Master

@aslom
Copy link

aslom commented Aug 20, 2020

Thank you @parano and @yubozhao for your help.

@yubozhao
Copy link
Contributor

@aslom Awesome. I was just typing about I missed the closing ]

@yubozhao
Copy link
Contributor

@fernandocamargoti @aslom We just released BentoML 0.8.6 It contains the fix from #1012.

I am closing this for now. Feel free to reopen if you still encounter this bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants