Skip to content

Containerize command - #847

Merged
parano merged 26 commits into
bentoml:masterfrom
jackyzha0:containerize-command
Jul 10, 2020
Merged

parano merged 26 commits into
bentoml:masterfrom
jackyzha0:containerize-command

Conversation

@jackyzha0

@jackyzha0 jackyzha0 commented Jun 26, 2020 •

Copy link
Copy Markdown
Contributor

Description

  • add implementation of bentoml containerize with --push and --tag flags

Motivation and Context

How Has This Been Tested?

  • locally

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature and improvements (non-breaking change which adds/improves functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Code Refactoring (internal change which is not user facing)
  • Documentation
  • Test, CI, or build

Component(s) if applicable

  • BentoService (service definition, dependency management, API input/output adapters)
  • Model Artifact (model serialization, multi-framework support)
  • Model Server (mico-batching, dockerisation, logging, OpenAPI, instruments)
  • YataiService gRPC server (model registry, cloud deployment automation)
  • YataiService web server (nodejs HTTP server and web UI)
  • Internal (BentoML's own configuration, logging, utility, exception handling)
  • BentoML CLI

Checklist:

  • My code follows the bentoml code style, both ./dev/format.sh and
    ./dev/lint.sh script have passed
    (instructions).
  • My change reduces project test coverage and requires unit tests to be added
  • I have added unit tests covering my code change
  • My change requires a change to the documentation
  • I have updated the documentation accordingly

@codecov

codecov Bot commented Jun 26, 2020 •

Copy link
Copy Markdown

Codecov Report

Merging #847 into master will decrease coverage by 0.10%.
The diff coverage is 65.09%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #847      +/-   ##
==========================================
- Coverage   62.35%   62.24%   -0.11%     
==========================================
  Files         117      118       +1     
  Lines        7783     7846      +63     
==========================================
+ Hits         4853     4884      +31     
- Misses       2930     2962      +32     
Impacted Files Coverage Δ
bentoml/cli/click_utils.py 70.83% <0.00%> (-0.60%) ⬇️
bentoml/saved_bundle/templates.py 100.00% <ø> (ø)
bentoml/cli/bento_service.py 62.10% <62.10%> (ø)
bentoml/cli/__init__.py 96.15% <100.00%> (+20.71%) ⬆️
bentoml/cli/bento_management.py 31.89% <100.00%> (ø)
bentoml/cli/utils.py 47.82% <100.00%> (+9.05%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7a23ebb...d406672. Read the comment docs.

Comment thread bentoml/cli/bento.py Resolved
Comment thread bentoml/cli/bento.py Resolved
Comment thread bentoml/cli/bento.py Resolved
Comment thread bentoml/cli/bento.py Resolved
Comment thread bentoml/cli/bento.py Outdated

try:
docker_api = docker.APIClient()
ensure_docker_available_or_raise()

@yubozhao yubozhao Jun 29, 2020 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are building with Docker, let's move the checking all the way to the top

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like Python Docker SDK will give an error if Docker isn't installed anyways, so I removed it because it was redundant

Comment thread bentoml/cli/bento.py Resolved
Comment thread bentoml/cli/bento.py Resolved
Comment thread bentoml/cli/utils.py Resolved
@pep8speaks

pep8speaks commented Jul 1, 2020 •

Copy link
Copy Markdown

Hello @jackyzha0, Thanks for updating this PR.

There are currently no PEP 8 issues detected in this PR. Cheers! 🍻

Comment last updated at 2020-07-09 06:55:26 UTC

@jackyzha0
jackyzha0 marked this pull request as ready for review July 1, 2020 05:52
Comment thread bentoml/cli/utils.py Resolved
Comment thread bentoml/cli/utils.py Resolved
Comment thread bentoml/cli/bento.py Resolved
Comment thread bentoml/cli/bento.py Resolved
Comment thread bentoml/cli/bento.py Resolved
Comment thread bentoml/cli/bento.py Outdated
click.echo('Service %s artifact directory => %s' % (name, target_dir))

@cli.command(
help='Containerize given Bento into a Docker image',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better help message, maybe something along the line of Containerize the given BentoService into a BentoML model service docker image.? You probably can find a better way to phrase it

Comment thread bentoml/cli/bento.py Outdated

click.echo('Service %s artifact directory => %s' % (name, target_dir))

@cli.command(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should rename the command to create-docker-image using the name=? I want something very simple and direct. What do you think?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO containerize is pretty self-explanatory and can be left as-is. Thoughts, @parano?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jackyzha0 yea I think containerize works

Comment thread bentoml/cli/bento.py Outdated
with Spinner(f"Building Docker image: {name}\n"):
_echo_docker_api_result(
docker_api.build(
path=bento_service_bundle_path, tag=tag, decode=True,

@yubozhao yubozhao Jul 2, 2020 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you write a test for building from the local path and remote s3 URL?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tested this locally with both local and remote s3 URL but I have no clue how to go about writing a unit test for this ://

Comment thread bentoml/cli/bento.py Outdated
tag=version,
stream=True,
decode=True,
auth_config=auth_config_payload,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when username and password are not provided via the CLI, does docker_api.push pick up the local docker authentication information?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

Comment thread bentoml/cli/bento.py Resolved
Comment thread bentoml/cli/bento.py Resolved
f'{cmd_group.name} {command_name} failed: {str(e)}'
)
msg = f'{cmd_group.name} {command_name} failed: {str(e)}'
raise ClickException(click.style(msg, fg='red'))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment thread bentoml/cli/bento.py Resolved
Comment thread bentoml/cli/bento.py Resolved
Comment thread bentoml/cli/bento.py Resolved
Comment thread bentoml/cli/utils.py Resolved

@flosincapite flosincapite left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests looking pretty snug!

Comment thread bentoml/cli/bento.py Resolved
Comment thread bentoml/cli/__init__.py
# See the License for the specific language governing permissions and
# limitations under the License.

import re

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI: it looks like you moved this stuff to bento_service.py. You can use git mv or git cp when you migrate a lot of code from one file into a new file--that way, the new file will retain the old file's version history (and diffs will be more reasonable).

def get_name_version_from_tag(bento):
# returns name:version
if ":" in bento:
return bento.split(":")[:2]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to log a warning or something if there is more than one colon in the string--[:2] is a nice way to make things run smoothly, but doing this cleanup silently might have surprising results for a naive user.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will log a warning here

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh wait actually it seems like the resolve_bundle_path function above this should catch this and raise an error

Name components may contain lowercase letters, digits and separators.
A separator is defined as a period, one or two underscores, or one or more dashes.

A tag name must be valid ASCII and may contain lowercase and uppercase letters,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are name and tag validated somewhere beforehand?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no but we decided to just reformat it to be compatible and notify the user as per https://bentoml.slack.com/archives/CKC2JP3TM/p1593107935076500

Comment thread tests/cli/test_cli.py
return file_path


def assert_equal_lists(res, expected):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, my understanding is that

assert res == expected

will work fine for lists. If you're concerned about arbitrary types of iterable, you can do

assert list(res) == list(expected)

Comment thread tests/cli/test_cli.py
assert all([a == b for a, b in zip(expected, res)])


def test_make_bento_name_docker_compatible():

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good use case for parametrized tests. Internally, pytest will run a separate fixture with each set of parameters, so if there's a failure, you'll see exactly which set of parameters failed.

Comment thread tests/cli/test_cli.py
},
]

with pytest.raises(BentoMLException) as e:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent :)


_echo(f"Found Bento: {bento_service_bundle_path}")

name, version = get_name_version_from_tag(bento)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will break when the bento parameter is a file path. You can get the name and version with load_bento_service_metadata(bento_service_bundle_path)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will allow us to use the containerize by giving it the saved_path:

bentoml containerize $SAVED_PATH -t test_image

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently it breaks with the follow error:

 $  bentoml containerize /Users/chaoyu/bentoml/repository/IrisClassifier/20200701222443_EC5EAB -t test_image
Found Bento: /Users/chaoyu/bentoml/repository/IrisClassifier/20200701222443_EC5EAB
Traceback (most recent call last):
  File "/Users/chaoyu/opt/miniconda3/envs/btml-dev/bin/bentoml", line 33, in <module>
    sys.exit(load_entry_point('BentoML', 'console_scripts', 'bentoml')())
  File "/Users/chaoyu/opt/miniconda3/envs/btml-dev/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/chaoyu/opt/miniconda3/envs/btml-dev/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/chaoyu/opt/miniconda3/envs/btml-dev/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/chaoyu/opt/miniconda3/envs/btml-dev/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/chaoyu/opt/miniconda3/envs/btml-dev/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/chaoyu/workspace/BentoML/bentoml/cli/click_utils.py", line 140, in wrapper
    return func(*args, **kwargs)
  File "/Users/chaoyu/workspace/BentoML/bentoml/cli/click_utils.py", line 117, in wrapper
    return_value = func(*args, **kwargs)
  File "/Users/chaoyu/workspace/BentoML/bentoml/cli/click_utils.py", line 101, in wrapper
    return func(*args, **kwargs)
  File "/Users/chaoyu/workspace/BentoML/bentoml/cli/bento_service.py", line 484, in containerize
    name, version = make_bento_name_docker_compatible(name, version)
  File "/Users/chaoyu/workspace/BentoML/bentoml/cli/bento_service.py", line 110, in make_bento_name_docker_compatible
    tag = tag.lstrip(".-")[:128]
AttributeError: 'NoneType' object has no attribute 'lstrip'

return


def make_bento_name_docker_compatible(name, tag):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the difference between name and tag here? It wasn't clear to me.

tag parameter here is the docker image version, not the entire docker image tag, right?

'--tag',
help="Optional image tag. If not specified, Bento will generate one from "
"the name of the Bento.",
required=False,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel if you could valid this tag parameter here with a validation callback here, the rest of the code could be simplified https://click.palletsprojects.com/en/7.x/options/#callbacks-for-validation

if tag is not None:
name, v = get_name_version_from_tag(tag)
if v is not None:
version = v

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a log message here? f"Tag {tag} does not specify an image version, using BentoService version {bento_service_version} as docker image version."

f'Bento tag was changed to be Docker compatible. \n'
f'"{bento}" -> "{full_tag}"',
CLI_COLOR_WARNING,
)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming you added some validation for the tag parameter, this part could be rewritten to something like this:

bs_metadata = load_bento_service_metadata(bento_service_bundle_path)

if ":" not in tag:
   _echo(f"Tag {tag} does not specify an image version, using BentoService version {bs_metadata.version} as docker image version.")
   version= to_docker_tag_compatible_string(bs_metadata.version)
   tag = f"{tag}:{version}"

_echo("Containerizing BentoService {bs_metadata.name}:{bs_metadata.version}, docker image tag: {tag}")

docker_api = docker.APIClient()
...



@parano

parano commented Jul 10, 2020

Copy link
Copy Markdown
Collaborator

Great work @jackyzha0! I tried it out locally and it works great and I really love the UX.

I just finished a pass with a few minor comments. I think the most important one being fixing running containerize with a file path to saved bundle. As the PR is getting really large, I will go ahead and merge it, feel free to address those comments in a follow-up PR.

@parano
parano merged commit 7adf84b into bentoml:master Jul 10, 2020
aarnphm pushed a commit to aarnphm/BentoML that referenced this pull request Jul 29, 2022
* add basic draft containerize

* more wip stuff

* linting + formatting

* refactored to use CLIException

* fix bad api catch and better docs

* fix bad push catching and add colour

* linting

* fixed bad layer size calculation

* better unpack

* address comments

* address naming

* update containerize docs

* wip migration

* fixed bad imports

* refactor bento_service tag handling behaviour

* fix bad patch

* update makefile and properly fix patch

* add _echo_docker_api_result tests

* fixed bad comparison in test

* add basic e2e and test_make_bento_name_docker_compatible

* remove e2e for another pr
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a command to containerize the model API server

5 participants