Add prediction logging to AWS Lambda deployment #790

jackyzha0 · 2020-06-12T17:48:03Z

Description

fix some basic typos in Lambda utils
made bad python version during Lambda deploy error more descriptive
add prediction logging to AWS Lambda endpoint

Motivation and Context

Improves usability of BentoML
fixes Prediction log in AWS Lambda deployment #524

How Has This Been Tested?

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Documentation
Test, CI, or build
None

Components (if applicable)

BentoService (model packaging, dependency management, handler definition)
Model Artifact (model serialization, multi-framework support)
Model Server (mico-batching, logging, metrics, tracing, benchmark, OpenAPI)
YataiService (model management, deployment automation)
Documentation

Checklist:

My code follows the bentoml code style, both ./dev/format.sh and
./dev/lint.sh script have passed
(instructions).
My change requires a change to the documentation.
I have updated the documentation accordingly.
My change requires a change in bentoml/gallery example notebooks
I have sent a pull request to bentoml/gallery to make that change

* make sagemaker docker image have same file structure * use consistent file names * move operator code out of __init__ to avoid loading unused code in model server startup * refactor deployment validator * reorganize bento repository code * deployment valiator test&linting error fix * more repository code cleanup * renaming and adding inline comments * move out lambda operator code to separate file

parano · 2020-06-12T18:59:37Z

Hi @jackyzha0, thanks for the PR! Looks like the AWS lambda related unit test is failing now, you can find more details by clicking the Travis CI links on this page below. https://travis-ci.org/github/bentoml/BentoML/jobs/697719556

You can also run that test locally and see if you could reproduce the error:

 $ pytest tests/deployment/aws_lambda/test_aws_lambda_deployment_operator.py

yubozhao · 2020-06-12T20:06:57Z

bentoml/yatai/deployment/aws_lambda/lambda_app.py

-        return bento_service_api.handle_aws_lambda_event(event)
+        print(f'Got prediction request with body "{event["body"]}"')
+        prediction = bento_service_api.handle_aws_lambda_event(event)
+        print(


This is great.

We can take one step further and make this into JSON format with the input and output. Users can stream the logs out to ElasticSearch for searching and/or use the result for training later on.

You can check out the current prediction log format in the BentoAPIServer for reference:

BentoML/bentoml/server/bento_api_server.py

Line 293 in a134e3a

request_log = {

…da-logging

pep8speaks · 2020-06-15T06:53:25Z

Hello @jackyzha0, Thanks for updating this PR.

There are currently no PEP 8 issues detected in this PR. Cheers! 🍻

Comment last updated at 2020-06-16 00:39:35 UTC

parano · 2020-06-15T17:21:45Z

Hi @jackyzha0, looks like CI failed on code formatting check, you will need to run ./dev/format.sh script again to format your code

bentoml/yatai/deployment/aws_lambda/lambda_app.py

parano · 2020-06-15T17:26:53Z

bentoml/yatai/deployment/aws_lambda/lambda_app.py

@@ -45,11 +49,11 @@
 if not os.path.exists(bento_bundle_path):
    bento_bundle_path = os.path.join('/tmp/requirements', bento_name)

-print(f'Loading BentoService bundle from path: "{bento_bundle_path}"')
+logger.info('Loading BentoService bundle from path: "%s"', bento_bundle_path)


how about changing all these to logger.debug and only keep the prediction log in logger.info?

parano · 2020-06-15T17:30:14Z

bentoml/yatai/deployment/aws_lambda/lambda_app.py

+import logging
+
+logger = logging.getLogger(__name__)
+logger.setLevel(logging.DEBUG)


after import bentoml below, the logging format and level might get reset, so probably don't need this setLevel here.

Could you move logger = logging.getLogger(__name__) to after the from bentoml import load line? and keep the logging message in download_extra_resources just using print?

It is a bit tricky here because bentoml package may not be available to import before the download_extra_resources call, as the package file might be stored in s3 and needs to be downloaded first.

parano · 2020-06-16T00:14:12Z

bentoml/yatai/deployment/aws_lambda/lambda_app.py

@@ -38,18 +40,21 @@
 os.environ['BENTOML_HOME'] = '/tmp/bentoml/'
 from bentoml import load  # noqa

+logger = logging.getLogger(__name__)
+logger.setLevel(logging.DEBUG)


I think we can remove this and hide debug logs by default, those lots are usually only used for BentoML developer/contributor to debug the feature and not so much for BentoML user. For BentoML developer, we can always enable debug log with a simple environment variable change BENTOML__LOGGING__LOGGING_LEVEL=debug

What is the default logging level for BentoML? Is it info? When I don't explicitly set the logging level to be DEBUG or INFO, my logger.info() statements don't show up

The default is INFO https://github.com/bentoml/BentoML/blob/master/bentoml/configuration/default_bentoml.cfg#L30

I just realized this will not be using the logger set for BentoML module because lambda_app.py will be copy'd to a separate directory when deployed to AWS lambda. To use BentoML's logging config here, you will need to use something like logger = logging.getLogger('bentoml.lambda_app')

codecov · 2020-06-16T00:14:31Z

Codecov Report

Merging #790 into master will increase coverage by 0.14%.
The diff coverage is 86.66%.

@@            Coverage Diff             @@
##           master     #790      +/-   ##
==========================================
+ Coverage   55.75%   55.89%   +0.14%     
==========================================
  Files         114      114              
  Lines        8346     8400      +54     
==========================================
+ Hits         4653     4695      +42     
- Misses       3693     3705      +12

Impacted Files	Coverage Δ
bentoml/yatai/deployment/aws_lambda/operator.py	`58.73% <ø> (ø)`
bentoml/yatai/deployment/aws_lambda/utils.py	`27.77% <66.66%> (ø)`
bentoml/yatai/deployment/aws_lambda/lambda_app.py	`89.47% <91.66%> (-0.85%)`	⬇️
bentoml/saved_bundle/config.py	`90.74% <0.00%> (-4.11%)`	⬇️
bentoml/yatai/repository/metadata_store.py	`66.03% <0.00%> (-1.68%)`	⬇️
bentoml/adapters/dataframe_input.py	`80.00% <0.00%> (-0.44%)`	⬇️
bentoml/handlers/__init__.py	`100.00% <0.00%> (ø)`
bentoml/adapters/base_input.py	`64.40% <0.00%> (ø)`
bentoml/configuration/__init__.py	`78.37% <0.00%> (+0.60%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 09f51aa...319d340. Read the comment docs.

parano · 2020-06-16T00:15:25Z

bentoml/yatai/deployment/aws_lambda/lambda_app.py

+        logger.info('Got prediction request with body "%s"', {event["body"]})
+        prediction = bento_service_api.handle_aws_lambda_event(event)
+
+        logger.debug(


following the comment about logging level above, I think here it should be updated to logger.info, and all others can be set to either logger.debug or logger.error, that way the user can get a very clean and usable prediction log

Would line 76 and line 90 be logger.debug then?

Actually, it's redundant information so I'll just remove those lines

parano · 2020-06-16T01:06:32Z

Thanks for updating the PR @jackyzha0, I ran the lambda end-to-end tests on my end and verified the prediction JSON logs showed up on AWS cloud watch, everything looks great! Merging now!

* Repository and Deployment refactor and cleanup (bentoml#771) * make sagemaker docker image have same file structure * use consistent file names * move operator code out of __init__ to avoid loading unused code in model server startup * refactor deployment validator * reorganize bento repository code * deployment valiator test&linting error fix * more repository code cleanup * renaming and adding inline comments * move out lambda operator code to separate file * fix some typos + adding logging to lambda callback * fix some linting problems * fix mock not returning proper form * add trailing comma * add better logging + docs * whitespace fixes * move logging and consolidate debug line * update docs + fixed logging * fix linting issues * remove redundant logging and switch to bento logger * update docs Co-authored-by: Chaoyu <paranoyang@gmail.com> Co-authored-by: cory <cory.massaro@gmail.com>

parano and others added 4 commits June 9, 2020 21:20

Merge remote-tracking branch 'upstream/master'

9d606af

fix some typos + adding logging to lambda callback

c253d5b

fix some linting problems

52eaa1e

parano requested a review from yubozhao June 12, 2020 18:21

parano changed the title ~~Lambda logging~~ Add prediction logging to AWS Lambda deployment Jun 12, 2020

jackyzha0 added 2 commits June 12, 2020 12:23

fix mock not returning proper form

24548f7

add trailing comma

1438d1e

yubozhao reviewed Jun 12, 2020

View reviewed changes

jackyzha0 added 2 commits June 13, 2020 09:33

Merge branch 'master' of https://github.com/bentoml/BentoML into lamb…

db0e9c9

…da-logging

add better logging + docs

76ba753

whitespace fixes

342a8cd

parano reviewed Jun 15, 2020

View reviewed changes

bentoml/yatai/deployment/aws_lambda/lambda_app.py Outdated Show resolved Hide resolved

parano reviewed Jun 15, 2020

View reviewed changes

jackyzha0 added 3 commits June 15, 2020 16:28

move logging and consolidate debug line

bf9f71d

update docs + fixed logging

aac04ed

fix linting issues

e6d37f1

parano reviewed Jun 16, 2020

View reviewed changes

jackyzha0 added 2 commits June 15, 2020 17:33

remove redundant logging and switch to bento logger

13b94f8

update docs

319d340

parano added the LGTM label Jun 16, 2020

parano merged commit 2f138fc into bentoml:master Jun 16, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add prediction logging to AWS Lambda deployment #790

Add prediction logging to AWS Lambda deployment #790

jackyzha0 commented Jun 12, 2020

parano commented Jun 12, 2020

yubozhao Jun 12, 2020

pep8speaks commented Jun 15, 2020 •

edited

Loading

parano commented Jun 15, 2020

parano Jun 15, 2020

parano Jun 15, 2020

parano Jun 16, 2020

jackyzha0 Jun 16, 2020

parano Jun 16, 2020 •

edited

Loading

codecov bot commented Jun 16, 2020 •

edited

Loading

parano Jun 16, 2020 •

edited

Loading

jackyzha0 Jun 16, 2020

jackyzha0 Jun 16, 2020

parano commented Jun 16, 2020

Add prediction logging to AWS Lambda deployment #790

Add prediction logging to AWS Lambda deployment #790

Conversation

jackyzha0 commented Jun 12, 2020

Description

Motivation and Context

How Has This Been Tested?

Types of changes

Components (if applicable)

Checklist:

parano commented Jun 12, 2020

yubozhao Jun 12, 2020

Choose a reason for hiding this comment

pep8speaks commented Jun 15, 2020 • edited Loading

Comment last updated at 2020-06-16 00:39:35 UTC

parano commented Jun 15, 2020

parano Jun 15, 2020

Choose a reason for hiding this comment

parano Jun 15, 2020

Choose a reason for hiding this comment

parano Jun 16, 2020

Choose a reason for hiding this comment

jackyzha0 Jun 16, 2020

Choose a reason for hiding this comment

parano Jun 16, 2020 • edited Loading

Choose a reason for hiding this comment

codecov bot commented Jun 16, 2020 • edited Loading

Codecov Report

parano Jun 16, 2020 • edited Loading

Choose a reason for hiding this comment

jackyzha0 Jun 16, 2020

Choose a reason for hiding this comment

jackyzha0 Jun 16, 2020

Choose a reason for hiding this comment

parano commented Jun 16, 2020

pep8speaks commented Jun 15, 2020 •

edited

Loading

parano Jun 16, 2020 •

edited

Loading

codecov bot commented Jun 16, 2020 •

edited

Loading

parano Jun 16, 2020 •

edited

Loading