Skip to content
Permalink
Browse files

Yatai server - Add REST api support (#566)

* add bff

* update web project structure

* use post instead of get

* Add script for generated grpc js

* Add rest apis

* update bff structure

* update cli yatai command

* include web dist in manifest and update release script

* use the correct method path format

* add good example for web client

* make sure debug mode use npm watch

* make sure we have dist for web ready

* update base on comment

* create new npm script dev

* raise exception when dist folder is missing

* update tsconfig setting

* make sure include web/dist in bentoml dist

* Update DEVELOPMENT.md

* Update yarn.lock

* Delete package-lock.json

* remove .then(response => response);

* ensure node available for yatai web ui

* update tsconfig

* use webpack to bundle yatai bff

* make sure add express/grpc and their dependencies

* Add yatai web logger

* update build script

We are removing `.bin` directory, because when `pip install` it will cause `ERROR: Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory` -> reason is Long install path.   Since we don't need the .bin directory in the dist version of web ui. We are going to remove it during the build script

* nix update base on comments

* use echo for display info

* move web ui loggin to node

* add logging to yatai bff

* fix import issue for pytests

Co-authored-by: Chaoyu <paranoyang@gmail.com>
  • Loading branch information
yubozhao and parano committed Mar 15, 2020
1 parent c57a453 commit ffae39979c35fb206e8f9e008d04fc203a4a9ca1
@@ -111,11 +111,8 @@ venv.bak/


# NodeJS / Javascript
package.json
node_modules/
yarn-error.log
yarn.lock
package-lock.json

# MacOS X
.DS_Store
@@ -28,19 +28,22 @@ $ bentoml --version
```


## How to run BentoML tests
## How to run unit tests

1. Install all test dependencies:
```bash
pip install .[test]
# For zsh users, use:
$ pip install -e .\[test\]
```

2. Run all unit tests with current python version and environment
```bash
$ pytest tests
```

3. Run test under all supported python versions using Conda
## Optional: Run unit test with all supported python versions

Make sure you [have conda installed](https://docs.conda.io/projects/conda/en/latest/user-guide/install/):
```bash
@@ -59,7 +62,7 @@ $ tox -e py37
$ tox -e py36
```

## Using forks/branches of BentoML
## Installing BentoML from forks/branches

When trying new BentoML feature that has not been released, testing a fork of
BentoML on Google Colab or trying out changes in a pull request, an easy way of
@@ -159,6 +162,29 @@ gRPC Web UI available at http://127.0.0.1:60551/...
Navigate to the URL from above


## How to run and develop BentoML Web UI

Make sure you have `yarn` installed: https://classic.yarnpkg.com/en/docs/install

Install all npm packages required by BentoML Web UI:

```bash
# install npm packages required by BentoML's Node.js Web Server
cd {PROJECT_ROOT}/bentoml/yatai/web/
yarn
# install npm packages required by BentoML web frontend
cd {PROJECT_ROOT}/bentoml/yatai/web/client/
yarn
```

Build the Web Server and frontend UI code:
```bash
cd {PROJECT_ROOT}/bentoml/yatai/web/
npm run build
```


## Creating Pull Request on Github


@@ -33,5 +33,12 @@ graft bentoml/configuration
graft bentoml/migrations
include bentoml/alembic.ini

# include yatai server UI distribution files
recursive-include bentoml/yatai/web/dist *

# include ".conf" file
include bentoml/deployment/sagemaker/sagemaker_nginx.conf

# exclude e2e_tests files
prune e2e_tests
prune tests
@@ -14,64 +14,17 @@


import logging
import time
from concurrent import futures

import click
import grpc

from bentoml import config
from bentoml.cli.click_utils import _echo
from bentoml.proto.yatai_service_pb2_grpc import add_YataiServicer_to_server
from bentoml.utils.usage_stats import track_cli, track_server
from bentoml.yatai import get_yatai_service
from bentoml.cli.click_utils import CLI_COLOR_ERROR, _echo
from bentoml.exceptions import BentoMLException
from bentoml.utils.usage_stats import track_cli
from bentoml.yatai import start_yatai_service_grpc_server

logger = logging.getLogger(__name__)


_ONE_DAY_IN_SECONDS = 60 * 60 * 24


def start_yatai_service_grpc_server(yatai_service, port):
track_server('yatai-service-grpc-server')
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
add_YataiServicer_to_server(yatai_service, server)
debug_mode = config().getboolean('core', 'debug')
if debug_mode:
try:
logger.debug('Enabling gRPC server reflection for debugging')
from grpc_reflection.v1alpha import reflection
from bentoml.proto import yatai_service_pb2

SERVICE_NAMES = (
yatai_service_pb2.DESCRIPTOR.services_by_name['Yatai'].full_name,
reflection.SERVICE_NAME,
)
reflection.enable_server_reflection(SERVICE_NAMES, server)
except ImportError:
logger.debug(
'Failed enabling gRPC server reflection, missing required package: '
'"pip install grpcio-reflection"'
)

server.add_insecure_port(f'[::]:{port}')
server.start()
_echo(
f'* Starting BentoML YataiService gRPC Server\n'
f'* Debug mode: { "on" if debug_mode else "off"}\n'
f'* Running on 127.0.0.1:{port} (Press CTRL+C to quit)\n'
f'* Usage: `bentoml config set yatai_service.url=127.0.0.1:{port}`\n'
f'* Help and instructions: '
f'https://docs.bentoml.org/en/latest/concepts/yatai_service.html'
)
try:
while True:
time.sleep(_ONE_DAY_IN_SECONDS)
except KeyboardInterrupt:
_echo("Terminating YataiService gRPC server..")
server.stop(grace=None)


def add_yatai_service_sub_command(cli):
# pylint: disable=unused-variable

@@ -90,9 +43,19 @@ def add_yatai_service_sub_command(cli):
'filesystem path(POSIX/Windows), or a S3 URL, usually starts with "s3://"',
)
@click.option(
'--port', type=click.INT, default=50051, help='Port for Yatai server to use'
'--grpc-port', type=click.INT, default=50051, help='Port for Yatai server'
)
@click.option(
'--ui-port', type=click.INT, default=3000, help='Port for Yatai web UI'
)
def yatai_service_start(db_url, repo_base_url, port):
@click.option(
'--ui/--no-ui', default=True, help='Start BentoML YataiService without Web UI'
)
def yatai_service_start(db_url, repo_base_url, grpc_port, ui_port, ui):
track_cli('yatai-service-start')
yatai_service = get_yatai_service(db_url=db_url, repo_base_url=repo_base_url)
start_yatai_service_grpc_server(yatai_service, port)
try:
start_yatai_service_grpc_server(
db_url, repo_base_url, grpc_port, ui_port, ui
)
except BentoMLException as e:
_echo(f'Yatai gRPC server failed: {str(e)}', CLI_COLOR_ERROR)
@@ -48,6 +48,8 @@ prediction_log_json_format = "(service_name) (service_version) (api) (request_id
feedback_log_filename = feedback.log
feedback_log_json_format = "(service_name) (service_version) (request_id) (asctime)"

yatai_web_server_log_filename = yatai_web_server.log


[tracing]
# example: http://127.0.0.1:9411/api/v1/spans
@@ -36,6 +36,7 @@ def get_logging_config_dict(logging_level, base_log_directory):
FEEDBACK_LOG_FILENAME = conf.get("feedback_log_filename")
FEEDBACK_LOG_JSON_FORMAT = conf.get("feedback_log_json_format")


return {
"version": 1,
"disable_existing_loggers": False,
@@ -16,22 +16,31 @@
from __future__ import division
from __future__ import print_function


import atexit
import logging
import os
import subprocess
import time
from concurrent import futures

import click
import grpc

from bentoml import config

from bentoml.exceptions import BentoMLException
from bentoml.proto.yatai_service_pb2_grpc import add_YataiServicer_to_server
from bentoml.utils.usage_stats import track_server
from bentoml.yatai.utils import ensure_node_available_or_raise

logger = logging.getLogger(__name__)
_ONE_DAY_IN_SECONDS = 60 * 60 * 24


def get_yatai_service(
channel_address=None, db_url=None, repo_base_url=None, default_namespace=None
):
channel_address = channel_address or config().get('yatai_service', 'url')
if channel_address:
import grpc
from bentoml.proto.yatai_service_pb2_grpc import YataiStub

if db_url is not None:
@@ -75,4 +84,114 @@ def get_yatai_service(
)


__all__ = ["get_yatai_service"]
def start_yatai_service_grpc_server(db_url, repo_base_url, grpc_port, ui_port, with_ui):
track_server('yatai-service-grpc-server')
yatai_service = get_yatai_service(db_url=db_url, repo_base_url=repo_base_url)
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
add_YataiServicer_to_server(yatai_service, server)
debug_mode = config().getboolean('core', 'debug')
if debug_mode:
try:
logger.debug('Enabling gRPC server reflection for debugging')
from grpc_reflection.v1alpha import reflection
from bentoml.proto import yatai_service_pb2

SERVICE_NAMES = (
yatai_service_pb2.DESCRIPTOR.services_by_name['Yatai'].full_name,
reflection.SERVICE_NAME,
)
reflection.enable_server_reflection(SERVICE_NAMES, server)
except ImportError:
logger.debug(
'Failed enabling gRPC server reflection, missing required package: '
'"pip install grpcio-reflection"'
)
server.add_insecure_port(f'[::]:{grpc_port}')
server.start()
if with_ui:
web_ui_log_path = os.path.join(
config("logging").get("BASE_LOG_DIR"),
config('logging').get("yatai_web_server_log_filename"),
)

ensure_node_available_or_raise()
yatai_grpc_server_addess = f'localhost:{grpc_port}'
async_start_yatai_service_web_ui(
yatai_grpc_server_addess, ui_port, web_ui_log_path, debug_mode
)

# We don't import _echo function from click_utils because of circular dep
click.echo(
f'* Starting BentoML YataiService gRPC Server\n'
f'* Debug mode: { "on" if debug_mode else "off"}\n'
f'* Web UI: {f"running on http://127.0.0.1:{ui_port}" if with_ui else "off"}\n'
f'* Running on 127.0.0.1:{grpc_port} (Press CTRL+C to quit)\n'
f'* Usage: `bentoml config set yatai_service.url=127.0.0.1:{grpc_port}`\n'
f'* Help and instructions: '
f'https://docs.bentoml.org/en/latest/concepts/yatai_service.html\n'
f'{f"* Web server log can be found here: {web_ui_log_path}" if with_ui else ""}'
)

try:
while True:
time.sleep(_ONE_DAY_IN_SECONDS)
except KeyboardInterrupt:
logger.info("Terminating YataiService gRPC server..")
server.stop(grace=None)


def _is_web_server_debug_tools_available(root_dir):
return (
os.path.exists(os.path.join(root_dir, 'node_modules/.bin', 'concurrently'))
and os.path.exists(os.path.join(root_dir, 'node_modules/.bin', 'ts-node'))
and os.path.exists(os.path.join(root_dir, 'node_modules/.bin', 'nodemon'))
)


def async_start_yatai_service_web_ui(
yatai_server_address, ui_port, base_log_path, debug_mode
):
if ui_port is not None:
ui_port = ui_port if isinstance(ui_port, str) else str(ui_port)
web_ui_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), 'web'))
if debug_mode:
# Only when src/index.ts exists, we will run dev (nodemon)
if os.path.exists(
os.path.join(web_ui_dir, 'src/index.ts')
) and _is_web_server_debug_tools_available(web_ui_dir):
web_ui_command = [
'npm',
'run',
'dev',
'--',
yatai_server_address,
ui_port,
base_log_path,
]
else:
web_ui_command = [
'node',
'dist/bundle.js',
yatai_server_address,
ui_port,
base_log_path,
]
else:
if not os.path.isdir(os.path.join(web_ui_dir, 'dist')):
raise BentoMLException(
'Yatai web client built is missing. '
'Please run `npm run build` in the bentoml/yatai/web directory '
'and then try again'
)
web_ui_command = ['node', 'dist/bundle.js', yatai_server_address, ui_port]
web_proc = subprocess.Popen(
web_ui_command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=web_ui_dir,
)
atexit.register(web_proc.terminate)


__all__ = [
"get_yatai_service",
"start_yatai_service_grpc_server",
"async_start_yatai_service_web_ui",
]
@@ -0,0 +1,31 @@
# Copyright 2019 Atalaya Tech, Inc.

# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at

# http://www.apache.org/licenses/LICENSE-2.0

# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import subprocess

from bentoml.exceptions import BentoMLException, MissingDependencyException


def ensure_node_available_or_raise():
try:
subprocess.check_output(['node', '--version'])
except subprocess.CalledProcessError as error:
raise BentoMLException(
'Error executing node command: {}'.format(error.output.decode())
)
except FileNotFoundError:
raise MissingDependencyException(
'Node is required for Yatai web UI. Please visit '
'www.nodejs.org for instructions'
)

0 comments on commit ffae399

Please sign in to comment.
You can’t perform that action at this time.