Skip to content

Commit

Permalink
TorchServe v0.1.1 release : Merge staging_0_1_1 => Master (#427)
Browse files Browse the repository at this point in the history
* fixed resnet152 custom handler to work with GPU

* issue template

* Moving issue templates to root folder per pull request template

* adding feature request template

* Minor changes related to sections and names.

* Update feature_template.md

* Update bug_template.md

* doc template

* minor changes

* Minor changes to upload custom model or handler

* issue templates (#273)

* issue template

* Moving issue templates to root folder per pull request template

* adding feature request template

* Minor changes related to sections and names.

* Update feature_template.md

* Update bug_template.md

* doc template

* minor changes

* Minor changes to upload custom model or handler

Co-authored-by: mycpuorg <mail@mycpu.org>

* Fixes #205 - Docker install not finding GPUs (#262)

Invokes docker with  --runtime=nvidia for GPU
    Added the option to specify the GPU / specific GPU ids in start.sh
    Fixed the documentation for start.sh script
    Fixed the JDK version in Dockerfile.gpu

Co-authored-by: Aaqib <maaquib@gmail.com>

* Fix pylint issue (#163)

* pylint fixes and code cleanup

* UT fixes, pylint updates per latest version and updated sanity script to run pylint

* fixed pylint and wrapt installation issue

* freeze pylint version to 2.4.4 due to bug in version 2.5

* removed duplicate statement to install pylint

Co-authored-by: Aaqib <maaquib@gmail.com>

* Updated exception handling for worker threads shutdown. (#172)

* updated code to now log errors only in case when exception is not genreated by unregister api call

* java formatting

* fixed illigalstateexception

* removed the unrequired handling for illegal monitor state exception

Co-authored-by: dhaniram kshirsagar <dhaniram_kshirsagar@persistent.com>
Co-authored-by: Aaqib <maaquib@gmail.com>

* changes for Homebrew  issue 98 (#126)

* changes for Homebrew  issue 98

* adding changes for path in benchmark.py

* CI Build corrected

* benchmark changes

corrected path

* changed PATH for homebrew

* Update Readme

Made changes for homebrew installation and instruction for GPU based machine instruction for install_dependencies.sh

* Updated Readme.md

removed unnecessory  lines

* corrected indentation

corrected indentation for  new lines added

* Update Readme.md fil

added separate commands for CPU and GPU based

Co-authored-by: Ubuntu <ubuntu@ip-172-31-35-42.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-42-124.ec2.internal>
Co-authored-by: harshbafna <harsh_bafna@persistent.co.in>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-67.ec2.internal>
Co-authored-by: Harsh Bafna <harshbafna619@gmail.com>
Co-authored-by: Aaqib <maaquib@gmail.com>

* added changes to clean model loaded through URL from model store on unregisted

* added UTs

* Moving the plugins SDK to org.pytorch. Added plugins example. (#261)

* Moved plugins SDK to org.pytorch

* Updating the repo for publishing plugins SDK

* Enabled bintray maven repo temporarily

* Reset tests

* Added GPG plugin to pom file to sign the plugins SDK

* Update the SDK version

* Removed bintray repo

* Modify the buildspec to not deploy the SDK

Co-authored-by: Vamshidhar Dantu <dantu@amazon.com>
Co-authored-by: Aaqib <maaquib@gmail.com>
Co-authored-by: mycpuorg <mail@mycpu.org>

* Added model zoo (#299)

Co-authored-by: mycpuorg <mail@mycpu.org>
Co-authored-by: Brad Heintz <bradheintz@fb.com>

* Removed snapshot dependency on model (#152)

* Snapshot is now independent of model

* fixed checkstyle issue

* fixed formatting

* doc review: configuration and custom_service

* addressed review comments

* fixed incorrect merge

* fixed review comments and corrected the error messages

* adding remaining fixes for string constants

Co-authored-by: harshbafna <harsh_bafna@persistent.co.in>
Co-authored-by: eslesar-aws <eslesar@amazon.com>
Co-authored-by: Aaron Markham <markhama@amazon.com>
Co-authored-by: Harsh Bafna <harshbafna619@gmail.com>

* Regression Tests Suite - AWS Code Build Infra - Initial Commit for #57 (#285)

* Regression Tests Suite - AWS Code Build Infra - Initial Commit for #57

* Script to clone Pytorch Repo & Run Post Man Scripts
* Basic Tests for Inference & Managment APIs
* AWS Code Build buildspec.yml

* Refactor postman tests, add empty postman collection for todos

* Add a readme for adding tests

* Add more management tests

* Fix typo

* Snapshot Regression tests - #57

* Fix typo

* Comment out snapshot test for now

* Weird typos are showing up, try saving everything

* Need to manually save every request in postman...

* When reregister don't use s3

* More workarounds for rereg

* Fix test messages

* Remove test for empty reqs

* Change get to post

* Fix pytest for subprocess - #57

* Enable pytest - #57

* Inference API Testcases / Rename Management Testcases  #57

* Wipe Model MARs in Modelstore between tests #54

* Fix Inference Endpoint Execution #54

* Fix file paths - #57

* Fix Initial Workers #57

* Fix Inference Assertions #57

* Fix DenseNet Inference #57

* Fix Densenet Register #54

* Redirect Logs to File #54

* Fix Test Execution Log #54

* README, Build Badge #57

* Format ReadMe #57

* Update README - Add Instruction to add new tests / Run tests locally

* Update README

* Update branch for build Codebuild Badge

Co-authored-by: alexwong <11878166+alexwong@users.noreply.github.com>
Co-authored-by: Aaqib <maaquib@gmail.com>
Co-authored-by: mycpuorg <mail@mycpu.org>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-178.us-west-2.compute.internal>

* Issue 312 install from src (#322)

* documentation: fixed issues in installation and quick start in README.md

* documentation: Reverting conda related doc changes from PR#286

Fixes #297

* remove minus y (#311)

update pip install instructions

* Update instructions for installation from source

This commit aims to address Issue #312

Tested on a fresh EC2 P3.2 instance:
* Followed instructions under
https://github.com/pytorch/serve#install-torchserve-for-development
* Started the server manually from command line
* Ran a ping query

Received a response from server with "Healthy" status

Signed-off-by: Manoj Rao <mail@mycpu.org>

* add separate files for installation

one each for linux and macos

* update README and remove relative path from script

* Install script - Dont uninstall if not previously installed - #132

* explicitly build the source code with gradle prior to install

* remove upgrade instructions in favor of clean reinstall from source

* update instructions to contain the correct script name

Co-authored-by: eslesar-aws <eslesar@amazon.com>
Co-authored-by: Aaqib Ansari <maaquib@gmail.com>
Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com>
Co-authored-by: Aaron Markham <markhama@amazon.com>
Co-authored-by: Manoj Rao <mail@mycpu.org>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-0-61.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-7-197.us-west-2.compute.internal>

* fix for macos script (#338)

* Move Issue Template to .github dir

* Updated sanity script to use install_from_src_ubuntu script (#350)

* updated sanity script to use install_from_src_ubuntu script

* added java version check

* fixed issue with if statement

* fixed variable name

* fixed repeated cd command

* updated artifacts

* removed artifacts from buildspec

* Corrected HTTP response code for Model version does not exists exception (#282)

* issue 196 patch

* Added openapi missing responses. - incorporated review comments

* updated method signature after resolving merge conflicts

Co-authored-by: Harsh Bafna <harshbafna619@gmail.com>
Co-authored-by: Aaqib <maaquib@gmail.com>
Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com>
Co-authored-by: harshbafna <harsh_bafna@persistent.co.in>
Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com>

* Corrected paths in command and added notes (#354)

Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com>

* Refactored UT for better reporting (#159)

* refactored UTs

* upgraded testng version and updated testcase names

* added testng.xml to handle sequential execution of test classes

* added waveglow example

* Updated readme for waveglow example

* fixed failing UTs after merging master

* removed unintentionally pushed waveglow example

* fixed failing test case

* moved set-default to utils and added couple of more add cases for default version

* java formatting

* corrected set default negative test case

* fixed java formatting

Co-authored-by: Aaqib <maaquib@gmail.com>
Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com>

* Huggingface Transformers generalization (#302)

* Scripts to seriliaze a Bert model using jit.trace and a handler for Bert model

* modified to include mapping of predicitons and decoding received text

* move back the original text

* moved the index_to_name to bert folder

* Transformers/ Bert serving with Torchserve readme

* deleting an unused dictionary

* just cleaning some of the commented out parts

* cleaning up

* Cleaning up

* uncommenting the logging and cleaning  the comments

* Download and save the checkpoints, config file and vocab of  the BERT from Transformers for demonstration purposes

* mapping of predicitons to labels

* a custom handler for transformers.

* README.

* sample of input text.

* modifying the descriptions

* modifying the descriptions

* adding comments

* adding commnets

* Fromatting modifications

* modified comments

* added the citations

* adding hugginface as the type of transformer model we are referring to

* renaming the folder to huggingface_transformers

* Removing the bert folder

* simple modification

* modifed the script to use a setup_config.json for loading and saving models

* added the section for transfomer_handler_generalized

* the generalized handler

* set_up config intialization

* edited for transfomer generalization

* modified to use just input_ids as inputs for models

* modified for question answering

* modified to include the number of labels

* corrected typos

* modifed variable names

* edited for clarity

* modifed variable names, simplified the initlialize in the handler class, removed the warnning from QA case

* modifed values for QA

* modifed the variable names

* added comments

* modified to add clarity

* add comment in preprocessing of QA

* added warning for the case mode is not selected properly and combined constructor and initlaize fucntion

* clean up

* clean up

* combined constructor and initliaze func

* combined constructor and initliaze func

* Separated the constructor and initliaze func

* Modified to remove the classification part and update with generalized version

* resolved the coflict and separated the costructor and initliaze func

* removed the handler for classification case as the generalized version handles the case

Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com>
Co-authored-by: mycpuorg <mail@mycpu.org>

* added waveglow example (#258)

* added waveglow example

* Updated readme for waveglow example

* enhanced example to remove runtime checkpoint download

* updated handler to return the audio file content instead of dumping it to tmp dir

* minor change to handler

* added pipefail

* added python snippet to run inference

* updated readme with know issue with the example

Co-authored-by: Aaqib <maaquib@gmail.com>

* made build suite compatible with latest gradle (#348)

Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com>
Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com>
Co-authored-by: Aaqib <maaquib@gmail.com>

* moved env section after context

* Deleting redundant ISSUE_TEMPLATE at root dir

* Fix typo in scripts/install_from_src_macos

* Corrected  OpenApi description  (#328)

* Correct query parameter name (Registration Api)in OpenApi description

* removed non existing path parameter

Co-authored-by: Aaqib <maaquib@gmail.com>

* Regression suite fixes and added test cases (#344)

* added test model zoo test cases
updated regression script

* Updated Readme

* update -y flag for install

* Updated docker command as per new imagename and tag generated in stagging branch

* Fix sudo usage in regression test script

* - addressed review comments
- reverted changes related to sudo in regression script.

* added https test collection

* - updated failed test cases on staging branch
- added environment and updated newman cmd  with --insecure for https

* updated script

* updated pytest

* update curl check for serve up

* Revert "update curl check for serve up"

This reverts commit 0047eae.

* Revert "update curl check for serve up"

This reverts commit 0047eae.

- update model zoo
- added resnet test case

* added workers to resnet model

* Update readme to run test suite on desired branch

* enhanced inference test cases to driven by datafile

* Updated readme

Co-authored-by: Dhanasekar Karuppasamy <dhanak@amazon.com>
Co-authored-by: Aaqib <maaquib@gmail.com>
Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com>

* Fix typo in mnist readme (#330)

Co-authored-by: Aaqib <maaquib@gmail.com>

* Fixed typo in ab installation. (#327)

* documentation: fixed issues in installation and quick start in README.md

* documentation: Reverting conda related doc changes from PR#286

Fixes #297

* remove minus y (#311)

update pip install instructions

* Fixed typo in ab installation.

Co-authored-by: eslesar-aws <eslesar@amazon.com>
Co-authored-by: Aaqib Ansari <maaquib@gmail.com>
Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com>
Co-authored-by: Aaron Markham <markhama@amazon.com>

* Setup Nightly Build for TorchServe (#371)

* Add Branch Flag for Nightly CodeBuild #57

* Update ENV Variable for Nightly Build Name

* Nigtly Build - Fix FileName for Build Artifacts

* Update README for accessing test logs

* Whitspace changes

* Build new images for build - README Instruction

* Fix for intermittent gradle build failure (#352)

* Fix for intermittent null pointer exception

* Fixed comments

Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com>
Co-authored-by: Aaqib <maaquib@gmail.com>

* documentation: Remove redundant curl options in example (#336)

* documentation: fixed issues in installation and quick start in README.md

* documentation: Reverting conda related doc changes from PR#286

Fixes #297

* remove minus y (#311)

update pip install instructions

* fix-doc

* fix remaining "-X POST"s

Co-authored-by: eslesar-aws <eslesar@amazon.com>
Co-authored-by: Aaqib Ansari <maaquib@gmail.com>
Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com>
Co-authored-by: Aaron Markham <markhama@amazon.com>

* Corrected port number validation range (#304)

* Corrected port number validation range

* added UT for issue 291

* added ut dependency file formatting

* correct param in config_test_env.properties

* removed incorrect flags from test case config files

* added negative test cases

* removed properties file and changes in test case

Co-authored-by: Harsh Bafna <harshbafna619@gmail.com>

* Fixes version number reporting for TorchServe (#360)

* Correct version and added vesion.txt for getting version info

* fixed description test cases and build issue

* Added snapshot test fixes
and resolved relative path issue for version.txt

* python formatting

* added version.txt in MANIFEST.in

* addressed review comments

* added version.txt for model archiver as well
- addressed review comments

* -fixed python UT
-added ts python UT run in sanity script

* removed blank lines from snapshot files

Co-authored-by: Aaqib <maaquib@gmail.com>
Co-authored-by: Harsh Bafna <harshbafna619@gmail.com>

* fixes installation script exits without error if no java present on machine (#376)

* fixes installation script exits without error if no java present on machine

* added changes for macos script

Co-authored-by: Aaqib <maaquib@gmail.com>

* Dockerfile enhancements (#308)

* dockerfile now points to latest torchserve on pypi and added separate files for dev

* Minor changes

* Made build and run commands consistent

* Fixing formatting

* Consistency issue

* Enhanced Dockerfile for binary per #178 and other misc.

* Fixed gpu related issues in start script

* Corrected the heading font

* Added mar creation steps from container

* added buildkit support for dev docker files

* Fix for buildkit

* Restoring psutil and future as absence leads intermittent hangged build

* Fixed review comments for doc

* Fixed typo

Co-authored-by: Ubuntu <ubuntu@ip-172-31-64-54.ec2.internal>
Co-authored-by: Aaqib <maaquib@gmail.com>

* Updated configuration.md for missing config properties (#337)

* Updated configuration.md for missing parameters

* changes TS_MODEL_SNAPSHOT to MODEL_SNAPSHOT
- removes ambiguity
- follows naming convention

* addressed review comments.

* removed debug property

* updated configuration.md

Co-authored-by: Harsh Bafna <harshbafna619@gmail.com>
Co-authored-by: Aaqib <maaquib@gmail.com>

* Adding missing document links to README (#379)

* Maven tests should output only errors (#326)

Co-authored-by: Vamshidhar Dantu <dantu@amazon.com>
Co-authored-by: Aaqib <maaquib@gmail.com>

* fixed code for build failure in python virtual environment (#341)

* fixed code for build failure in python virtual environment

* added exception missed on resolving conflicts

Co-authored-by: Aaqib <maaquib@gmail.com>
Co-authored-by: Vamshidhar Dantu <36211508+vdantu@users.noreply.github.com>

* updated object detector handler to use only default cuda device for torchvision version less then 0.6 (#265)

* updated object detector handler to use default cuda device for torchvision version 0.5.0

* updated version check logic

* Updated documentation for torchvision version dependency

* enhanced sanity script to validate all default handlers

* fixed sanity script

* added torchtext install step in sanity script

* fixed issue related to inference after restart with snapshot.

* moved torchtext to codebuild docker files

* added more log statements

* enhanced run inference function

* fixed register model function

* added unregister model function

* incorporated code review comments

* added back inference call for snapshot test

* fixed if block

Co-authored-by: Aaqib <maaquib@gmail.com>

* Adding a barebones ec2 template to run torchserve on ubuntu (#301)

* Adding a barebones ec2 template to run torchserve on ubuntu

* Adding VPC with Private subnet

* Address review comments #1

* Address review comments #2

* Rebase for #365 (#389)

* Updated README.md for #364

See #364 for a deep description of the problem

* updated README.md

* updated README.md

Co-authored-by: Francesco Saverio Zuppichini <zuppif@usi.ch>

* Document the docker settings for running in production mode (#388)

* Docker Production Options #234

* Docker Production Options - Typo Fixes / Fit & Finish  #234

* Typos - Docker Docs #234

* grammar check  - docker/README.md

Co-authored-by: Aaqib <maaquib@gmail.com>

* grammar check  - docker/README.md

Co-authored-by: Aaqib <maaquib@gmail.com>

* grammar check  - docker/README.md

Co-authored-by: Aaqib <maaquib@gmail.com>

Co-authored-by: Aaqib <maaquib@gmail.com>

* Huggingface transformers integration test (#382)

* Add Huggingface transformers example integration tests

* Add transformers intallation

* Fix a missing bracket

* move transformers installation to chain installations

Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com>

* Updated docker README.md (#401)

* Update README.md

* Fixed cd command

* Made source cloning steps explicit!!

* Fixed online editing mistake

Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com>

* Fixed response status message for inferencing on model with zero workers (#305)

* Changes for issue_29

* Changed error Messages

* changed message for scalling request

* changed defaultVersion string

* Corrected error message

* changed the code for best practice

* resolved review comment for version

* updated management api doc as per new responses in worker scale api

Co-authored-by: Ubuntu <ubuntu@ip-172-31-80-124.ec2.internal>
Co-authored-by: Harsh Bafna <harshbafna619@gmail.com>
Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com>
Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com>
Co-authored-by: Aaqib <maaquib@gmail.com>

* made model server port tests independent (#396)

* Added backend profiling and fixed benchmark's install dependency scripts (#402)

* added backend profiling
* updated install dependency scripts to make them jmeter version independent
* updated readme
* Fix benchmarks/README typo and JMeter versioning issue
* updated documentation

Co-authored-by: Aaqib Ansari <maaquib@gmail.com>

* Add sphinx directory and conf.py (#274)

* Add sphinx folder that contains files for docs build

* Delete .DS_Store

* Delete .DS_Store

* Add conf.py in docs/

* documentation: fixed issues in installation and quick start in README.md

* documentation: Reverting conda related doc changes from PR#286

Fixes #297

* remove minus y (#311)

update pip install instructions

* Update to the latest version of sphinx

* Add recommonmark to extensions

Add recommonmark to extensions so sphinx project has markdown support.
Configuration details here: https://www.sphinx-doc.org/en/master/usage/markdown.html

* Update source suffix to use latest recommonmark syntax

Updating to the syntax found here: https://www.sphinx-doc.org/en/master/usage/markdown.html

Co-authored-by: Aaron Markham <markhama@amazon.com>

* Remove old recommonmark parser syntax

* Pin sphinx version to work with latest version of recommonmark

Co-authored-by: Aaron Markham <markhama@amazon.com>

* Pin version latest version recommonmark

Co-authored-by: Aaron Markham <markhama@amazon.com>

* Delete theme file generated during build

* de-duplicate install package of psutil (#329)

The Dockerfile of GPU install package psutil duplicated.

* Remove pytorch-sphinx-theme files

* Update to match latest staging_0_1_1

* Update docker directory to match staging_0_1_1 branch

* Delete .DS_Store

* Add sphinxcontrib.katex and torch

Co-authored-by: eslesar-aws <eslesar@amazon.com>
Co-authored-by: Aaqib Ansari <maaquib@gmail.com>
Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com>
Co-authored-by: Aaron Markham <markhama@amazon.com>
Co-authored-by: Mario Cho <hephaex@gmail.com>
Co-authored-by: mycpuorg <mail@mycpu.org>

* Image classifier example readme fixes (#405)

* Changed the model name in torch.jit.save to be consistent with docs

* Remove extra handler flag and fix the torchservr start

* fix the doc

* fix the doc

Co-authored-by: mycpuorg <mail@mycpu.org>

* Porting memory fix changes to TorchServe (#422)

Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com>

* Benchmark documentation fix (#425)

* Fix Benchmark Documentation

* Documentaion fixes

* .circleci: Add initial circleci configuration (#431)

Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

* Torchscript support huggingface transfrmers (#421)

* Add support for torchscript

* Add support for torchscript

* Add key values for torchscript support

* Add Torchscript suport

* Add Torchscript support

* Add Torchscript support in loading the models

* keep the example to QA

* modify to reflect Torchscript example?

* Add torchscript support guides

* Add fintued BERT for QA

* Add QA artifacts

* Add seqenece classification artifacts

* Token classification artifacts

* Correct typos

* Add max length to tokenizer

* clean up

* enhance the doc

* change setup to seq classification

* Add detail to torchscript support

* set the model for QA

* Add torchscript bert seq classification example

* Add  bert seq classification example

* Modify the packaging model command

* Modify torchscript packaging command

* clean up

* clean up

* correct spacing between list items

Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com>
Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com>

* Integration test huggingface transfromers torchscript support (#433)

* made model server port tests independent

* Add integration tests for Huggingface transfromers with new handler that support torchscript

Co-authored-by: harshbafna <harsh_bafna@persistent.co.in>
Co-authored-by: mycpuorg <mail@mycpu.org>
Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com>
Co-authored-by: dhanainme <60679183+dhanainme@users.noreply.github.com>

* Apachebench benchmark - cleanup script / documentation

* Revert "Apachebench benchmark - cleanup script / documentation"

This reverts commit c8f8682.

* Documentation updates for clarity on Eager mode / Torch scripted models (#439)

* Model Zoo updates for Eager Mode / Torchscripted models

* Update examples/README.md

Co-authored-by: Aaqib <maaquib@gmail.com>

Co-authored-by: Aaqib <maaquib@gmail.com>

* Apachebench benchmarking - Script / Documentation updates (#438)

* Apachebench benchmarking script - Script / Documentation updates

* Stop TS after benchmark completion.

Co-authored-by: harshbafna <harsh_bafna@persistent.co.in>
Co-authored-by: dhaniram-kshirsagar <26479924+dhaniram-kshirsagar@users.noreply.github.com>
Co-authored-by: dhaniram kshirsagar <dhaniram_kshirsagar@persistent.com>
Co-authored-by: Aaqib <maaquib@gmail.com>
Co-authored-by: mycpuorg <mail@mycpu.org>
Co-authored-by: Harsh Bafna <harshbafna619@gmail.com>
Co-authored-by: mjpsl <57522192+mjpsl@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-35-42.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-42-124.ec2.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-16-67.ec2.internal>
Co-authored-by: Vamshidhar Dantu <36211508+vdantu@users.noreply.github.com>
Co-authored-by: Vamshidhar Dantu <dantu@amazon.com>
Co-authored-by: Brad Heintz <bradheintz@fb.com>
Co-authored-by: eslesar-aws <eslesar@amazon.com>
Co-authored-by: Aaron Markham <markhama@amazon.com>
Co-authored-by: alexwong <11878166+alexwong@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-12-178.us-west-2.compute.internal>
Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-0-61.us-west-2.compute.internal>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-7-197.us-west-2.compute.internal>
Co-authored-by: dkasar-aws <57526278+dkasar-aws@users.noreply.github.com>
Co-authored-by: Shivam Shriwas <shivamshriwas21@gmail.com>
Co-authored-by: Hamid Shojanazeri <hamid.nazeri2010@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-91.us-west-2.compute.internal>
Co-authored-by: shivamshriwas <44968110+shivamshriwas@users.noreply.github.com>
Co-authored-by: Takayoshi Nishida <takayoshi.nishida@gmail.com>
Co-authored-by: alvations <alvations@gmail.com>
Co-authored-by: Tzu-Wei Huang <huang.dexter@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-64-54.ec2.internal>
Co-authored-by: Francesco Saverio Zuppichini <zuppif@usi.ch>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-80-124.ec2.internal>
Co-authored-by: Jessica Lin <jplin@fb.com>
Co-authored-by: Mario Cho <hephaex@gmail.com>
Co-authored-by: Eli Uriegas <1700823+seemethere@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-78-248.ec2.internal>
  • Loading branch information
1 parent 3761100 commit 2efec61
Show file tree
Hide file tree
Showing 182 changed files with 8,219 additions and 1,902 deletions.
6 changes: 6 additions & 0 deletions .circleci/Dockerfile
@@ -0,0 +1,6 @@
FROM ubuntu:18.04 as dev
RUN apt-get update && apt-get install -y openjdk-11-jdk
COPY --from=continuumio/miniconda3:4.8.2 /opt/conda /opt/conda
ENV PATH="/opt/conda/bin/:${PATH}"
ARG PYTORCH_CHANNEL=pytorch
RUN conda install -c "${PYTORCH_CHANNEL}" -c powerai -y pytorch torchvision torchtext cpuonly psutil
74 changes: 74 additions & 0 deletions .circleci/config.yml
@@ -0,0 +1,74 @@
version: 2.1

jobs:
build_docker_environment:
parameters:
pytorch_channel:
type: string
default: "pytorch"
machine:
image: ubuntu-1604:201903-01
environment:
- DOCKER_BUILDKIT=1
steps:
- checkout
- run:
name: Building docker image
command: |
cat .circleci/Dockerfile | docker build \
-t pytorch/serve-circleci-dev \
--progress=plain \
--build-arg PYTORCH_CHANNEL="<< parameters.pytorch_channel >>" -
- run:
name: Save docker image to archive
command: mkdir /tmp/workspace && docker image save pytorch/serve-circleci-dev -o /tmp/workspace/dev-image.tar
- persist_to_workspace:
root: /tmp/workspace
paths:
- dev-image.tar
build_wheel:
parameters:
build_directory:
type: string
default: "."
machine:
image: ubuntu-1604:201903-01
steps:
- attach_workspace:
at: /tmp/workspace
- run:
name: Loading docker image
command: cat /tmp/workspace/dev-image.tar | docker load
- checkout
- run:
name: Build wheel
command: |
docker run --rm -i \
-v "$(pwd):/v" \
-w "/v/<< parameters.build_directory >>" \
pytorch/serve-circleci-dev \
python setup.py bdist_wheel --release --universal
- run:
name: Chown workspace back to circleci user
command: |
docker run --rm -v "$(pwd)":/v -w /v alpine chown -R "$(id -u):$(id -g)" .
- store_artifacts:
path: << parameters.build_directory >>/dist/
- persist_to_workspace:
root: << parameters.build_directory >>/dist
paths:
- "*.whl"

workflows:
build:
jobs:
- build_docker_environment
- build_wheel:
name: wheel_torchserve
requires:
- build_docker_environment
- build_wheel:
name: wheel_model_archiver
requires:
- build_docker_environment
build_directory: "model-archiver"
51 changes: 51 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_template.md
@@ -0,0 +1,51 @@
---
name: "\U0001F41B Bug report"
about: Create a report to help us improve

---

Your issue may already be reported!
Please search on the [issue tracker](https://github.com/pytorch/serve/issues) before creating one.

## Context
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->
* torchserve version:
* torch version:
* torchvision version [if any]:
* torchtext version [if any]:
* torchaudio version [if any]:
* java version:
* Operating System and version:

## Your Environment
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Installed using source? [yes/no]:
* Are you planning to deploy it using docker container? [yes/no]:
* Is it a CPU or GPU environment?:
* Using a default/custom handler? [If possible upload/share custom handler/model]:
* What kind of model is it e.g. vision, text, audio?:
* Are you planning to use local models from model-store or public url being used e.g. from S3 bucket etc.?
[If public url then provide link.]:
* Provide config.properties, logs [ts.log] and parameters used for model registration/update APIs:
* Link to your project [if any]:

## Expected Behavior
<!--- If you're describing a bug, tell us what should happen -->

## Current Behavior
<!--- If describing a bug, tell us what happens instead of the expected behavior -->

## Possible Solution
<!--- Not obligatory, but suggest a fix/reason for the bug -->

## Steps to Reproduce
<!--- Provide a link to a live example, or an unambiguous set of steps to -->
<!--- reproduce this bug. Include code to reproduce, if relevant -->
1.
2.
...

## Failure Logs [if any]
<!--- Provide any relevant log snippets or files here. -->

9 changes: 9 additions & 0 deletions .github/ISSUE_TEMPLATE/doc_template.md
@@ -0,0 +1,9 @@
---
name: "\U0001F4DA Documentation"
about: Report an issue related to https://pytorch.org/serve/

---

## 📚 Documentation

<!-- A clear and concise description of what content in https://pytorch.org/serve/ is an issue. If this has to do with the general https://pytorch.org website, please file an issue at https://github.com/pytorch/pytorch.github.io/issues/new/choose instead. If this has to do with https://pytorch.org/tutorials, please file an issue at https://github.com/pytorch/tutorials/issues/new -->
20 changes: 20 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_template.md
@@ -0,0 +1,20 @@
---
name: "\U0001F680 Feature request"
about: Suggest an idea for this project

---

<!--
Thank you for suggesting an idea to improve torchserve model serving experience.
Please fill in as much of the template below as you're able.
-->

## Is your feature request related to a problem? Please describe.
<!-- Please describe the problem you are trying to solve. -->

## Describe the solution
<!-- Please describe the desired behavior. -->

## Describe alternatives solution
<!-- Please describe alternative solutions or features you have considered. -->
4 changes: 4 additions & 0 deletions .gitignore
@@ -0,0 +1,4 @@
build/
dist/
ts/
*.egg-info/
1 change: 1 addition & 0 deletions MANIFEST.in
@@ -1,3 +1,4 @@
include ts/frontend/model-server.jar
include PyPiDescription.rst
include ts/configs/*
include ts/version.txt
93 changes: 16 additions & 77 deletions README.md
Expand Up @@ -100,42 +100,21 @@ Now you are ready to [package and serve models with TorchServe](#serve-a-model).

If you plan to develop with TorchServe and change some of the source code, you must install it from source code.

1. Install Java 11

```bash
sudo apt-get install openjdk-11-jdk
```

1. Install dependencies

```bash
pip install psutil future
```

1. Clone the repo

```bash
git clone https://github.com/pytorch/serve
cd serve
```
Please deactivate any conda env that you might be within.
Run the following script from the top of the source directory.

1. Make your changes executable
NOTE: This script uninstalls existing `torchserve` and `torch-model-archiver` installations

```bash
pip install -e .
```
#### For Debian Based Systems
Verified on EC2 instances running Ubuntu DL AMI 28.x

* To develop with torch-model-archiver:

```bash
cd serve/model-archiver
pip install -e .
```
./scripts/install_from_src_ubuntu
```
#### For MAC OS

* To upgrade TorchServe or model archiver from source code and make changes executable, run:

```bash
pip install -U -e .
```
./scripts/install_from_src_macos
```

For information about the model archiver, see [detailed documentation](model-archiver/README.md).
Expand Down Expand Up @@ -207,7 +186,7 @@ The following code completes all three steps:

```bash
curl -O https://s3.amazonaws.com/model-server/inputs/kitten.jpg
curl -X POST http://127.0.0.1:8080/predictions/densenet161 -T kitten.jpg
curl http://127.0.0.1:8080/predictions/densenet161 -T kitten.jpg
```

The predict endpoint returns a prediction response in JSON. It will look something like the following result:
Expand Down Expand Up @@ -247,58 +226,15 @@ torchserve --stop
You see output specifying that TorchServe has stopped.

## Quick Start with Docker

### Prerequisites

* docker - Refer to the [official docker installation guide](https://docs.docker.com/install/)
* git - Refer to the [official git set-up guide](https://help.github.com/en/github/getting-started-with-github/set-up-git)
* TorchServe source code. Clone and enter the repo as follows:

```bash
git clone https://github.com/pytorch/serve.git
cd serve
```

### Build the TorchServe Docker image

The following are examples on how to use the `build_image.sh` script to build Docker images to support CPU or GPU inference.

To build the TorchServe image for a CPU device using the `master` branch, use the following command:

```bash
./build_image.sh
```

To create a Docker image for a specific branch, use the following command:

```bash
./build_image.sh -b <branch_name>
```

To create a Docker image for a GPU device, use the following command:

```bash
./build_image.sh --gpu
```

To create a Docker image for a GPU device with a specific branch, use following command:

```bash
./build_image.sh -b <branch_name> --gpu
```

To run your TorchServe Docker image and start TorchServe inside the container with a pre-registered `resnet-18` image classification model, use the following command:

```bash
./start.sh
```
Refer [torchserve docker](docker/README.md) for details.

## Learn More

* [Full documentation on TorchServe](docs/README.md)
* [Manage models API](docs/management_api.md)
* [Inference API](docs/inference_api.md)
* [Package models for use with TorchServe](model-archiver/README.md)
* [TorchServe model zoo for pre-trained and pre-packaged models-archives](docs/model_zoo.md)

## Contributing

Expand All @@ -307,3 +243,6 @@ We welcome all contributions!
To learn more about how to contribute, see the contributor guide [here](https://github.com/pytorch/serve/blob/master/CONTRIBUTING.md).

To file a bug or request a feature, please file a GitHub issue. For filing pull requests, please use the template [here](https://github.com/pytorch/serve/blob/master/pull_request_template.md). Cheers!


*TorchServe acknowledges the [Multi Model Server (MMS)](https://github.com/awslabs/multi-model-server) project from which it was derived*

0 comments on commit 2efec61

Please sign in to comment.