Skip to content
Permalink
master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Go to file
* Updating examples

* Updated integrations and internals sections

* Updated integrations and internals sections

* corrected alignment

* corrected alignment

* changed prometheus logo

* changed prometheus logo

* updated main readme

* updated main readme

* updated main readme

* Review comments

* added onnx and tensorrt

* Review comments

* Fixed links

* spelling checks

* spelling checks

* spelling checks

* added link to HF large models

* added link to HF large models

---------

Co-authored-by: Geeta Chauhan <4461127+chauhang@users.noreply.github.com>
65 contributors

Users who have contributed to this file

@harshbafna @msaroufim @maaquib @eslesar-aws @dhanainme @aaronmarkham @dhaniram-kshirsagar @chauhang @mycpuorg @HamidShojanazeri @fbbradheintz @shivamshriwas

TorchServe

TorchServe is a flexible and easy to use tool for serving and scaling PyTorch models in production.

Requires python >= 3.8

curl http://127.0.0.1:8080/predictions/bert -T input.txt

🚀 Quick start with TorchServe

# Install dependencies
# cuda is optional
python ./ts_scripts/install_dependencies.py --cuda=cu102

# Latest release
pip install torchserve torch-model-archiver torch-workflow-archiver

# Nightly build
pip install torchserve-nightly torch-model-archiver-nightly torch-workflow-archiver-nightly

🚀 Quick start with TorchServe (conda)

# Install dependencies
# cuda is optional
python ./ts_scripts/install_dependencies.py --cuda=cu102

# Latest release
conda install -c pytorch torchserve torch-model-archiver torch-workflow-archiver

# Nightly build
conda install -c pytorch-nightly torchserve torch-model-archiver torch-workflow-archiver

Getting started guide

🐳 Quick Start with Docker

# Latest release
docker pull pytorch/torchserve

# Nightly build
docker pull pytorch/torchserve-nightly

Refer to torchserve docker for details.

Why TorchServe

🤔 How does TorchServe work

🏆 Highlighted Examples

For more examples

🤓 Learn More

https://pytorch.org/serve

🫂 Contributing

We welcome all contributions!

To learn more about how to contribute, see the contributor guide here.

📰 News

💖 All Contributors

Made with contrib.rocks.

⚖️ Disclaimer

This repository is jointly operated and maintained by Amazon, Meta and a number of individual contributors listed in the CONTRIBUTORS file. For questions directed at Meta, please send an email to opensource@fb.com. For questions directed at Amazon, please send an email to torchserve@amazon.com. For all other questions, please open up an issue in this repository here.

TorchServe acknowledges the Multi Model Server (MMS) project from which it was derived