Release 0.10.0-rc.0 (#851)

* minor copy update (#838) * minor copy update * correct TensorFlow capitalization * Refactor `truss.server` out of templates to run as normal python program (#829) * [RFC][POC] Run truss server as normal python process * WIP: control server * Working draft server * Fix tests * WIP: continue fixing tests; issue with patch typing * Fix some mypy issues * Clean up some more imports * Move test files * Move some things around for readabliity * Cleanup everything except fs in tests * Only 4 tests left * Revert changes to test data * Fix all unit tests * Drop unnecessary config dump * Fix some integration tests; move packages dir * Clean up logging and patches * Fix packages path for TRT * Fix issue holding control server integration tests from passing * Fix rest of integration tests * Fix unit test and drop test_durations * Fix last integration test by passing APP_HOME and adding to path * Clean old templates, training code, deprecated serveres, and examples (#834) * [RFC][POC] Run truss server as normal python process * WIP: control server * Working draft server * Fix tests * WIP: continue fixing tests; issue with patch typing * Fix some mypy issues * Clean up some more imports * Move test files * Move some things around for readabliity * Cleanup everything except fs in tests * Only 4 tests left * Revert changes to test data * Fix all unit tests * Drop unnecessary config dump * Fix some integration tests; move packages dir * Clean up logging and patches * Fix packages path for TRT * Fix issue holding control server integration tests from passing * Fix rest of integration tests * Fix unit test and drop test_durations * Fix last integration test by passing APP_HOME and adding to path * Clean old templates, training code, deprecated serveres, and examples * reset vscode settings * Fix integration test using example * Improve `truss watch`: respect `.truss_ignore` when calculating patches (#812) Although we currently account for truss ignore patterns when detecting file changes with watchfiles.watch, we do not account for the ignore patterns when calculating patches or the truss hash and signature. This means that changes in ignored files will still be patched. This PR fixes that, along with some refactoring and bug fixes. * Fix integration tests (#843) * add pathspec to pyproject * poetry lock --no-update * fix calc_patch call * Fix os.PathLike type annotation This changes the `os.PathLike[Any]` annotation to `os.PathLike`. `os.PathLike` isn't subscriptable in Python 3.8, so `os.PathLike[Any]` results in `TypeError: 'ABCMeta' object is not subscriptable`. * Fall back to default .truss_ignore in truss upload If a user doesn't provide a .truss_ignore in their truss directory, fall back to the default .truss_ignore file specified in the truss package. * Move build configuration script to truss (#847) * Move build configuration script to truss * Update pyproject.toml * Move build configuration script to truss * Update pyproject.toml * Fix unintentially ignore build directory * Bump version to 0.10.0-rc.0 --------- Co-authored-by: Philip Kiely - Baseten <98474633+philipkiely-baseten@users.noreply.github.com> Co-authored-by: Bola Malek <bola@baseten.co> Co-authored-by: helenlyang <helen.yang@baseten.co>
basetenlabs · Mar 6, 2024 · f5cab3c · f5cab3c
1 parent 1211956
commit f5cab3c
Show file tree

Hide file tree

Showing 256 changed files with 585 additions and 7,681 deletions.
diff --git a/.test_durations b/.test_durations
diff --git a/README.md b/README.md
@@ -9,7 +9,7 @@
 
 * **Write once, run anywhere:** Package and test model code, weights, and dependencies with a model server that behaves the same in development and production.
 * **Fast developer loop:** Implement your model with fast feedback from a live reload server, and skip Docker and Kubernetes configuration with a batteries-included model serving environment.
-* **Support for all Python frameworks**: From `transformers` and `diffusors` to `PyTorch` and `Tensorflow` to `XGBoost` and `sklearn`, Truss supports models created with any framework, even entirely custom models.
+* **Support for all Python frameworks**: From `transformers` and `diffusers` to `PyTorch` and `TensorFlow` to `TensorRT` and `Triton`, Truss supports models created and served with any framework.
 
 See Trusses for popular models including:
 

diff --git a/bin/generate_base_images.py b/bin/generate_base_images.py
@@ -87,8 +87,6 @@ def _build(
 
         if job_type == "server":
             reqs_copy_from = templates_path / "server" / "requirements.txt"
-        elif job_type == "training":
-            reqs_copy_from = templates_path / "training" / "requirements.txt"
         else:
             raise ValueError(f"Unknown job type {job_type}")
 
@@ -128,7 +126,7 @@ def _build_all(
     dry_run: bool = False,
 ):
     if job_types is None:
-        job_types = ["server", "training"]
+        job_types = ["server"]
 
     if python_versions is None:
         python_versions = SUPPORTED_PYTHON_VERSIONS
@@ -173,7 +171,7 @@ def _build_all(
         "--job-type",
         nargs="?",
         default="all",
-        choices=["server", "training", "all"],
+        choices=["server", "all"],
         help="Create images for server",
     )
     parser.add_argument(
@@ -204,7 +202,7 @@ def _build_all(
         python_versions = {args.python_version}
 
     if args.job_type == "all":
-        job_types = ["server", "training"]
+        job_types = ["server"]
     else:
         job_types = [args.job_type]
 

diff --git a/docker/base_images/vllm.Dockerfile b/docker/base_images/vllm.Dockerfile
diff --git a/docs/_snippets/config-params.mdx b/docs/_snippets/config-params.mdx
@@ -299,31 +299,6 @@ Not required.
 ### `build`
 
 The `build` section is used to define options for custom servers.
-The two main model servers we support are `TGI` an `vLLM`. These are
-highly optimized servers that are built to support specific LLMs.
-
-See the following examples for how to use each of these:
-* [TGI](/examples/07-high-performance-tgi)
-* [vLLM](/examples/08-high-performance-vllm)
-
-Example configuration for TGI, running Falcon-7B:
-
-```yaml config.yaml
-build:
-  arguments:
-    endpoint: generate_stream
-    model_id: tiiuae/falcon-7b
-  model_server: TGI
-```
-
-#### `build.model_server`
-
-Either `VLLM` for vLLM, or `TGI` for TGI.
-
-#### `build.arguments`
-
-The arguments for the model server. This includes information such as which model you intend to load, and
-which endpoin from the server you'd like to use.
 
 ### `model_cache`
 

diff --git a/docs/contribute/base-images.mdx b/docs/contribute/base-images.mdx
@@ -5,7 +5,7 @@ description: "Add a custom base image to Truss"
 
 Base images are used to create Docker images for containers. Truss uses customized base images to:
 
-* Reduce Docker image build times for model serving and training
+* Reduce Docker image build times for model serving
 * Reduce container startup time
 * Re-use common parts of containers
 

diff --git a/docs/examples/07-high-performance-tgi.mdx b/docs/examples/07-high-performance-tgi.mdx