From bf01b37238d577b554f90d5272025bc5b5f87309 Mon Sep 17 00:00:00 2001 From: Cyrus Leung Date: Thu, 9 May 2024 11:11:43 +0000 Subject: [PATCH] Move sampling parameters to offline inference docs --- docs/source/index.rst | 2 +- docs/source/{dev => offline_inference}/sampling_params.rst | 2 +- docs/source/serving/openai_compatible_server.md | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) rename docs/source/{dev => offline_inference}/sampling_params.rst (76%) diff --git a/docs/source/index.rst b/docs/source/index.rst index d25293e70b9..e1e81778dbd 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -72,6 +72,7 @@ Documentation :caption: Offline Inference offline_inference/llm + offline_inference/sampling_params .. toctree:: :maxdepth: 1 @@ -107,7 +108,6 @@ Documentation :maxdepth: 2 :caption: Developer Documentation - dev/sampling_params dev/engine/engine_index dev/kernel/paged_attention dev/dockerfile/dockerfile diff --git a/docs/source/dev/sampling_params.rst b/docs/source/offline_inference/sampling_params.rst similarity index 76% rename from docs/source/dev/sampling_params.rst rename to docs/source/offline_inference/sampling_params.rst index ef3d1509bda..cff8dde55e1 100644 --- a/docs/source/dev/sampling_params.rst +++ b/docs/source/offline_inference/sampling_params.rst @@ -1,4 +1,4 @@ -Sampling Params +Sampling Parameters =============== .. autoclass:: vllm.SamplingParams diff --git a/docs/source/serving/openai_compatible_server.md b/docs/source/serving/openai_compatible_server.md index c157d8ba998..ad8834fae74 100644 --- a/docs/source/serving/openai_compatible_server.md +++ b/docs/source/serving/openai_compatible_server.md @@ -48,7 +48,7 @@ completion = client.chat.completions.create( ``` ### Extra Parameters for Chat API -The following [sampling parameters (click through to see documentation)](../dev/sampling_params.rst) are supported. +The following [sampling parameters (click through to see documentation)](../offline_inference/sampling_params.rst) are supported. ```{literalinclude} ../../../vllm/entrypoints/openai/protocol.py :language: python @@ -65,7 +65,7 @@ The following extra parameters are supported: ``` ### Extra Parameters for Completions API -The following [sampling parameters (click through to see documentation)](../dev/sampling_params.rst) are supported. +The following [sampling parameters (click through to see documentation)](../offline_inference/sampling_params.rst) are supported. ```{literalinclude} ../../../vllm/entrypoints/openai/protocol.py :language: python