From ab88d1e072de2ef40f1c1b2e4e4132e702c13c1b Mon Sep 17 00:00:00 2001 From: Omar Sanseviero Date: Fri, 29 Mar 2024 11:20:09 +0100 Subject: [PATCH 1/2] Update quick_tour.md --- docs/source/en/quick_tour.md | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/docs/source/en/quick_tour.md b/docs/source/en/quick_tour.md index 995031d1..d07226ea 100644 --- a/docs/source/en/quick_tour.md +++ b/docs/source/en/quick_tour.md @@ -39,12 +39,12 @@ docker run --gpus all -p 8080:80 -v $volume:/data --pull always ghcr.io/huggingf -Here we pass a `revision=refs/pr/5`, because the `safetensors` variant of this model is currently in a pull request. +Here we pass a `revision=refs/pr/5` because the `safetensors` variant of this model is currently in a pull request. We also recommend sharing a volume with the Docker container (`volume=$PWD/data`) to avoid downloading weights every run. -Once you have deployed a model you can use the `embed` endpoint by sending requests: +Once you have deployed a model, you can use the `embed` endpoint by sending requests: ```bash curl 127.0.0.1:8080/embed \ @@ -72,7 +72,7 @@ volume=$PWD/data docker run --gpus all -p 8080:80 -v $volume:/data --pull always ghcr.io/huggingface/text-embeddings-inference:1.2 --model-id $model --revision $revision ``` -Once you have deployed a model you can use the `rerank` endpoint to rank the similarity between a query and a list +Once you have deployed a model, you can use the `rerank` endpoint to rank the similarity between a query and a list of texts: ```bash @@ -101,3 +101,23 @@ curl 127.0.0.1:8080/predict \ -d '{"inputs":"I like you."}' \ -H 'Content-Type: application/json' ``` + +## Batching + +You can send multiple inputs in a batch. For example, for embeddings + +```bash +curl 127.0.0.1:8080/embed \ + -X POST \ + -d '{"inputs":[["Today is a nice day"], ["I like you"]]}' \ + -H 'Content-Type: application/json' +``` + +And for Sequence Classification: + +```bash +curl 127.0.0.1:8080/predict \ + -X POST \ + -d '{"inputs":[["I like you."], ["I hate pineapples"]]}' \ + -H 'Content-Type: application/json' +``` From e31e7a3501615dca6f7d4133a0ebf9288bfd1564 Mon Sep 17 00:00:00 2001 From: Omar Sanseviero Date: Fri, 29 Mar 2024 12:05:44 +0100 Subject: [PATCH 2/2] Update docs/source/en/quick_tour.md Co-authored-by: OlivierDehaene --- docs/source/en/quick_tour.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/en/quick_tour.md b/docs/source/en/quick_tour.md index d07226ea..c0fe008c 100644 --- a/docs/source/en/quick_tour.md +++ b/docs/source/en/quick_tour.md @@ -109,7 +109,7 @@ You can send multiple inputs in a batch. For example, for embeddings ```bash curl 127.0.0.1:8080/embed \ -X POST \ - -d '{"inputs":[["Today is a nice day"], ["I like you"]]}' \ + -d '{"inputs":["Today is a nice day", "I like you"]}' \ -H 'Content-Type: application/json' ```