Skip to content

Commit

Permalink
Update quickstart quides (#10593)
Browse files Browse the repository at this point in the history
Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>
  • Loading branch information
harupy committed Dec 5, 2023
1 parent abb76c9 commit db1e9e8
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 17 deletions.
18 changes: 9 additions & 9 deletions docs/source/llms/deployments/guides/step1-create-deployments.rst
Expand Up @@ -43,33 +43,33 @@ configuration file that is defined at server start, permitting dynamic route cre
.. code-block:: yaml
:name: server-config
routes:
- name: my_completions_route
route_type: llm/v1/completions
endpoints:
- name: completions
endpoint_type: llm/v1/completions
model:
provider: openai
name: gpt-3.5-turbo
config:
openai_api_key: $OPENAI_API_KEY
- name: my_chat_route_gpt_4
route_type: llm/v1/chat
- name: chat
endpoint_type: llm/v1/chat
model:
provider: openai
name: gpt-4
config:
openai_api_key: $OPENAI_API_KEY
- name: my_chat_route_gpt_3.5_turbo
route_type: llm/v1/chat
- name: chat_3.5
endpoint_type: llm/v1/chat
model:
provider: openai
name: gpt-3.5-turbo
config:
openai_api_key: $OPENAI_API_KEY
- name: my_embeddings_route
route_type: llm/v1/embeddings
- name: embeddings
endpoint_type: llm/v1/embeddings
model:
provider: openai
name: text-embedding-ada-002
Expand Down
16 changes: 8 additions & 8 deletions docs/source/llms/deployments/guides/step2-query-deployments.rst
@@ -1,5 +1,5 @@
Querying routes in the MLflow Deployment Server
===============================================
Querying endpoints in the MLflow Deployment Server
==================================================
Now that the deployment server is operational, it's time to send it some data. You can interact with the
deployments server using the deployments APIs or REST APIs. In this instance, we'll utilize the deployments APIs for simplicity.

Expand Down Expand Up @@ -33,15 +33,15 @@ various other parameters. For detailed information, please refer to the document
from mlflow.deployments import get_deploy_client
client = get_deploy_client("http://localhost:5000")
route_name = "my_completions_route"
name = "completions"
data = dict(
prompt="Name three potions or spells in harry potter that sound like an insult. Only show the names.",
n=2,
temperature=0.2,
max_tokens=1000,
)
response = client.predict(endpoint=route_name, inputs=data)
response = client.predict(endpoint=name, inputs=data)
print(response)
Expand All @@ -66,7 +66,7 @@ For further details, please consult the documentation.
from mlflow.deployments import get_deploy_client
client = get_deploy_client("http://localhost:5000")
route_name = "my_chat_route_gpt_3.5_turbo"
name = "chat_3.5"
data = dict(
messages=[
{"role": "system", "content": "You are the sorting hat from harry potter."},
Expand All @@ -77,7 +77,7 @@ For further details, please consult the documentation.
temperature=.5,
)
response = client.predict(endpoint=route_name, inputs=data)
response = client.predict(endpoint=name, inputs=data)
print(response)
Expand All @@ -97,7 +97,7 @@ respective numerical vectors. Let's proceed with an example...
from mlflow.deployments import get_deploy_client
client = get_deploy_client("http://localhost:5000")
route_name = "my_embeddings_route"
name = "embeddings"
data = dict(
input=[
"Gryffindor: Values bravery, courage, and leadership.",
Expand All @@ -107,7 +107,7 @@ respective numerical vectors. Let's proceed with an example...
],
)
response = client.predict(endpoint=route_name, inputs=data)
response = client.predict(endpoint=name, inputs=data)
print(response)
And there you have it! You've successfully set up your first Deployments Server and served three OpenAI models.

0 comments on commit db1e9e8

Please sign in to comment.