AQUA HTTP client

[mrDzurb]

Description

This PR introduces a centralized and reusable client for interacting with OCI Model Deployment. The primary motivation for this development is to address redundancy and streamline integrations with third-party frameworks like LangChain, LlamaIndex, evaluation, and other frameworks. By centralizing the client functionality within the Oracle-ADS library, we establish a single source of truth for managing and modifying the client, ensuring consistency and maintainability across all integrations.

Implementation Highlights:

• Provides both synchronous (Client) and asynchronous (AsyncClient) implementations using httpx library.
• Implements a custom OCIAuth authentication mechanism, fully integrated with OCI Signer for secure and compliant communication.
• Leverages the tenacity library for configurable retry behavior, enabling robust handling of transient errors.

Usage

Sync

Text Completion

from ads.aqua.client.client import Client
ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

client = Client(endpoint="https://<MD_OCID>/predict")
response = client.generate(
    prompt="Tell me a joke",
    payload={"model":"odsc-llm"},
    stream=False,
)
print(response)

Chat Completion

from ads.aqua.client.client import Client
ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

client = Client(endpoint="https://<MD_OCID>/predict")
response = client.generate(
    messages=[{"role": "user", "content": "Tell me a joke."}],
    payload={"model":"odsc-llm"},
    stream=False,
)
print(response)

Streaming

from ads.aqua.client.client import Client
ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

client = Client(endpoint="https://<MD_OCID>/predict")
response = client.generate(
    messages=[{"role": "user", "content": "Tell me a joke."}],
    payload={"model":"odsc-llm"},
    stream=True,
)

for chunk in response:
  print(chunk)

Embedding

from ads.aqua.client.client import Client
ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

client = Client(endpoint="https://<MD_OCID>/predict")
response = client.embeddings(
     input = ["one", "two"]
)

print(response)

Async

Text Completion

from ads.aqua.client.client import AsyncClient
ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

client = AsyncClient(endpoint="https://<MD_OCID>/predict")
response = await client.generate(
    prompt="Tell me a joke",
    payload={"model":"odsc-llm"},
    stream=False,
)
print(response)

Streaming

from ads.aqua.client.client import AsyncClient
ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

client = AsyncClient(endpoint="https://<MD_OCID>/predict")
async for chunk in await client.generate(
    prompt="Tell me a joke",
    payload={"model":"odsc-llm"},
    stream=True,
):
    print(chunk)

Embedding

from ads.aqua.client.client import AsyncClient
ads.set_auth(auth="security_token", profile="<replace-with-your-profile>")

client = AsyncClient(endpoint="https://<MD_OCID>/predict")
response = await client.embeddings(
     input = ["one", "two"]
)

print(response)

Tests

NB Example

llama_index_test_client.ipynb.zip

[github-actions]

⚠️ This PR changed pyproject.toml file. ⚠️

• PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
• PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

[github-actions]

⚠️ This PR changed pyproject.toml file. ⚠️

• PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
• PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[github-actions]

⚠️ This PR changed pyproject.toml file. ⚠️

• PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
• PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

[github-actions]

⚠️ This PR changed pyproject.toml file. ⚠️

• PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
• PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[github-actions]

⚠️ This PR changed pyproject.toml file. ⚠️

• PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
• PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[github-actions]

⚠️ This PR changed pyproject.toml file. ⚠️

• PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
• PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[github-actions]

⚠️ This PR changed pyproject.toml file. ⚠️

• PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
• PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

[github-actions]

📌 Cov diff with main:

📌 Overall coverage: