Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(feat) support azure/gpt-instruct models #2471

Merged
merged 3 commits into from Mar 12, 2024
Merged

(feat) support azure/gpt-instruct models #2471

merged 3 commits into from Mar 12, 2024

Conversation

ishaan-jaff
Copy link
Contributor

@ishaan-jaff ishaan-jaff commented Mar 12, 2024 •
edited

Azure GPT Instruct Models

Usage

import litellm

## set ENV variables
os.environ["AZURE_API_KEY"] = ""
os.environ["AZURE_API_BASE"] = ""
os.environ["AZURE_API_VERSION"] = ""

response = litellm.completion(
    model="azure_text/<your-deployment-name",
    messages=[{"role": "user", "content": "What is the weather like in Boston?"}]
)

print(response)

Azure Instruct Models

Use model="azure_text/<your-deployment>"

Model Name Function Call
gpt-3.5-turbo-instruct response = completion(model="azure_text/<your deployment name>", messages=messages)
gpt-3.5-turbo-instruct-0914 response = completion(model="azure_text/<your deployment name>", messages=messages)

@vercel Vercel
Copy link

vercel bot commented Mar 12, 2024 •
edited

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
litellm ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 12, 2024 6:06pm
litellm-dashboard ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 12, 2024 6:06pm

@railway-app Railway App
Copy link

railway-app bot commented Mar 12, 2024 •
edited

This PR is being deployed to Railway 🚅

litellm: ◻️ REMOVED

github-advanced-security[bot]
github-advanced-security bot found potential problems Mar 12, 2024
View reviewed changes
litellm/llms/azure_text.py

### CHECK IF CLOUDFLARE AI GATEWAY ###
### if so - set the model as part of the base url
if "gateway.ai.cloudflare.com" in api_base:

Check failure

Code scanning / CodeQL

Incomplete URL substring sanitization High

The string
gateway.ai.cloudflare.com
may be at an arbitrary position in the sanitized URL.
litellm/tests/test_completion.py Fixed Show fixed Hide fixed
github-advanced-security[bot]
github-advanced-security bot found potential problems Mar 12, 2024
View reviewed changes
litellm/tests/test_completion.py
messages=[{"role": "user", "content": "What is the weather like in Boston?"}],
max_tokens=10,
)
print("response", response)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
@ishaan-jaff ishaan-jaff mentioned this pull request Mar 12, 2024
Closed
@ishaan-jaff ishaan-jaff linked an issue Mar 12, 2024 that may be closed by this pull request
[Bug]: Azure Instruct Models (e.g. gpt-35-turbo-instruct) Raise Error #2414
Closed
@ishaan-jaff ishaan-jaff merged commit 01921ae into main Mar 12, 2024
6 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Bug]: Azure Instruct Models (e.g. gpt-35-turbo-instruct) Raise Error
1 participant
@ishaan-jaff