Skip to content

Commit bd41683

Browse files
Python: add Foundry local sample (#12214)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> Sample on how to use Foundry Local with SK. ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄
1 parent cef5ae7 commit bd41683

File tree

1 file changed

+101
-0
lines changed

1 file changed

+101
-0
lines changed
Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# Copyright (c) Microsoft. All rights reserved.
2+
3+
import asyncio
4+
5+
from foundry_local import FoundryLocalManager
6+
from openai import AsyncOpenAI
7+
8+
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion, OpenAIChatPromptExecutionSettings
9+
from semantic_kernel.contents import ChatHistory
10+
11+
"""
12+
This samples demonstrates how to use the Foundry Local model with the OpenAIChatCompletion service.
13+
The Foundry Local model is a local model that can be used to run the OpenAIChatCompletion service.
14+
To use this sample, you need to install the Foundry Local SDK and service.
15+
For the service refer to this guide: https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-local/get-started
16+
To install the SDK, run the following command:
17+
`pip install foundry-local-sdk`
18+
"""
19+
20+
# The way Foundry Local works, is that it picks the right variant of a model based on the
21+
# hardware available on the machine. For example, if you have a GPU, it will pick the GPU variant
22+
# of the model. If you have a CPU, it will pick the CPU variant of the model.
23+
# The model alias is the name of the model that you want to use.
24+
model_alias = "phi-4-mini"
25+
manager = FoundryLocalManager(model_alias)
26+
# next, download the model to the machine
27+
manager.download_model(model_alias)
28+
# load the model into memory
29+
manager.load_model(model_alias)
30+
31+
service = OpenAIChatCompletion(
32+
ai_model_id=manager.get_model_info(model_alias).id,
33+
async_client=AsyncOpenAI(
34+
base_url=manager.endpoint,
35+
api_key=manager.api_key,
36+
),
37+
)
38+
# if needed, set the other parameters for the execution
39+
request_settings = OpenAIChatPromptExecutionSettings()
40+
# This is the system message that gives the chatbot its personality.
41+
system_message = """
42+
You are a chat bot. Your name is Mosscap and
43+
you have one goal: figure out what people need.
44+
Your full name, should you need to know it, is
45+
Splendid Speckled Mosscap. You communicate
46+
effectively, but you tend to answer with long
47+
flowery prose. Use the tools you have available!
48+
"""
49+
50+
# Create a chat history object with the system message.
51+
chat_history = ChatHistory(system_message=system_message)
52+
53+
54+
async def chat() -> bool:
55+
try:
56+
user_input = input("User:> ")
57+
except KeyboardInterrupt:
58+
print("\n\nExiting chat...")
59+
return False
60+
except EOFError:
61+
print("\n\nExiting chat...")
62+
return False
63+
64+
if user_input == "exit":
65+
print("\n\nExiting chat...")
66+
return False
67+
68+
# Add the user message to the chat history so that the chatbot can respond to it.
69+
chat_history.add_user_message(user_input)
70+
71+
# Get the chat message content from the chat completion service.
72+
response = await service.get_chat_message_content(
73+
chat_history=chat_history,
74+
settings=request_settings,
75+
)
76+
if response:
77+
print(f"Mosscap:> {response}")
78+
79+
# Add the chat message to the chat history to keep track of the conversation.
80+
chat_history.add_message(response)
81+
82+
return True
83+
84+
85+
async def main() -> None:
86+
# Start the chat loop. The chat loop will continue until the user types "exit".
87+
chatting = True
88+
while chatting:
89+
chatting = await chat()
90+
91+
92+
"""
93+
Sample output:
94+
User:> Why is the sky blue in one sentence?
95+
Mosscap:> The sky appears blue due to Rayleigh scattering, where shorter blue wavelengths of sunlight are scattered in
96+
all directions by the gases and particles in Earth's atmosphere more than other colors.
97+
"""
98+
99+
100+
if __name__ == "__main__":
101+
asyncio.run(main())

0 commit comments

Comments
 (0)