Semantic Kernel against an OpenAI-compatible endpoint in .NET and Python: endpoint overload, own OpenAIClient for retries, auto function calling

[apigoto]

I work on the APIGOTO team. APIGOTO is an independent OpenAI-compatible gateway, not affiliated with OpenAI, Anthropic or Google. This is a side-by-side for both SDKs of pointing the OpenAI connector at any /v1/chat/completions endpoint (a gateway, vLLM, LM Studio...). It covers where the URL goes, how to control retries, and how to check that function calling really works with the model you picked.

.NET: the endpoint overload

using Microsoft.SemanticKernel;

var kernel = Kernel.CreateBuilder()
    .AddOpenAIChatCompletion(
        modelId: "MODEL_ID",                                   // exactly as in GET /v1/models
        endpoint: new Uri("https://api.china-aiapi.com/v1"),   // include /v1
        apiKey: Environment.GetEnvironmentVariable("GATEWAY_API_KEY"))
    .Build();

This (modelId, Uri endpoint, apiKey, ...) overload is documented as "Custom OpenAI Compatible Message API endpoint". Unlike some sibling overloads, it isn't marked [Experimental], so no SKEXP0010 pragma is needed.

.NET: bring your own OpenAIClient when you need retry or timeout control

using System.ClientModel;
using System.ClientModel.Primitives;
using OpenAI;

var options = new OpenAIClientOptions
{
    Endpoint = new Uri("https://api.china-aiapi.com/v1"),
    NetworkTimeout = TimeSpan.FromSeconds(60),
    RetryPolicy = new ClientRetryPolicy(maxRetries: 1),
};
var client = new OpenAIClient(new ApiKeyCredential(Environment.GetEnvironmentVariable("GATEWAY_API_KEY")!), options);

var kernel = Kernel.CreateBuilder()
    .AddOpenAIChatCompletion("MODEL_ID", client)
    .Build();

This is the same pattern as samples/Concepts/ChatCompletion/OpenAI_CustomClient.cs. You can also set Transport to a custom HttpClient there if you want to log request URIs.

Python: pass an AsyncOpenAI client

import os
from openai import AsyncOpenAI
from semantic_kernel import Kernel
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion

client = AsyncOpenAI(
    base_url="https://api.china-aiapi.com/v1",
    api_key=os.environ["GATEWAY_API_KEY"],
    max_retries=1,
    timeout=60,
)
kernel = Kernel()
chat = OpenAIChatCompletion(ai_model_id="MODEL_ID", async_client=client, service_id="gw")
kernel.add_service(chat)

OpenAIChatCompletion raises at construction if it gets neither an async_client nor an API key. Env vars use the OPENAI_ prefix (OPENAI_API_KEY, OPENAI_CHAT_MODEL_ID), so if you construct it without a client, a leftover OPENAI_API_KEY in your shell is what gets used.

Auto function calling: a 2-minute capability check

Don't assume every model behind a gateway does tool calls well. Register one trivial function and see whether it gets called.

from semantic_kernel.connectors.ai import FunctionChoiceBehavior
from semantic_kernel.connectors.ai.open_ai import OpenAIChatPromptExecutionSettings
from semantic_kernel.contents import ChatHistory
from semantic_kernel.functions import kernel_function

class Clock:
    @kernel_function(description="Return the current UTC time in ISO 8601.")
    def utc_now(self) -> str:
        from datetime import datetime, timezone
        return datetime.now(timezone.utc).isoformat()

kernel.add_plugin(Clock(), plugin_name="clock")
settings = OpenAIChatPromptExecutionSettings(function_choice_behavior=FunctionChoiceBehavior.Auto())

history = ChatHistory()
history.add_user_message("What time is it in UTC right now?")
reply = await chat.get_chat_message_content(chat_history=history, settings=settings, kernel=kernel)
print(reply)

In .NET it's the same idea: get kernel.GetRequiredService<IChatCompletionService>() and pass new OpenAIPromptExecutionSettings { FunctionChoiceBehavior = FunctionChoiceBehavior.Auto() } to GetChatMessageContentAsync(history, settings, kernel). If the answer has a plausible time but the function never ran (add a print), the model guessed: pick another MODEL_ID for agent work.

Troubleshooting

• 404: the endpoint is missing /v1, or you added /chat/completions yourself (the client appends it).
• 401 with a correct key: a different key won, e.g. from OPENAI_API_KEY in the environment.
• Model not found: modelId / ai_model_id must match /models character for character.

Retries, twice over

Our gateway already retries a failed upstream request on another route (at most 3 times), and the OpenAI SDK retries again on top. That's why both examples set retries to 1. The gateway's per-request log shows what SK actually sent, which helps when debugging function-call payloads. Pay-as-you-go, BYOK supported: www.china-aiapi.com

For those running SK agents on non-OpenAI models: do you gate model choice with a tool-call smoke test like this, or something more formal?