Compatibility requirements for OpenAI compatible API translation layer?

[timjuedemann]

Hello everyone,
after having successfully set up multiple providers as described in the docs, I am currently struggling with the task of setting up an OpenAI compatible translation layer for the 1min.AI API as a "custom provider".

The following code currently works in OpenWebUI but I can’t get it to work in OmniRoute where I tried to add my translation layer via the "Add OpenAI Compatible" button. I did not specify an API key because I’m providing it through an environment variable.

require("dotenv").config();

import express from "express";
import fetch from "node-fetch";

const app = express();
app.use(express.json());

const API_KEY = process.env.API_KEY;
const ALLOWED_MODELS = [
    "gpt-4o-mini",
    "gpt-4o"
];

app.post("/v1/chat/completions", async (req, res) => {
    try {
        const { messages, model } = req.body;

        const prompt = messages
            .map((m) => {
                if (m.role === "system") return `System: ${m.content}`;
                if (m.role === "user") return `User: ${m.content}`;
                if (m.role === "assistant") return `Assistant: ${m.content}`;
                return "";
            })
            .join("\n");

        const response = await fetch("https://api.1min.ai/api/chat-with-ai?isStreaming=false", {
            method: "POST",
            headers: {
                "API-KEY": API_KEY,
                "Content-Type": "application/json",
            },
            body: JSON.stringify({
                type: "UNIFY_CHAT_WITH_AI",
                model: model,
                promptObject: {
                    prompt,
                    settings: {
                        webSearchSettings: { webSearch: false },
                    },
                },
            }),
        });

        const data = await response.json();

        const content = data?.aiRecord?.aiRecordDetail?.resultObject?.[0] || "No response";

        res.json({
            id: `chatcmpl-${Date.now()}`,
            object: "chat.completion",
            created: Math.floor(Date.now() / 1000),
            model: model,
            choices: [
                {
                    index: 0,
                    message: {
                        role: "assistant",
                        content,
                    },
                    finish_reason: "stop",
                },
            ],
        });
    } catch (err) {
        console.error(err);
        res.status(500).json({ error: "Proxy error" });
    }
});

app.get("/v1/models", (req, res) => {
    res.json({
        object: "list",
        data: ALLOWED_MODELS.map((m) => ({
            id: m,
            object: "model",
        })),
    });
});

app.listen(3001, () => {
    console.log("API Proxy running on port 3001");
});

Could anyone help point me towards a solution? I am not sure whether this is a problem of me not setting the provider up correctly or whether my endpoints are returning an incompatible format that OmniRoute can’t process correctly?

Thanks very much in advance!

[diegosouzapw]

Hi @timjuedemann!

Your translation layer looks correct for the response format. A few things to check when using it with OmniRoute:

1. Base URL: When adding the provider in OmniRoute, make sure the Base URL points to where your proxy is reachable from OmniRoute's perspective. If both run in Docker, use the container name or host network instead of localhost.
2. API Key handling: OmniRoute sends an Authorization: Bearer header with every request. Your proxy should either:
3. • Accept any value (ignore the header)
4. • Or configure OmniRoute with a dummy API key that matches
5. Model list endpoint: Ensure your /v1/models endpoint is reachable. OmniRoute calls this on provider setup to discover available models.
6. Streaming support: Even though your proxy returns non-streaming responses, OmniRoute may request "stream": true. Consider adding a check for req.body.stream and either supporting SSE streaming or returning a non-streamed response while ignoring the flag.
7. Health check: Try curl http://:3001/v1/models from the OmniRoute container to verify connectivity.
   Could you share the error you're seeing in OmniRoute's logs? That would help pinpoint the exact issue.

[timjuedemann]

Hello @diegosouzapw,

thanks for taking the time to help me with this. I should have mentioned that I’m running OmniRoute on a local homelab style server because that probably would have cleared up my mistake immediately. Instead, I had to learn the hard way that OmniRoute is set to block private/local IPs for security reasons per default.

Fortunately, adding OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS: "true" to my compose.yaml was all it took to get it working. Your list was a great help in pointing me in the right direction, nonetheless. Thanks again for your work on this great tool and your efforts in managing the issues and discussions in this repo.

Best regards,
Tim

[diegosouzapw]

Glad you got it working, @timjuedemann! The OMNIROUTE_ALLOW_PRIVATE_PROVIDER_URLS=true flag is the right escape hatch for homelab / private-network setups — I'll see about surfacing that more prominently in the docs and in the "Add Provider" UI so the next person doesn't have to learn it the hard way. Thanks for the kind words!