[JS] Model name parsing fails with multiple slashes in OpenAI-compatible plugin

[k33g]

Describe the bug

The @genkit-ai/compat-oai plugin has a model name parsing issue when the actual model ID contains slashes (/). The defineCompatOpenAIModel function extracts the model name by removing only the first segment (plugin name), but this causes problems when model IDs themselves contain slashes.

This issue affects users working with:
Ollama with models from HuggingFace (e.g., hf.co/namespace/model:tag)
Docker Model Runner with namespaced models (e.g., ai/model:tag, mistral/model:tag)
...

To Reproduce

🟢 Works: Simple model name

const response = await ai.generate({
  model: "openai/qwen2.5:0.5b",
  prompt: 'Who is Jean-Luc Picard?',
  config: {
    temperature: 0.9,
  },
});

🔴 Fails: Model name with slashes

const response = await ai.generate({
  model: "openai/hf.co/Menlo/Jan-nano-gguf:Q4_K_M",
  prompt: 'Who is Jean-Luc Picard?',
  config: {
    temperature: 0.9,
  },
});

• Parsed model name: hf.co/Menlo/Jan-nano-gguf:Q4_K_M
• But somewhere in the process, only Menlo/Jan-nano-gguf:Q4_K_M is sent to the API
• Error: NotFoundError: 404 model 'Menlo/Jan-nano-gguf:Q4_K_M' not found

Workaround: Override model in config

const response = await ai.generate({
  model: "openai/hf.co/Menlo/Jan-nano-gguf:Q4_K_M",
  prompt: 'Who is Jean-Luc Picard?',
  config: {
    temperature: 0.9,
    model: 'hf.co/Menlo/Jan-nano-gguf:Q4_K_M', // Override with full model ID
  },
});

Expected behavior

The plugin should correctly handle model names containing multiple slashes and send the complete model ID to the API without requiring the config.model workaround.

Runtime (please complete the following information):

• OS: macOS
• Version 26.0.1

Node version:

• v23.10.0

[ssbushi]

Hi, thanks for filing this issue. I'm trying to reproduce this and have a few questions. (I'm new to using HF as a provider).

• First, can you please confirm the Genkit version number?

• Based on the OpenAI-compatible "Inference providers" section, the modelName should not contain hf.co .

const completion = await client.chat.completions.create({
  model: "deepseek-ai/DeepSeek-R1:fastest",
  messages: [{ role: "user", content: "Hello!" }],
});

Following that syntax, shouldn't your modelName be "Menlo/Jan-nano-gguf:Q4_K_M"?

I tried using ZAI GLM-Air with the compatOai plugin and it is working as intended (models shows up and works in the Dev UI, and in ai.generate):

const DECLARED_MODELS = ['zai-org/GLM-4.5-Air'];

const ai = genkit({
  plugins: [
    ...
    openAICompatible({
      name: 'hf',
      baseURL: "https://router.huggingface.co/v1",
      apiKey: process.env['HF_TOKEN'],
      async initializer(client) {
        return DECLARED_MODELS.map((model) =>
          defineCompatOpenAIModel({
            name: `hf/${model}`,
            client,
            modelRef: compatOaiModelRef({
              name: `hf/${model}`,
            }),
          })
        );
      },
    }),
  ],
});


....

const llmResponse = await ai.generate({
      model: 'hf/zai-org/GLM-4.5-Air',
      prompt: `tell me a joke about ${subject}`,
      config: {
        temperature: 0.9,
      },
 });
console.log(llmResponse.text);

When I try this with Menlo/Jan-nano, I get a "The requested model is not supported by any provider" error, which (as the article suggests), needs a dedicated provider? Not sure if you have this setup already.

[k33g]

Hello @ssbushi, I do not use HuggingFace as a provider.
I'm using GGUF HuggingFace models with Ollama as a provider or Docker Model Runner as a provider.
Both (Ollama and Docker Model Runner) expose a REST API compliant with the OpenAI API and both are able to use all GGUF HuggingFace models.

If you are using HuggingFace as a provider, it's normal that you cannot use jan-nano, because HuggingFace provides a limited list of models: https://huggingface.co/docs/inference-providers/en/index#partners

With Genkit Go, I can use Docker Model Runner or Ollama with jan-nano without any problem using this openai/hf.co/menlo/jan-nano-gguf:q4_k_m

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/firebase/genkit/go/ai"
	"github.com/firebase/genkit/go/genkit"
	"github.com/firebase/genkit/go/plugins/compat_oai/openai"
	"github.com/openai/openai-go/option"
)

// Structure for flow input
type ChatRequest struct {
	Message string `json:"message"`
}

// Structure for final flow output
type ChatResponse struct {
	Response string `json:"response"`
}

func main() {
	ctx := context.Background()

	oaiPlugin := &openai.OpenAI{
		APIKey: "",
		Opts: []option.RequestOption{
			// Docker Model Runner endpoint
			//option.WithBaseURL("http://localhost:12434/engines/v1/"),
			// Ollama endpoint
			option.WithBaseURL("http://localhost:11434/v1/"),
		},
	}

	genKitInstance := genkit.Init(ctx, genkit.WithPlugins(oaiPlugin))

	chatFlow := genkit.DefineFlow(genKitInstance, "chat-flow", func(ctx context.Context, input *ChatRequest) (*ChatResponse, error) {

		resp, err := genkit.Generate(ctx, genKitInstance,
			ai.WithModelName("openai/hf.co/menlo/jan-nano-gguf:q4_k_m"),
			ai.WithSystem("You are the dungeon master of a D&D game."),
			ai.WithPrompt(input.Message),
			ai.WithConfig(map[string]any{"temperature": 0.7}),
		)
		if err != nil {
			return nil, err
		}
		return &ChatResponse{Response: resp.Text()}, nil

	})

	// Run the flow once to test it
	responseFlow, err := chatFlow.Run(ctx, &ChatRequest{
		Message: "Generate a D&D NPC Elf name and all its characteristics.",
	})
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(responseFlow.Response)

}

You can pull jan-nano from https://huggingface.co/Menlo/Jan-nano-gguf
with these commands:

# Ollama
ollama pull hf.co/Menlo/Jan-nano-gguf:Q4_K_M

# Docker Model Runner
docker model pull hf.co/Menlo/Jan-nano-gguf:Q4_K_M

and with Genkit JS it will work with this workaround:

const response = await ai.generate({
  model: "openai/hf.co/Menlo/Jan-nano-gguf:Q4_K_M",
  prompt: 'Who is Jean-Luc Picard?',
  config: {
    temperature: 0.9,
    model: 'hf.co/Menlo/Jan-nano-gguf:Q4_K_M', // Override with full model ID
  },
});