Ollama Local hosted

[wminson]

Is there any way to use crush with ollama, I have a robust installation and I would like to use qwen3 with it? any suggestions, or could this be a feature in the future?

[frantjc]

Here's a starting point crush.json configuration file that I've got using my local ollama:

{
  "$schema": "https://charm.land/crush.json",
  "providers": {
    "Ollama": {
      "type": "openai",
      "base_url": "http://localhost:11434/v1",
      "models": [
        {
          "id": "qwen3:8b",
          "name": "Qwen3 8b"
        }
      ]
    }
  }
}

[strikeoncmputrz]

This is working for me with llama.cpp llama-server.

{
  "$schema": "https://charm.land/crush.json",
  "providers": {
    "Llama.cpp": {
      "type": "openai",
      "base_url": "https://llama.lan/",
      "api_key": "test",
      "models": [
        {
          "id": "qwen/qwen3-coder",
          "name": "Qwen: Qwen3 Coder",
          "cost_per_1m_in": 0,
          "cost_per_1m_out": 0,
          "cost_per_1m_in_cached": 0,
          "cost_per_1m_out_cached": 0,
          "context_window": 262144,
          "default_max_tokens": 26214,
          "can_reason": false,
          "has_reasoning_efforts": false,
          "supports_attachments": false
        },
        {
          "id": "Llama 4 Scout 17x16 UD-Q4_K_XL",
          "name": "Llama 4 Scout 17x16",
          "cost_per_1m_in": 0,
          "cost_per_1m_out": 0,
          "cost_per_1m_in_cached": 0,
          "cost_per_1m_out_cached": 0,
          "context_window": 262144,
          "default_max_tokens": 26214,
          "can_reason": false,
          "has_reasoning_efforts": false,
          "supports_attachments": false
        }
      ]
    }
  }
}

Is there a way to specify an embedding endpoint?

[taf2]

got this working with llama-server: ``` cat ~/.local/share/crush/crush.json
{
"$schema": "https://charm.land/crush.json",
"providers": {
"llamacpp": {
"type": "openai",
"base_url": "http://127.0.0.1:8080/v1",
"name": "Llama.cpp",
"api_key": "none",
"models": [
{
"name": "Qwen 3 30B",
"id": "unsloth/Qwen3-30B-A3B-GGUF:Q4_K_M",
"context_window": 12800,
"default_max_tokens": 20000
}
]
},
...

running llama-server from llama.cpp after build as: ```./build/bin/llama-server -hf unsloth/Qwen3-30B-A3B-GGUF:Q4_K_M --jinja --gpu-layers 99```