This is a plugin for llm that uses GitHub Models via the Azure AI Inference SDK.
$ llm install llm-github-models
To set the API key, use the llm keys set github command or use the GITHUB_MODELS_KEY environment variable.
To get an API key, create a personal access token (PAT) inside GitHub Settings.
Learn about rate limits here
All model names are affixed with github/ to distinguish the OpenAI ones from the builtin models.
$ llm prompt 'top facts about cheese' -m github/mistral-large
Sure, here are some interesting facts about cheese:
1. There are over 2000 types of cheese: The variety of cheese is vast, with different flavors, textures, and aromas. This is due to factors like the type of milk used, the aging process, and the specific bacteria and mold cultures involved.
2. Cheese is an ancient food: The earliest evidence of cheese-making dates back to around 6000 BC, found in ancient Polish sites.
Multi-modal vision models can accept image attachments using the LLM attachments options:
llm -m github/Llama-3.2-11B-Vision-Instruct "Describe this image" -a https://static.simonwillison.net/static/2024/pelicans.jpgProduces
This image depicts a dense gathering of pelicans, with the largest birds situated in the center, showcasing their light brown plumage and long, pointed beaks. The pelicans are standing on a rocky shoreline, with a serene body of water behind them, characterized by its pale blue hue and gentle ripples. In the background, a dark, rocky cliff rises, adding depth to the scene.
The overall atmosphere of the image exudes tranquility, with the pelicans seemingly engaging in a social gathering or feeding activity. The photograph's clarity and focus on the pelicans' behavior evoke a sense of observation and appreciation for the natural world.Usage: llm -m github/AI21-Jamba-1.5-Large
Publisher: AI21 Labs
Description: A 398B parameters (94B active) multilingual model, offering a 256K long context window, function calling, structured output, and grounded generation.
Usage: llm -m github/AI21-Jamba-1.5-Mini
Publisher: AI21 Labs
Description: A 52B parameters (12B active) multilingual model, offering a 256K long context window, function calling, structured output, and grounded generation.
Usage: llm -m github/Codestral-2501
Publisher: Mistral AI
Description: Codestral 25.01 by Mistral AI is designed for code generation, supporting 80+ programming languages, and optimized for tasks like code completion and fill-in-the-middle
Usage: llm -m github/Cohere-command-r
Publisher: Cohere
Description: Command R is a scalable generative model targeting RAG and Tool Use to enable production-scale AI for enterprise.
Usage: llm -m github/Cohere-command-r-08-2024
Publisher: Cohere
Description: Command R is a scalable generative model targeting RAG and Tool Use to enable production-scale AI for enterprise.
Usage: llm -m github/Cohere-command-r-plus
Publisher: Cohere
Description: Command R+ is a state-of-the-art RAG-optimized model designed to tackle enterprise-grade workloads.
Usage: llm -m github/Cohere-command-r-plus-08-2024
Publisher: Cohere
Description: Command R+ is a state-of-the-art RAG-optimized model designed to tackle enterprise-grade workloads.
Usage: llm -m github/Cohere-embed-v3-english
Publisher: Cohere
Description: Cohere Embed English is the market's leading text representation model used for semantic search, retrieval-augmented generation (RAG), classification, and clustering.
Usage: llm -m github/Cohere-embed-v3-multilingual
Publisher: Cohere
Description: Cohere Embed Multilingual is the market's leading text representation model used for semantic search, retrieval-augmented generation (RAG), classification, and clustering.
Usage: llm -m github/DeepSeek-R1
Publisher: DeepSeek
Description: DeepSeek-R1 excels at reasoning tasks using a step-by-step training process, such as language, scientific reasoning, and coding tasks.
Usage: llm -m github/DeepSeek-V3
Publisher: DeepSeek
Description: A strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token.
Usage: llm -m github/DeepSeek-V3-0324
Publisher: DeepSeek
Description: DeepSeek-V3-0324 demonstrates notable improvements over its predecessor, DeepSeek-V3, in several key aspects, including enhanced reasoning, improved function calling, and superior code generation capabilities.
Usage: llm -m github/Llama-3.2-11B-Vision-Instruct
Publisher: Meta
Description: Excels in image reasoning capabilities on high-res images for visual understanding apps.
Usage: llm -m github/Llama-3.2-90B-Vision-Instruct
Publisher: Meta
Description: Advanced image reasoning capabilities for visual understanding agentic apps.
Usage: llm -m github/Llama-3.3-70B-Instruct
Publisher: Meta
Description: Llama 3.3 70B Instruct offers enhanced reasoning, math, and instruction following with performance comparable to Llama 3.1 405B.
Usage: llm -m github/Llama-4-Maverick-17B-128E-Instruct-FP8
Publisher: Meta
Description: Llama 4 Maverick 17B 128E Instruct FP8 is great at precise image understanding and creative writing, offering high quality at a lower price compared to Llama 3.3 70B
Usage: llm -m github/Llama-4-Scout-17B-16E-Instruct
Publisher: Meta
Description: Llama 4 Scout 17B 16E Instruct is great at multi-document summarization, parsing extensive user activity for personalized tasks, and reasoning over vast codebases.
Usage: llm -m github/Meta-Llama-3-70B-Instruct
Publisher: Meta
Description: A powerful 70-billion parameter model excelling in reasoning, coding, and broad language applications.
Usage: llm -m github/Meta-Llama-3-8B-Instruct
Publisher: Meta
Description: A versatile 8-billion parameter model optimized for dialogue and text generation tasks.
Usage: llm -m github/Meta-Llama-3.1-405B-Instruct
Publisher: Meta
Description: The Llama 3.1 instruction tuned text only models are optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.
Usage: llm -m github/Meta-Llama-3.1-70B-Instruct
Publisher: Meta
Description: The Llama 3.1 instruction tuned text only models are optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.
Usage: llm -m github/Meta-Llama-3.1-8B-Instruct
Publisher: Meta
Description: The Llama 3.1 instruction tuned text only models are optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.
Usage: llm -m github/Ministral-3B
Publisher: Mistral AI
Description: Ministral 3B is a state-of-the-art Small Language Model (SLM) optimized for edge computing and on-device applications. As it is designed for low-latency and compute-efficient inference, it it also the perfect model for standard GenAI applications that have
Usage: llm -m github/Mistral-Large-2411
Publisher: Mistral AI
Description: Mistral Large 24.11 offers enhanced system prompts, advanced reasoning and function calling capabilities.
Usage: llm -m github/Mistral-Nemo
Publisher: Mistral AI
Description: Mistral Nemo is a cutting-edge Language Model (LLM) boasting state-of-the-art reasoning, world knowledge, and coding capabilities within its size category.
Usage: llm -m github/Mistral-large
Publisher: Mistral AI
Description: Mistral's flagship model that's ideal for complex tasks that require large reasoning capabilities or are highly specialized (Synthetic Text Generation, Code Generation, RAG, or Agents).
Usage: llm -m github/Mistral-large-2407
Publisher: Mistral AI
Description: Mistral Large (2407) is an advanced Large Language Model (LLM) with state-of-the-art reasoning, knowledge and coding capabilities.
Usage: llm -m github/Mistral-small
Publisher: Mistral AI
Description: Mistral Small can be used on any language-based task that requires high efficiency and low latency.
Usage: llm -m github/Phi-3-medium-128k-instruct
Publisher: Microsoft
Description: Same Phi-3-medium model, but with a larger context size for RAG or few shot prompting.
Usage: llm -m github/Phi-3-medium-4k-instruct
Publisher: Microsoft
Description: A 14B parameters model, proves better quality than Phi-3-mini, with a focus on high-quality, reasoning-dense data.
Usage: llm -m github/Phi-3-mini-128k-instruct
Publisher: Microsoft
Description: Same Phi-3-mini model, but with a larger context size for RAG or few shot prompting.
Usage: llm -m github/Phi-3-mini-4k-instruct
Publisher: Microsoft
Description: Tiniest member of the Phi-3 family. Optimized for both quality and low latency.
Usage: llm -m github/Phi-3-small-128k-instruct
Publisher: Microsoft
Description: Same Phi-3-small model, but with a larger context size for RAG or few shot prompting.
Usage: llm -m github/Phi-3-small-8k-instruct
Publisher: Microsoft
Description: A 7B parameters model, proves better quality than Phi-3-mini, with a focus on high-quality, reasoning-dense data.
Usage: llm -m github/Phi-3.5-MoE-instruct
Publisher: Microsoft
Description: A new mixture of experts model
Usage: llm -m github/Phi-3.5-mini-instruct
Publisher: Microsoft
Description: Refresh of Phi-3-mini model.
Usage: llm -m github/Phi-3.5-vision-instruct
Publisher: Microsoft
Description: Refresh of Phi-3-vision model.
Usage: llm -m github/Phi-4
Publisher: Microsoft
Description: Phi-4 14B, a highly capable model for low latency scenarios.
Usage: llm -m github/Phi-4-mini-instruct
Publisher: Microsoft
Description: 3.8B parameters Small Language Model outperforming larger models in reasoning, math, coding, and function-calling
Usage: llm -m github/Phi-4-multimodal-instruct
Publisher: Microsoft
Description: First small multimodal model to have 3 modality inputs (text, audio, image), excelling in quality and efficiency
Usage: llm -m github/gpt-4.1
Publisher: OpenAI
Description: gpt-4.1 outperforms gpt-4o across the board, with major gains in coding, instruction following, and long-context understanding
Usage: llm -m github/gpt-4.1-mini
Publisher: OpenAI
Description: gpt-4.1-mini outperform gpt-4o-mini across the board, with major gains in coding, instruction following, and long-context handling
Usage: llm -m github/gpt-4.1-nano
Publisher: OpenAI
Description: gpt-4.1-nano provides gains in coding, instruction following, and long-context handling along with lower latency and cost
Usage: llm -m github/gpt-4o
Publisher: OpenAI
Description: OpenAI's most advanced multimodal model in the gpt-4o family. Can handle both text and image inputs.
Usage: llm -m github/gpt-4o-mini
Publisher: OpenAI
Description: An affordable, efficient AI solution for diverse text and image tasks.
Usage: llm -m github/jais-30b-chat
Publisher: Core42
Description: JAIS 30b Chat is an auto-regressive bilingual LLM for Arabic & English with state-of-the-art capabilities in Arabic.
Usage: llm -m github/mistral-small-2503
Publisher: Mistral AI
Description: Enhanced Mistral Small 3 with multimodal capabilities and a 128k context length.
Usage: llm -m github/o1
Publisher: OpenAI
Description: Focused on advanced reasoning and solving complex problems, including math and science tasks. Ideal for applications that require deep contextual understanding and agentic workflows.
Usage: llm -m github/o1-mini
Publisher: OpenAI
Description: Smaller, faster, and 80% cheaper than o1-preview, performs well at code generation and small context operations.
Usage: llm -m github/o1-preview
Publisher: OpenAI
Description: Focused on advanced reasoning and solving complex problems, including math and science tasks. Ideal for applications that require deep contextual understanding and agentic workflows.
Usage: llm -m github/o3-mini
Publisher: OpenAI
Description: o3-mini includes the o1 features with significant cost-efficiencies for scenarios requiring high performance.
Usage: llm -m github/text-embedding-3-large
Publisher: OpenAI
Description: Text-embedding-3 series models are the latest and most capable embedding model from OpenAI.
Usage: llm -m github/text-embedding-3-small
Publisher: OpenAI
Description: Text-embedding-3 series models are the latest and most capable embedding model from OpenAI.