Skip to content

v1.0.14

Latest

Choose a tag to compare

@csinva csinva released this 15 Oct 14:42

LLM providers in imodelsx.llm

get_llm now supports seven providers, chosen with a provider: prefix on the checkpoint (or inferred from the name):

Provider Example checkpoint Credentials
hf (default) gpt2, Qwen/Qwen2.5-0.5B-Instruct HF_TOKEN for gated models
hf-api hf-api:meta-llama/Llama-3.1-8B-Instruct HF_TOKEN
openai gpt-4o-mini OPENAI_API_KEY
azure azure:gpt-4o AZURE_OPENAI_ENDPOINT + AZURE_OPENAI_API_KEY, or Azure CLI login
anthropic claude-haiku-5-5 ANTHROPIC_API_KEY
claude-code claude-code:haiku Claude Code login
codex codex:gpt-5-codex Codex login

Keys can also be passed with api_key= or stored in ~/.<ENV_VAR>. Install extras: pip install "imodelsx[openai]", "imodelsx[anthropic]", "imodelsx[all]". Existing OpenAI caches stay valid.

Fixes

  • SASC passed its cache dir as the seed and defaulted to the retired text-davinci-003 (now gpt-4o-mini); summaries from chat models are cleaned up
  • Aug-Tree keyword expansion no longer uses the removed openai.Completion API
  • QA-Emb sends any API checkpoint through the chat path
  • LinearFinetune works with datasets Column inputs and no longer pads to the model's max length
  • embeddings.get_embs runs on CPU; iPrompt supports llama/qwen-style models
  • HF models: no error without HF_TOKEN, float32 on CPU, output decoded from new tokens only

Tests

New offline provider tests, plus live end-to-end tests gated by IMODELSX_TEST_HF_CHECKPOINT / IMODELSX_TEST_API_CHECKPOINT (52/52 pass with Qwen2.5-0.5B-Instruct and claude-code:haiku).