Drop-in cost map URL with daily-refreshed pricing (~340 models missing from the community map)

[Gaploid]

LiteLLM can pull model_prices_and_context_window.json live from GitHub, which is great, but that file is community-maintained. So prices lag, new models take a while to show up (or never do), and sometimes the numbers are just wrong until someone opens a PR.

We ended up building our own drop-in replacement. Same schema, same env var, so it works for both the Python SDK and the gateway proxy: export LITELLM_MODEL_COST_MAP_URL="https://cloudprice.net/api/v2/ai/litellm_model_prices.json"

What's different:

• We pull straight from each provider and refresh daily. We built quite comprehensive ingestion pipeline to pull data across major providers.
• It covers image / audio / video / rerank / OCR pricing, not just chat + embeddings.
• Right now ~340 models come back with pricing that isn't in the LiteLLM map at all — mostly fresh releases like openrouter/z-ai/glm-5.2, openrouter/deepseek/deepseek-V4, and Vercel models.
• Completely free (some throttling to keep it stable), no key, CORS enabled.

The thing I actually wanted to ask the maintainers/community: would it make sense for LiteLLM to support multiple cost map sources with fallback, right in the gateway UI? A primary URL plus fallbacks, if one is missing a model, it falls through to the next. That'd fix the whole stale/missing problem regardless of whose map you use.

Here is more details: https://cloudprice.net/models/api?litellm=1

[mateo-berri]

Hey, thanks for sharing this! This is very interesting. :)

would it make sense for LiteLLM to support multiple cost map sources with fallback, right in the gateway UI?

Hmm... I think this might not be a bad idea. However, I want to see what the community thinks and gauge interest from there. If folks who are interested in this could reply to this thread. If there's enough traction, I'll implement it.

I was planning to set up an automatic provider ingestion pipeline as part of the product. However, with the current backlog of issues, would you be interested in being a vendor we can use to automatically update our JSON daily?

[oskarkocol]

Jumping in here as well! Automating this is the right move. Whenever there is manual work - there will be staleness.

From my point of view the big issue with the JSON is that it's quick to catch merge conflicts. I know the maintainers have a big chunk of contributions to review. If it's not happening in time, even the updates go stale.

I'm developing over at narev.ai, and we'd love to step up as the official data vendor for this. We build the pricing engine that powers usage-based billing for AI (directly integrating with Stripe, Polar, Lago, OpenMeter).

• We refresh almost in real time (prices change multiple times a day for certain providers)
• We get confidence score by checking against other major pricing sources (OpenRouter, Vercel Gateway, Models.dev, Bifrost) to name a few
• Completely free, distributed globally on the edge, and in multiple formats (API, Websockets, database syncs, S3 dumps, you name it)

I was planning to set up an automatic provider ingestion pipeline as part of the product

This will work for providers that do give pricing in their endpoint. Many providers don't (see openai, anthropic). Unfortunately pricing in gateways (like OpenRouter) differs from the pricing directly from the provider

being a vendor we can use to automatically update our JSON daily

Would love to do it!

[Gaploid]

However, with the current backlog of issues, would you be interested in being a vendor we can use to automatically update our JSON daily

Yep, we’d be happy to serve as a provider for this data. We’ve been providing pricing data for the past eight years with ~40k users per month, starting with cloud instances, and we already have a comprehensive, actively monitored data pipeline in place.

I think there should be an option to configure multiple providers in litellm's admin console. The default setup could use cloudprice.net as the primary source with the most fresh data and litellm_model_prices.json from GitHub as a fallback.

We currently fetch data dynamically from the following providers, in priority order:

SOURCE_PRIORITY = (
"openai",
"anthropic",
"google_gemini",
"google_vertex_ai",
"mistral",
"cohere",
"xai",
"deepseek",
"alibaba",
"perplexity",
"snowflake",
"amazon_bedrock",
"amazon_bedrock_global",
"amazon_bedrock_inference_profiles",
"amazon_sagemaker",
"artificialanalysis",
"openrouter",
"vercel",
"vercel_ai_gateway",
"huggingface_inference",
"azure_aifoundry",
"litellm",
)

As you can see, we also use the LiteLLM file to cover the long tail of smaller providers. Let me know how we can help here.

[oskarkocol]

Hey @mateo-berri, before committing to new UI dependencies, you can actually solve the staleness issue right now with minimal effort.

1. Fix the existing GitHub Action (and run it daily)

You already have a pipeline set up:
https://github.com/BerriAI/litellm/blob/litellm_internal_staging/.github/workflows/auto_update_price_and_context_window.yml

Right now, it’s scheduled weekly, but it's failing (link). Example traceback:

Traceback (most recent call last):
  File "/workspaces/litellm-contrib/.github/workflows/auto_update_price_and_context_window_file.py", line 159, in <module>
    main()
  File "/workspaces/litellm-contrib/.github/workflows/auto_update_price_and_context_window_file.py", line 145, in main
    vercel_data = transform_vercel_ai_gateway_data(vercel_data)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/litellm-contrib/.github/workflows/auto_update_price_and_context_window_file.py", line 91, in transform_vercel_ai_gateway_data

The issue: It fails on models that have no output pricing (e.g., embeddings or video models like alibaba/qwen3-embedding-0.6b). A missing pricing.output makes float(row["pricing"]["output"]) throw an error.

The fix: I fixed the script and tested it:

• Here is the PR: fix(actions): pricing auto update action #33073
• Proof of successful run: GitHub Action Run
• (Note: I followed the repo convention of using ${{ secrets.GITHUB_TOKEN }}. If you see an error creating PRs due to repo permissions, just use a PAT stored as ${{ secrets.GH_TOKEN }}).

2. Pull directly from providers that expose pricing

For providers that ship pricing in their /models endpoint, you should pull directly. Novita AI is a good example. They have pricing and don’t even require an API key to get it:

curl --request GET \
  --url https://api.novita.ai/openai/v1/models

Here is a PR implementing the direct pull pattern for Novita: oskarkocol#3

3. Use a data vendor for the rest

For providers that hide pricing (OpenAI) or require API keys (Cloudflare), Narev can fill the gaps.

I implemented Cerebras as a test case here (they have a simple offering of 3 models so it’s easy to cross-check).

Here is the draft PR: oskarkocol#4

On aggregators

Relying on generic aggregators (like what's proposed above) creates a circular dependency.

For example, Cerebras charges for prompt caching (source). The initial models.dev data missed this (I opened a PR to fix it here), and aggregator APIs end up serving that incorrect pricing:

• Cloudprice - Zhipu GLM-4.7 (Echoes the mistake)
• Infron - Z.AI GLM 4.7 (Lists cache read/write as empty)
• Bifrost - Cerebras GLM 4.7 (Cost calculator ignores cache entirely)

To be fair to everyone in this thread, tracking LLM pricing is just a genuinely hard problem right now. Gateways frequently get it wrong, edge cases like caching are a mess and prices change daily. It takes real, active care to keep it all in sync.

[Gaploid]

Hey @mateo-berri we can try to create PR with ability to add multiple sources and fall back to .json file from our side. What do you think?