[Bug]: Proper usage handling from VLLM (cached_tokens)

[mfolnovic]

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

When using VLLM as a provider, token cost calculator does not handle information about the cached tokens.

I've looked through the implementation and based on my understanding, this logic is missing in: https://github.com/BerriAI/litellm/blob/main/litellm/types/utils.py#L1538 , am I correct?

Steps to Reproduce

1. Configure vllm provider
2. Send a request, in LiteLLM logs for that request, see VLLM response:

usage:{
    total_tokens:150687,
    prompt_tokens:150383,
    completion_tokens:304,
    prompt_tokens_details:{
        text_tokens:null,
        audio_tokens:null,
        image_tokens:null,
        cached_tokens:149936
    },
    completion_tokens_details:{
        text_tokens:null,
        audio_tokens:null,
        image_tokens:null,
        reasoning_tokens:15,
        accepted_prediction_tokens:null,
        rejected_prediction_tokens:null
    }
},

3. and see "Cache Read Tokens: 0", "Cache Creation Tokens: -".

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.18.3

Twitter / LinkedIn details

No response

[nehaaprasad]

@mfolnovic
please check out this PR!
thanks

[JKratto]

I bump this, it would be nice to have this functionality.

[Mte90]

any hope for this @ishaan-jaff ?