Java SDK, does it have caching support like the Python and JS SDKs?

[bdavid]

Describe your question

In the Caching of Prompts in Client SDKs page, it talks about caching built in. Does langfuse-java have the same caching support?

Langfuse Cloud or Self-Hosted?

Langfuse Cloud

If Self-Hosted

No response

If Langfuse Cloud

No response

SDK and integration versions

langfuse-java 0.2.0

Pre-Submission Checklist

• I have checked for existing issues/discussions and consulted Langfuse AI.

[inkeep-ai-assistant[bot]]

The available documentation on prompt caching specifically covers the Python SDK and JS/TS SDK only. (1) The caching documentation shows code examples exclusively for these two SDKs, with features like cache_ttl_seconds for Python and cacheTtlSeconds for JS/TS. (1)

I don't have information in the available sources about whether the langfuse-java SDK implements the same client-side caching behavior. The Java SDK isn't mentioned in the prompt caching documentation.

You may want to:

• Check the langfuse-java GitHub repository directly for caching implementation details
• Open a GitHub discussion to ask the maintainers about Java SDK caching support

Check out these potentially useful sources: Caching of Prompts in Client SDKs, Prompt Management, Public API

Would you like me to try to answer generally based on my knowledge instead?

📚 Sources:

1. Caching of Prompts in Client SDKs

---

Have another question? Just tag @inkeep.

[unusdon]

Confirmed — as of current main in langfuse-java, the SDK has no equivalent to the Python SDK's cache_ttl_seconds / JS SDK's cacheTtlSeconds. PromptsClient.get(name, request) and its async counterpart are thin HTTP wrappers around GET /api/public/v2/prompts/{name} — every call is a network fetch.

I checked:

• com.langfuse.client.resources.prompts.PromptsClient#get() — no memoization, no eviction hooks
• com.langfuse.client.resources.prompts.AsyncPromptsClient#get() — same
• Code search for Cache / Caffeine / LoadingCache under com.langfuse.client.** — no hits related to prompt caching

Your langfuse-java#44 is the right place for the feature request; a maintainer will need to add a cache layer to the SDK itself since there's no flag to flip today.

Workaround until the SDK grows a cache

Wrap PromptsClient.get() in Caffeine with a TTL matching what you'd have set on cache_ttl_seconds. Caffeine is what the Python/JS SDKs' cache behaviour maps to conceptually — bounded size, TTL-based expiry, optional refresh-ahead to hide latency spikes.

import com.github.benmanes.caffeine.cache.Caffeine;
import com.github.benmanes.caffeine.cache.LoadingCache;
import com.langfuse.client.LangfuseClient;
import com.langfuse.client.resources.prompts.requests.GetPromptRequest;
import com.langfuse.client.resources.prompts.types.Prompt;
import java.time.Duration;

public class CachedLangfusePrompts {
    private final LoadingCache<String, Prompt> cache;

    public CachedLangfusePrompts(LangfuseClient langfuse, Duration ttl) {
        this.cache = Caffeine.newBuilder()
            .expireAfterWrite(ttl)                       // matches cache_ttl_seconds semantics
            .refreshAfterWrite(ttl.dividedBy(2))         // background refresh at half-TTL (optional)
            .maximumSize(1000)                           // cap unbounded growth
            .build(name -> langfuse.prompts().get(name, GetPromptRequest.builder().build()));
    }

    public Prompt get(String promptName) {
        return cache.get(promptName);
    }
}

For versioned or labelled prompts, key the cache on (name, version) or (name, label) and pass the corresponding builder field inside the loader.

Not to be confused with Anthropic prompt caching

Worth flagging because the terminology overlaps: what we're discussing (cache_ttl_seconds) is client-side caching of the prompt fetch — avoid round-tripping to Langfuse for the same prompt. Anthropic's prompt caching is a server-side inference-cost optimisation set via cache_control: {type: "ephemeral"} on message content. The two are orthogonal — the Java SDK lacking one has no effect on the other.

[NovaRouteAI]

@bdavid given the docs only cover caching for the Python and JS SDKs, the honest answer is likely "not yet" — but the cost argument for parity is strong, since prompt caching is what turns repeated traces from paying full input price every time.

If you're running high-volume Java services against Langfuse today, the workaround worth measuring is how much of your prompt traffic is byte-identical across runs; if it's a large share, client-side dedup of observation payloads buys part of the benefit until the SDK catches up.

Prompt-cache economics across SDKs is one of the things we've been watching while testing an OpenAI-compatible multi-model API layer with routing and usage/cost visibility.

Is the Java service high-volume enough that caching would move the bill, or is it mainly a completeness question?