Skip to content

[http-client-csharp] Add Tier 1 in-memory response cache to playground-server#10718

Draft
Copilot wants to merge 3 commits into
mainfrom
copilot/improve-csharp-emitter-performance
Draft

[http-client-csharp] Add Tier 1 in-memory response cache to playground-server#10718
Copilot wants to merge 3 commits into
mainfrom
copilot/improve-csharp-emitter-performance

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 18, 2026

The playground's C# /generate endpoint spawns a fresh dotnet subprocess on every request, with no caching anywhere in the pipeline. Identical replays (undo/redo, share-link reloads, demo specs hit by many users) pay the full cost. This implements Item 3, Tier 1 of the perf plan: a container-local in-memory response cache.

Changes

  • playground-server/GenerationCache.cs (new)IGenerationCache + MemoryGenerationCache backed by IMemoryCache. ComputeKey(generatorName, codeModel, configuration, generatorVersion) returns a SHA-256 hex over a length-prefixed concatenation (avoids cross-boundary collisions like "Foo"+"Bar" vs "FooBar"+""). Entry Size = body byte length; backing cache SizeLimit = 256 MB; 1-hour sliding expiration.
  • playground-server/Program.cs — registers AddMemoryCache + IGenerationCache singleton. Reads the generator DLL's FileVersionInfo.FileVersion once at startup and folds it into the key, so a deploy of a new binary implicitly invalidates every entry. /generate short-circuits on hit (returns cached bytes, skips the subprocess) and serializes-once-then-caches on miss. Responses carry X-Cache: HIT|MISS.
  • playground-server.Tests/ (new NUnit project, 15 tests) — key determinism & format; sensitivity to each of the four components (including version → invalidation); length-prefix unambiguity; null-argument guards; get/miss/set/overwrite round-trip; SizeLimit compaction; sliding-expiration eviction.

Cache flow

var cacheKey = MemoryGenerationCache.ComputeKey(generatorName, body.CodeModel!, body.Configuration!, generatorVersion);
if (cache.TryGet(cacheKey, out var cached) && cached is not null)
{
    request.HttpContext.Response.Headers["X-Cache"] = "HIT";
    return Results.Bytes(cached.Body, cached.ContentType); // skips dotnet subprocess
}
request.HttpContext.Response.Headers["X-Cache"] = "MISS";
// ...run generator, then:
var responseBytes = JsonSerializer.SerializeToUtf8Bytes(new GenerateResponse(files), GenerateJsonContext.Default.GenerateResponse);
cache.Set(cacheKey, new CachedGenerationResponse(responseBytes, "application/json"));
return Results.Bytes(responseBytes, "application/json");

Out of scope

Tier 2 file-based cache and Items 1/2/4/5 of the issue. Generator internals are untouched; production tsp codegen is unaffected.

…d-server

Agent-Logs-Url: https://github.com/microsoft/typespec/sessions/693fd6ec-31da-4c6f-a4be-4324b946a6f4

Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
@microsoft-github-policy-service microsoft-github-policy-service Bot added the emitter:client:csharp Issue for the C# client emitter: @typespec/http-client-csharp label May 18, 2026
Copilot AI changed the title [WIP] Enhance performance of C# emitter in TypeSpec playground [http-client-csharp] Add Tier 1 in-memory response cache to playground-server May 18, 2026
Copilot AI requested a review from jorgerangel-msft May 18, 2026 20:22
builder.Services.AddCors();
builder.Services.AddMemoryCache(options =>
{
// Tier 1 cache cap (Item 3 of playground perf plan): 256 MB of response bodies.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot remove any comments pertaining to the issue. In general, we should only add comments when the code itself is not self explanatory.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the Item 3 / Tier 1 / perf-plan references and trimmed other inline comments where the code was self-explanatory in 48ddd6c.

Agent-Logs-Url: https://github.com/microsoft/typespec/sessions/656e84a4-17f1-4c10-87de-4f80c2df8412

Co-authored-by: jorgerangel-msft <102122018+jorgerangel-msft@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

emitter:client:csharp Issue for the C# client emitter: @typespec/http-client-csharp

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Brainstorm perf improvements for running c# emitter in playground

2 participants