perf: implement local L1 memory cache in front of Upstash Redis

[YashKrTripathi]

Hi @Priyanshu-byte-coder here is the new focused pr please have a look:

Description

This PR implements a fast, server-side in-memory L1 cache layer in front of the existing Upstash Redis (L2) cache helper (withMetricsCache).

By performing an $O(1)$ local memory lookup before sending network requests to Redis, this optimization significantly reduces latency for dashboard metric updates, prevents Redis rate-limit consumption during high traffic, and gracefully falls back to a pure in-memory cache for local development when Upstash environment variables are not configured.

Changes

• L1 Memory Cache Layer: Added a local memoryCache Map within src/lib/metrics-cache.ts.
• L2 Cache Integration: Modified the withMetricsCache function to:
  1. Retrieve from the local L1 memory cache first.
  2. Fall back to the L2 Upstash Redis cache if L1 is a miss, populating L1 upon retrieving L2 values.
  3. Load fresh data from the GitHub API if both L1 and L2 caches miss, saving the fresh result to both cache layers.
• Bypass Handling: Ensures cache bypassing parameters (e.g. refresh/sync/bypassCache) ignore both cache levels and rewrite the fresh data to both layers.

[vercel]

@YashKrTripathi is attempting to deploy a commit to the PRIYANSHU DOSHI's projects Team on Vercel.

A member of the Team first needs to authorize it.

[YashKrTripathi]

@Priyanshu-byte-coder If any changes are needed, please let me know and I’ll make them. If everything looks good and you’re satisfied with the implementation, please approve and merge it.

[Priyanshu-byte-coder]

1. Unbounded memory growth — module-level Map with no eviction. Add max-size cap (LRU at ~500 entries) or document serverless-only assumption. 2. value: any — use unknown with cast on return. Architecture (memory L1 → Redis L2) is sound, bypass logic correct.

[YashKrTripathi]

1. Unbounded memory growth — module-level Map with no eviction. Add max-size cap (LRU at ~500 entries) or document serverless-only assumption. 2. value: any — use unknown with cast on return. Architecture (memory L1 → Redis L2) is sound, bypass logic correct.

Status: Changes Implemented that you asked @Priyanshu-byte-coder ,

What I changed

• Added a bounded LRU for the L1 in-memory cache (max 500 entries) to avoid unbounded memory growth.
• Replaced the in-memory value type from any to unknown and use casts when returning cached values.
• Preserved existing bypass logic and L2 (Upstash Redis) behavior — L1 is a fast front layer only.

Files modified

• src/lib/metrics-cache.ts

Verification performed

• Ran npm run type-check — no type errors.
• Simulated a non-committing merge against the fork — no merge conflicts.

Branch / Commit

• Branch: l1-metrics-cache
• Commit message: "L1 metrics cache: add bounded LRU (500 entries) and tighten types"

[YashKrTripathi]

@Priyanshu-byte-coder PLEASE CHECK AND REVERT BACK!