[Docs] Add Service Discovery Pattern Documentation (Watch API, Stale Read, Client Caching)

[JoshuaChi]

Background

Based on research of embedded etcd/Raft use cases in production (CoreDNS, SkyDNS, Consul), the service discovery pattern represents ~20% of d-engine's target users. These applications have specific requirements:

• Read-heavy workload: DNS queries are 99% reads
• Stale read acceptable: 50ms staleness is tolerable for most DNS use cases
• Watch mechanism: Need to monitor key changes and update local cache
• Separation of concerns: Read-only replicas, external controller handles writes

Current Gap

While d-engine has implemented ReadConsistencyPolicy and related APIs, we lack comprehensive documentation for developers building service discovery systems.

Goal

Create documentation suite for Type 2 users (DNS/Service Discovery developers):

1. Watch API Guide (if implemented) or Roadmap (if planned)

   • How to watch key changes
   • Event notification mechanism
   • Error handling and reconnection
   • Example: DNS cache invalidation on service endpoint changes

2. Stale Read Performance Guide

   • Performance comparison: Linearizable vs Lease vs Eventual
   • Latency/throughput benchmarks
   • When to use each consistency level
   • Trade-offs and best practices

3. Client-Side Caching Best Practices

   • Cache invalidation strategies
   • TTL recommendations
   • Memory footprint considerations
   • Example: CoreDNS-style caching pattern