Month 4 Plan

Month 4 — Scale, Concurrency, and Performance

You take the system from "works in a demo" to "works under load too". The line between junior and mid-level usually runs right through here.

Learning core (both of you)

• Concurrency in Java (thread pool, shared state)
• Asynchronous work in Python
• Background task queue
• Caching strategies
• Cost and latency optimization
• Load testing

Concepts you must be able to explain

• Thread pool · [ ] Blocking/non-blocking operation · [ ] Race condition · [ ] Deadlock
• Task queue · [ ] Cache hit ratio · [ ] Latency vs throughput · [ ] p95/p99 · [ ] Token cost

Project steps

1. Move the agent task to the background (queue + status update)
2. Deliberate thread pool configuration in Java
3. Async flow in Python
4. Caching + hit ratio measurement
5. Token/cost/duration tracking
6. Load test: p50/p95/p99, throughput, cost per task
7. Document the before/after comparison — and run the eval at the same time

Why is this month after eval? Because without the eval infrastructure, you can't see whether a performance improvement broke the agent's success. The eval set is your safety net against the "got faster but broke" trap.

Division of labor

Split and swap at the rotation — you decide who takes what. Two rough tracks: (a) queue + concurrency (Java) + cost tracking, (b) async flow + cache (Python) + load test.

Content plan

• Before/after chart clip ("we halved the token cost")
• Live load-test recording
• "What is p95 latency, and why the average fools you"
• Cost transparency post

Retro