- Add URLRegistry to track URL health and prevent re-fetching failed URLs
- Implement pool monitor to gracefully terminate when all URLs are exhausted
- Add context-aware token wait for clean shutdown on cancellation
- Improve HTTP request headers with realistic browser User-Agent and Sec-Fetch headers
- Expand URL list with additional test endpoints
- Add OpenSpec documentation for HTTP headers and URL health management