Replace polling-based Redis semaphore with pub/sub notifications

[artugro]

Problem

RedisSemaphore.acquire() in src/workflow/utils/concurrency.py uses a busy-wait loop with asyncio.sleep(0.25). Under contention:

• Each waiter does 4 Redis ops (INCR + EXPIRE + DECR + TTL check) every 250ms
• 100 waiters = 1600 Redis ops/second just for polling
• Adds up to 250ms unnecessary latency per acquisition
• No backoff — fixed interval regardless of contention level

Proposed Solution

Replace the poll loop with Redis pub/sub notification:

1. When release() is called, publish to channel conc:notify:{key}
2. In acquire(), subscribe to the channel and await a message instead of polling
3. Keep the INCR/DECR counter approach for the semaphore itself
4. Keep a timeout deadline for safety

This eliminates wasted Redis ops and reduces acquisition latency from ~125ms avg to near-instant.

Files

• src/workflow/utils/concurrency.py