Skip to content

v1.0.9

Choose a tag to compare

@hkjang hkjang released this 03 Sep 15:11
· 22 commits to main since this release
fb2a8d0

Bound how many async jobs run at once. CreateJob started an unbounded go s.runJob(jobID, input) per request and the closure held the whole upload (up to 50MB) alive, so repeated POST /v1/jobs calls grew memory linearly with the number of in-flight goroutines and could exhaust the server. Service now owns a jobSlots semaphore sized by PII_MASKER_MAX_CONCURRENT_JOBS (default 4, falling back to 4 when configured at zero or below), and only a runner that acquired a slot executes. The value handed to the runner shrank to just upstage.ParseOptions, and a new loadJobInput reads the document back from job.InputPath — a field that until now was recorded but never read — after the slot is acquired, so a queued job costs one goroutine instead of one full document in memory and stays in queued state until its turn; a job whose stored input cannot be read fails with storage_read_failed. Adds an integration test that holds the upstream handler open and counts concurrent requests (limit 2, 5 jobs → never more than 2 at once, all 5 completed after release) plus 2 loadJobInput unit tests.