Repository navigation
v1.7.2
Finalizing a large job no longer runs out of memory, a job whose executor dies is picked up again by itself, and the corrected copies keep the quality — and the size — of the pages that were scanned.
Finalizing
- A job of any size finishes. The cover pages are written in batches, each in a process of its own that stops once it has used
MAX_RAM_GB, so the memory the imaging libraries do not give back goes back to the system between batches. A 546-copy job used to be killed by the container at copy 314 and left in FINALIZING. A batch that is killed anyway costs its copy and the next one resumes after it (#255). - A run that starts again only redraws what changed. Each copy records what its graded page was drawn from, so an interrupted finalization resumes instead of redoing the whole job, and correcting two grades redraws two pages (#255).
- The graded page no longer overwrites the scan. It is written beside it, under
cover_pages/<job>/graded/, and the merge falls back to the scan for a copy whose table could not be read. The correction screens therefore always show the page as it was scanned (#255).
Grades on the cover page
- They are written where the table actually is. The overlay used the position the template records while the cells came from whichever widened crop matched, so every copy that needed a retry had its grades drawn about 25 px off — 472 of the 546 copies of the last exam (#255).
- Tables the old retries never found are found now. Widening the crop evenly never reaches a table that sits left of the recorded box; the crops tried now include one-sided ones. Replayed over 546 real cover pages: 546 found, against 543 before (#255).
- They are written in the page, not over a picture of it. The page was rasterised at 300 dpi and re-encoded, which turned a 94 kB scan into 1.4 MB and lost what the scanner had captured. The grades are now text in the page: 185 kB for a page whose scan is 184 kB. For a 546-copy job that is about 85 MB of graded pages instead of 664 MB, and the Moodle download goes back to two parts (#259, after #257 had raised the jpeg quality that this removes entirely).
Cluster
- A job whose executor died is requeued on a clock. The sweep that does it used to run only when an executor ended, and an executor only starts when the queue is not empty, so a crashed job waited for somebody else to submit one. The
idle-sweepCronJob runs it every ten minutes, and a dead finalization comes back as the status the executor acts on (#255). - The storage is a directory of the node. NFS on a single-node cluster is a loopback mount that stalls every few days; it becomes an opt-in kustomize component, selected with
scripts/deploy.sh --storage nfs(#256). - One queued job starts one executor. The queue still showed a job during the two and a half seconds its executor spent importing OpenCV and LiteRT, so the scaler started a second one. The tasks are imported once the job has been taken, and the scaling strategy now counts what is running (#258).
Web application
- The histogram stops at the maximum of the question. A question marked out of 3.5 ended on a bar at 4 that no copy could reach (#254).
Deploying
kubectl apply -k deployment/ for the storage and the scaler; the executor changes need the image. This tag builds the versioned images but does not move :main — publish manually with "Push images" checked for that.