v0.10.0
This release fixes the long-standing "the file uploaded but the UI shows it failed" symptom on large uploads, plus a related cluster of race conditions and a couple of long-overdue ergonomics gaps. There are intentional contract changes — see Breaking changes at the bottom.
Fixed
- Concurrent
AssembleFileJobruns no longer corrupt each other.UploadChunkControllerdispatches the assembly job whenever the tracker reportsis_complete=true. The v0.9.3lockForUpdateonly protected the chunk-list write, not the subsequentisCompleteread, so two parallel chunk requests near the end of a large upload could each observe completion and each enqueue a job. The first job assembled the file and rancleanup(); the second job crashed mid-assemble()because the chunks were gone, dispatchedUploadFailed, and the user saw a failure even though the file was on disk. AddsUploadTracker::claimForAssembly()(CAS update fromPendingtoAssemblingon the database tracker, status-guarded write underflockon the filesystem tracker), andAssembleFileJobreturns silently when another worker already won the claim. - Concurrent client-side workers stop on the first
is_completeresponse. When the backend reportedis_complete=true, only the worker that received the response returned — the other workers continued POSTing their already-in-flight chunks, which fed the server-side race above. A sharedcompletedflag now bails out the remaining workers before the next request. DefaultChunkHandler::assemble()now works on every Flysystem driver. The previous implementation called$disk->path()and usedfopen()/mkdir()directly, which only works on the local driver — S3, GCS, and friends raised aRuntimeExceptionfor every assembly and the chunks were never cleaned up. Streams chunk-by-chunk throughreadStream()into asys_get_temp_dir()temp file, then uploads withwriteStream(). Memory stays at the 8 KB read buffer regardless of file size, and the temp file is unlinked even if an error is thrown. Missing chunks now raise a descriptiveRuntimeExceptionwith the chunk index instead of a warning-levelfopen()failure.ChunkUploaderresets its internal state after a successful upload. When the same instance was reused for a second file (the default forChunkDropzone,useChunkUpload, and any UI that holds a single uploader reference), the leftoveruploadIdfrom the previous run madeupload()enter the resume branch, hit/statuswith the stale id, and either upload nothing or throw. ClearsuploadId,pendingChunks,lastFile, andlastMetadatawhen emitting thecompleteevent.isComplete,progress, andcurrentFileare intentionally preserved so the UI can still display the finished file.- Late listeners receive a sticky
complete/errorreplay. BothChunkUploaderandBatchUploaderfiredcompleteanderrorsynchronously, so any listener that registered after the upload finished — for example because the parent component mounted while the upload was in flight — never received the event.on('complete', cb)andon('error', cb)now schedule the callback in a microtask if the event has already happened. The cache is cleared on the nextupload()and oncancel(). pause()/resume()/retry()no longer leak unhandled promise rejections. The fire-and-forgetthis.upload(...)call insideresume()andretry()had no.catch(), so a network failure surfaced as anUnhandledPromiseRejectionin browser devtools. The error itself was already delivered through theerrorevent, so we just swallow the rejection.- Per-chunk N+1 query is gone.
ChunkyManager::uploadChunk()previously calledmarkChunkUploaded+getMetadata+isComplete— three reads per chunk, on top of the chunk write. For a 1000-chunk upload that was 3000+ DB queries, and the unlockedisCompleteread was part of the assembly race above.markChunkUploaded()now returns the freshly updatedUploadMetadatafrom inside thelockForUpdatetransaction;uploadChunk()consumes that one snapshot. VerifyChunkIntegritymiddleware andDefaultChunkHandler::store()no longer buffer the chunk twice into memory. Each chunk request used to allocate2 × chunk_sizeof PHP heap (one read for the SHA-256, one for the disk write). The middleware nowhash_file()s the upload's temp path; the handler streams it viawriteStream(). Both fall back togetContent()when no temp file is available.BatchUploader.pause()actually pauses the batch worker loop. It used to pause only the active per-file uploaders; the outer worker loop kept pulling files from the queue and starting fresh uploaders. Adds a Promise-based barrier the loop awaits between files whenisPausedBatchis true.BatchUploader.cancel()resetsisCompleteand emits a dedicatedcancelevent ({ batchId }). Previously a cancel after the lastfileCompleteleftisComplete=true, and consumers had no way to tell apart "user cancelled" from "upload finished". The sticky event cache is also cleared so a late listener cannot replay a stale event after the user cancelled.BatchUploader.fetchJsoncaptures theAbortSignallocally. It used to readthis.abortController?.signalat await-time, which could attach a request to a freshly-replaced controller in destroy/cancel flows.- HTTP errors preserve the response body. Both
fetchJsonpaths used to collapse non-2xx responses intonew Error('HTTP {status}: {body}'), hiding Laravel validation arrays behind an opaque string. They now throwUploadHttpErrorwithstatusand parsedbodyfields. Existingerror.messageconsumers keep working. FilesystemTrackermutations are guarded byflock(). v0.9.3 fixed theDatabaseTrackerrace; the filesystem tracker still did a bare read-modify-write onmetadata.json, which dropped chunk indices under concurrent writes.markChunkUploaded,updateStatus, andclaimForAssemblynow run under an exclusive lock on a sibling.lockfile. The guard is best-effort: when the disk does not expose a local path (S3, etc.) the callback runs unguarded — that combination was already unsupported.- Batch completion broadcasts deduplicate. When several
AssembleFileJobworkers finished within the same tick, each one persisted the terminal status and dispatched aBatchCompletedevent, so the frontend received N notifications for one logical transition. The DB path now uses a CAS UPDATE that only matches non-terminal statuses; the filesystem path runs its check under the new batch flock.
Added
DELETE /api/chunky/upload/{uploadId}cancel endpoint (CancelUploadController). The frontendChunkUploader.cancel()now fires a backgroundDELETEagainst it so the chunks are released immediately instead of waiting for the expiration sweep.UploadStatus::Cancelledenum case.chunky:cleanupArtisan command. Removes expired uploads (chunk files + tracker metadata) for both database and filesystem trackers. Supports--dry-run. The previously-orphanedauto_cleanupconfig option is now respected — whentrue, the service provider schedules the command daily withwithoutOverlapping().UploadHttpErrorexported from@netipar/chunky-corewithstatus+ parsedbodyfor granular client error handling.BatchCancelEventevent onBatchUploaderand corresponding type export.chunk_indexvalidation now rejects values above the upload'stotal_chunks(resolved through the tracker).
Changed (Breaking)
UploadTrackercontract. Custom tracker implementations must update:markChunkUploaded()returnsUploadMetadatainstead ofvoid(the freshly updated snapshot from inside the lock).- New required methods:
claimForAssembly(string $uploadId): bool,expiredUploadIds(): array<int, string>,forget(string $uploadId): void.
- New
UploadStatus::Cancelledcase. Any consumer doing an exhaustivematchonUploadStatusneeds to add a branch. - New
DELETE /api/chunky/upload/{uploadId}route. If you publish and customiseroutes/api.php, re-runphp artisan vendor:publish --tag=chunky-routes(if you publish them) or merge the new route manually.
npm packages
- All packages bumped to
0.10.0(core, vue3, react, alpine). Sister packages now require@netipar/chunky-core: ^0.10.0.