Skip to content

v1.20.0

Choose a tag to compare

@ravirajsinh45 ravirajsinh45 released this 23 Oct 14:17
· 43 commits to main since this release
bd7236d

A rebuilt upload experience. Files now upload through a queue instead of all at once, so a large drop finishes whole files rather than leaving many half-sent; uploads belong to the account that added them and survive account switches, sign-outs, files changed on disk, stalled parts and storage outages; the browser warns before you leave a page with uploads in flight, and Log out asks first; and the Uploads panel stays responsive under hundreds of files. This release also fixes a security issue in the sign-in redirect (see Security), and carries fixes to project settings, the notification drawer and failed notification fetches.

Upgrade notes

  • NEXT_PUBLIC_UPLOAD_CONCURRENCY now counts parts for the whole browser tab, not for each file. If you raised it so that several files at once would upload faster, those files now share that many parts and take turns; a single file uploads as fast as before. Values above 6 only help when the storage endpoint speaks HTTP/2, because over HTTP/1.1 a browser opens at most 6 connections per host. The setting is still read at build time, so changing it needs a web image rebuild. (#535)
  • A bucket lifecycle rule that aborts incomplete multipart uploads after a few days is now recommended. Starting an upload now gives storage a single try, and a try that times out after storage had already created the upload leaves an empty multipart upload behind. docs/deployment.md shows the rule and how to apply it. The rule counts days from when an upload started, so pick a number above the longest upload you expect people to keep resuming. (#535)
  • Reload browser tabs left open across the upgrade. The API now answers a storage outage during an upload with a coded error that only the new web client reads. A tab still running the old client shows that error's raw text on the row. (#528)

Added

  • The browser warns before you leave a page while uploads are on their way. Reloading, closing the tab or navigating away while any of your files in that tab is waiting, starting, sending or finishing shows the browser's own leave-page prompt, so a stray reload no longer stops uploads without a word. It is best effort: desktop browsers show it only after you have interacted with the page, with their own text, and iOS Safari never shows it. Being sent to the sign-in page because the session ended never prompts. (#532)
  • Log out asks first when uploads would stop. With files waiting or sending in this tab, Log out asks how many will stop ("3 uploads will stop. Log out anyway?") before anything is signed out. Cancel keeps the session and the uploads. The count is this tab's uploads only: Log out signs the account out of every tab, so uploads of the same account in other tabs stop too, their waiting files marked Failed and their sending ones Interrupted. With nothing on its way in this tab, Log out works as before. (#532)
  • Cancel all queued, and Retry on a failed upload. "Cancel all queued" in the Uploads panel removes every file that has not started and leaves the ones already uploading alone. A Resume still waiting for its turn goes back to Interrupted, its parts kept, rather than being removed. A file the server refuses, for example because the storage limit is reached or its project was deleted, is marked Failed with the reason and a Retry button, offered while the page that added it is still open. Retry, like Resume on an interrupted upload, goes next, ahead of the files that have not started; a Resume waiting for its turn shows "Waiting to resume". (#535)

Changed

  • Uploads wait in a queue instead of all starting at once. Dropping or picking many files used to start every one of them at the same moment, splitting the bandwidth between them, so stopping part way left many half-sent files and none finished. Now up to 3 files are active at a time, in the order they were added, and the earliest file gets the bandwidth first, so a stop leaves whole finished files behind it. The queue covers the project page, folder drops, the Upload dialog and New Version. Files that have not started show "Waiting to start" in the Uploads panel and have nothing on the server until their turn, and on an asset's page a waiting new version shows "New version queued, waiting for other uploads". The panel's header counts them apart ("3 active · 2 queued"). If storage or the connection cannot be reached when a file's turn comes, the panel shows "Waiting for storage" or "Waiting for connection", the files behind it wait, and the queue tries again by itself, waiting longer each time up to a minute, and carries on once storage or the connection answers again, without failing any file. A file the server refuses fails on its own and the queue moves on. The screen is kept awake only while files are starting, sending or finishing, so a laptop can sleep while files only wait, which pauses the queue until it wakes. GET /upload/{version_id}/parts also answers server_time, which a queued Resume uses to tell whether the upload is still running somewhere else. (#535)
  • NEXT_PUBLIC_UPLOAD_CONCURRENCY limits the whole browser tab, not each file; the default stays 5. See the upgrade note. (#535)

Fixed

  • Starting an upload while storage is unreachable now answers within about 15 seconds. It used to hang for minutes and then fail the file with a bare server error. The storage call that starts an upload, for a new file or a new version, is now sent once, with a 5 second connect limit and a 10 second read limit. When storage cannot be reached, times out, cuts the TLS handshake short (a proxy in front of storage that is restarting), or is shedding load or throttling (a 429, or a throttling code such as SlowDown or TooManyRequests), the API answers 503 with the code storage_unavailable and Retry-After: 5, which the browser can now read, and the file waits as "Waiting for storage" until storage answers again. GET /upload/{version_id}/parts and POST /upload/complete answer the same coded 503 when they cannot reach storage, so a Resume waiting for its turn says "Waiting for storage" too. When storage answers and refuses (access denied, a missing bucket, a call it does not implement), or its TLS setup is wrong (a certificate that does not validate, a protocol version mismatch), the API answers 500 with the code storage_error, and the file fails with "Storage refused the upload. Ask your admin to check the storage settings." (#528)
  • An upload part that stops moving is now retried instead of hanging. A part that has sent some of its bytes and then makes no progress for 90 seconds is cancelled and sent again, with the usual retry backoff. A part that has not sent its first byte may only be waiting for a connection, so it is given up on after 90 seconds only when nothing else in the tab has moved either, and otherwise after 5 minutes. A part whose last byte has gone out waits up to 5 minutes for storage to answer. Before, such a part was waited on for good, and its upload sat at the same percentage with no error. Slow uploads are not affected, because the clock restarts with every bit of progress. (#530)
  • A file that was changed, replaced or moved after it was added now fails clearly instead of hanging. A file picked or dropped into the browser is a handle on the file on disk, not a copy, so changing it before its upload had read it left a stale handle, and Safari hung on it with no error and no progress. Each file is now checked just before it is sent. A file that cannot be read shows "The file changed or was moved after it was added. Add it again.", nothing is created on the server for it, and the queue goes on to the next file. A file that goes away while it is sending, with parts still to send, ends as Interrupted with the same message. (#530)
  • An upload part that the server refuses for good now fails at once with the server's reason. When the request to send a part was refused because the upload or its file is gone or no longer yours (403, 404 or 409), the panel tried it up to eight times and then offered a Resume that could never work. The upload is now marked Failed with the server's message straight away, without Resume. Lost connections, server errors and rate limits are still retried. (#510)
  • Discard cleans up an upload in storage even when its version is already gone. When the version no longer exists on the server, for example because its project was deleted, Discard used to drop the row and leave the parts already sent in storage until the overnight cleanup. It now still sends the abort. Uploads started before this release are not covered, because they did not record the storage key Discard needs. Discarding an upload in a project in the trash, or as an uploader who is no longer an editor of the project, aborts it in storage at once and leaves its version to the stale-upload reaper, which removes the version and its asset together; it used to remove the version and keep the asset, so restoring the project showed a card with no versions that could not be opened. (#510)
  • The Uploads panel stops polling an upload whose file was deleted. If the file was moved to the trash, or its project was deleted, the row used to stay on Processing forever. It now shows "This file is no longer available." and leaves the active list, and Dismiss clears it. A dropped connection or a server error says nothing about the file, so the panel keeps polling through those. (#509)
  • Uploads now belong to the account that added them. The Uploads panel, its count and the sidebar badge used to show every upload saved in the browser, so after one person signed out and another signed in on the same browser, the second person saw the first person's uploads and could resume or discard them. The panel now shows only the signed-in account's uploads. Another account's uploads are hidden, not deleted, and come back when that account signs in again. Uploads saved by an earlier release are shown to an account only after the server confirms the upload is still that account's to finish. (#531)
  • An upload never lands in an account that signed in during it. An upload that was sending when the account changed used to carry on with the new account's sign-in. Every request an upload makes is now tied to the account that started it. When another tab signs out, files that had not started are marked Failed with "Signed out in another tab. Add these files again after signing in.", and files that were sending stop as Interrupted, so their owner can resume them later. When the session ends in the tab itself (its sign-in expired or was revoked, or Log out), the files that had not started are marked Failed with "Your session ended. Add these files again after signing in." and saved before the page goes to the sign-in page, so they are listed again after signing back in instead of disappearing. When another account signs in, files that had not started are marked Failed with "Another account signed in. Add these files again after signing back in." Dropping files with no one signed in shows "Sign in to upload" and adds nothing. (#531)
  • Two tabs that press Resume on the same upload at the same moment no longer both send it. Each could miss the other's start, so both sent every part and one could end Failed with parts missing. Now only one of them sends it, and the other shows it as uploading elsewhere. (#531)
  • Large drops no longer flood the API or slow the page down. Once one of a project's uploads had finished in the browser, every status change of every upload in it refetched the project's asset grid, two reads each, and every status change re-rendered the whole project page, and every progress update re-rendered every row of the Uploads panel. A drop of a few hundred small files could use up the account's limit of 600 API reads a minute, after which the page's reads were refused until the minute ran out, and kept the browser busy for most of the drop. While uploads are on their way the grid now refetches at most once every 2 seconds, and at once when the last of them finishes, so a single small upload still shows Ready as soon as it is; a status change no longer re-renders the page, and a progress update re-renders only its own row. (#533)
  • A finished upload's time in the Uploads panel counts from when it finished. It counted from when the file was added, so a file that waited in the queue, or an old upload resumed today, said "25 minutes ago" or "3 days ago" the moment it finished. (#533)
  • Project settings keep a chosen poster and unsaved edits while open. Choosing a poster used to flash the preview and then drop it on return from the file picker, so Save uploaded nothing; name and description edits were lost the same way. Reopening the dialog now starts from the saved project. (#465, #512 by @jay23606)
  • The notification drawer no longer covers an expanded sidebar. (#513 by @jay23606)
  • A failed notifications fetch no longer throws an uncaught error. Opening Notifications in a tab whose session had ended in another tab, or any failed GET /me/notifications (offline, 429, 500), logged Uncaught (in promise) ApiError and showed the Next error overlay in development. The notifications store now handles the failure, keeping the current list and unread count and clearing the loading state. (#522, #534 by @ColumbusLabs)

Security

  • The sign-in page could be used to send a visitor off-site or run script in the app. A crafted /login?from=... link could redirect a signed-in visitor to another origin, or run script in the app through a javascript: target. The page now follows from only when it is a path inside the app. (GHSA-vq88-94mj-28ww, #519)

Contributors

Thanks to @jay23606 and @ColumbusLabs for contributions in this release.