Skip to content

fix(s3): improve error on move/copy operations when bucket quota exceeded #59939

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mejo- wants to merge 1 commit into
base: master
Choose a base branch
from
Open

fix(s3): improve error on move/copy operations when bucket quota exceeded #59939

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions apps/dav/lib/Connector/Sabre/Directory.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,14 @@
use OC\Files\Utils\PathHelper;
use OC\Files\View;
use OCA\DAV\AppInfo\Application;
use OCA\DAV\Connector\Sabre\Exception\EntityTooLarge;
use OCA\DAV\Connector\Sabre\Exception\FileLocked;
use OCA\DAV\Connector\Sabre\Exception\Forbidden;
use OCA\DAV\Connector\Sabre\Exception\InvalidPath;
use OCA\DAV\Storage\PublicShareWrapper;
use OCP\App\IAppManager;
use OCP\Constants;
use OCP\Files\EntityTooLargeException;
use OCP\Files\FileInfo;
use OCP\Files\Folder;
use OCP\Files\ForbiddenException;
Expand Down Expand Up @@ -447,6 +449,8 @@ public function moveInto($targetName, $fullSourcePath, INode $sourceNode) {
if (!$renameOkay) {
throw new \Sabre\DAV\Exception\Forbidden('');
}
} catch (EntityTooLargeException $e) {
throw new EntityTooLarge($e->getMessage());
} catch (StorageNotAvailableException $e) {
throw new ServiceUnavailable($e->getMessage(), $e->getCode(), $e);
} catch (ForbiddenException $ex) {
Expand Down Expand Up @@ -482,6 +486,8 @@ public function copyInto($targetName, $sourcePath, INode $sourceNode) {
}

return true;
} catch (EntityTooLargeException $e) {
throw new EntityTooLarge($e->getMessage());
} catch (StorageNotAvailableException $e) {
throw new ServiceUnavailable($e->getMessage(), $e->getCode(), $e);
} catch (ForbiddenException $ex) {
Expand Down
2 changes: 2 additions & 0 deletions apps/files/src/actions/moveOrCopyAction.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,8 @@ export async function* handleCopyMoveNodesTo(nodes: INode[], destination: IFolde
logger.debug(`Error while trying to ${method === MoveCopyAction.COPY ? 'copy' : 'move'} node`, { node, error })
if (error.response?.status === 412) {
throw new HintException(t('files', 'A file or folder with that name already exists in this folder'))
} else if (error.response?.status === 413) {
throw new HintException(t('files', 'Insufficient storage, quota exceeded'))
Copy link
Copy Markdown
Contributor

@cuppett cuppett Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend the error here being "file too large" versus quota specifically. The conditions we're adding here are for bucket quota, but also bubbling "EntityTooLarge" which could happen under multiple, future conditions. This message is a lot more re-usable just indicating a size issue in general.

} else if (error.response?.status === 423) {
throw new HintException(t('files', 'The files are locked'))
} else if (error.response?.status === 404) {
Expand Down
6 changes: 6 additions & 0 deletions lib/private/Files/ObjectStore/ObjectStoreStorage.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
use OCP\Files\Cache\ICache;
use OCP\Files\Cache\ICacheEntry;
use OCP\Files\Cache\IScanner;
use OCP\Files\EntityTooLargeException;
use OCP\Files\FileInfo;
use OCP\Files\GenericFileException;
use OCP\Files\IMimeTypeDetector;
Expand Down Expand Up @@ -529,6 +530,11 @@ public function writeStream(string $path, $stream, ?int $size = null): int {
}

$stat['size'] = $totalWritten;
} catch (EntityTooLargeException $ex) {
if (!$exists) {
$this->getCache()->remove($uploadPath);
}
throw $ex;
} catch (\Exception $ex) {
if (!$exists) {
/*
Expand Down
54 changes: 48 additions & 6 deletions lib/private/Files/ObjectStore/S3ObjectTrait.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
use GuzzleHttp\Psr7\Utils;
use OC\Files\Stream\SeekableHttpStream;
use OCA\DAV\Connector\Sabre\Exception\BadGateway;
use OCP\Files\EntityTooLargeException;
use Psr\Http\Message\StreamInterface;

trait S3ObjectTrait {
Expand Down Expand Up @@ -94,6 +95,24 @@ private function buildS3Metadata(array $metadata): array {
return $result;
}

private function isStorageFullException(\Throwable $e) {
while ($e !== null) {
if ($e instanceof AwsException) {
// MinIO: dedicated error code for storage-full
Copy link
Copy Markdown
Member Author

@mejo- mejo- Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was only able to test this with rustfs so far. The MinIO error code is taken from the MinIO code search.

Would be good to put a breakpoint here and debug with AWS S3 and other S3 providers if possible, in order to find out their exact error code/message and adjust the conditions here.

Copy link
Copy Markdown
Contributor

@cuppett cuppett Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazon S3 buckets have no maximum size or object count limit.

There is an EntityTooLarge error for individual PutObject requests > 5GB. Multipart uploads can be up to 50TB now.

if ($e->getAwsErrorCode() === 'XMinioStorageFull') {
return true;
}
// RustFS: returns generic error code but with a recognisable message
if (str_starts_with($e->getAwsErrorMessage() ?? '', 'Bucket quota exceeded.')) {
return true;
}
}
$e = $e->getPrevious();
}

return false;
}

/**
* Single object put helper
*
Expand Down Expand Up @@ -121,7 +140,14 @@ protected function writeSingle(string $urn, StreamInterface $stream, array $meta
$args['ContentLength'] = $size;
}

$this->getConnection()->putObject($args);
try {
$this->getConnection()->putObject($args);
} catch (AwsException $e) {
if ($this->isStorageFullException($e)) {
throw new EntityTooLargeException('Quota exceeded on S3 storage', 0, $e);
}
throw $e;
}
}


Expand Down Expand Up @@ -198,6 +224,10 @@ protected function writeMultiPart(string $urn, StreamInterface $stream, array $m
$this->getConnection()->abortMultipartUpload($uploadInfo);
}

if ($this->isStorageFullException($exception)) {
throw new EntityTooLargeException('Quota exceeded on S3 storage', 0, $exception);
}

throw new BadGateway('Error while uploading to S3 bucket', 0, $exception);
}
}
Expand Down Expand Up @@ -290,12 +320,24 @@ public function copyObject($from, $to, array $options = []) {
'params' => $this->getServerSideEncryptionParameters() + $this->getServerSideEncryptionParameters(true),
'source_metadata' => $sourceMetadata
], $options));
$copy->copy();
try {
$copy->copy();
} catch (\Throwable $e) {
if ($this->isStorageFullException($e)) {
throw new EntityTooLargeException('Quota exceeded on S3 storage', 0, $e);
}
}
} else {
$this->getConnection()->copy($this->getBucket(), $from, $this->getBucket(), $to, 'private', array_merge([
'params' => $this->getServerSideEncryptionParameters() + $this->getServerSideEncryptionParameters(true),
'mup_threshold' => PHP_INT_MAX,
], $options));
try {
$this->getConnection()->copy($this->getBucket(), $from, $this->getBucket(), $to, 'private', array_merge([
'params' => $this->getServerSideEncryptionParameters() + $this->getServerSideEncryptionParameters(true),
'mup_threshold' => PHP_INT_MAX,
], $options));
} catch (AwsException $e) {
if ($this->isStorageFullException($e)) {
throw new EntityTooLargeException('Quota exceeded on S3 storage', 0, $e);
}
}
}
}

Expand Down
3 changes: 3 additions & 0 deletions lib/private/Files/Storage/Common.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
use OCP\Files\Cache\IScanner;
use OCP\Files\Cache\IUpdater;
use OCP\Files\Cache\IWatcher;
use OCP\Files\EntityTooLargeException;
use OCP\Files\FileInfo;
use OCP\Files\ForbiddenException;
use OCP\Files\GenericFileException;
Expand Down Expand Up @@ -530,6 +531,8 @@ public function copyFromStorage(IStorage $sourceStorage, string $sourceInternalP
try {
$this->writeStream($targetInternalPath, $source);
$result = true;
} catch (EntityTooLargeException $e) {
throw $e;
} catch (\Exception $e) {
Server::get(LoggerInterface::class)->warning('Failed to copy stream to storage', ['exception' => $e]);
}
Expand Down
Loading