[Bug]: files_external AmazonS3::test() uses HeadBucket which is rejected by Backblaze B2 bucket-restricted application keys

[jg-shinji-ueda]

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
• Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• I agree to follow Nextcloud's Code of Conduct.

Bug description

Summary

apps/files_external/lib/Lib/Storage/AmazonS3.php::test() calls headBucket() to verify the configured external S3 storage is reachable. The Availability storage wrapper invokes test() and, when it throws, records available=false in the storage availability cache. Subsequent file operations short-circuit to HTTP 503 until RECHECK_TTL_SEC (10 min) and the next test() again succeeds.

Backblaze B2 (which Nextcloud supports as an S3-compatible target via the AmazonS3 external storage backend) returns HTTP 403 Forbidden for HeadBucket on any application key that has a bucket restriction (i.e., any key created with bucketIds), regardless of the key's capabilities. Therefore an external storage mount backed by a B2 bucket-restricted key is permanently marked unavailable, even though the key can perform all relevant file operations (Get / Put / List / Delete) on that bucket.

Affected version

Nextcloud Server 32.0.9 (confirmed). The same code path exists in older versions and main.

Steps to reproduce

1. Create a Backblaze B2 application key restricted to a single bucket, e.g.:

   b2 key create --bucket <bucket-name> mykey listFiles readFiles writeFiles deleteFiles readBuckets writeBuckets listAllBucketNames

2. In Nextcloud, configure an external storage mount with the "Amazon S3" backend:
   • Bucket: <bucket-name>
   • Hostname: s3.<region>.backblazeb2.com
   • Region: B2 region (e.g., us-west-004)
   • Use SSL / use path-style: enabled
   • Authentication: Access key with the application key id and key from step 1
3. Open the Files app or run occ files_external:verify <mount-id>.

Expected behavior

The mount is usable. test() succeeds whenever the underlying credentials can list / read / write objects in the configured bucket, even if the S3-compatible service does not grant HeadBucket on that credential.

Current behavior

occ files_external:verify <mount-id>:

- status: error
- code: 1
- message: Aws\S3\Exception\S3Exception: Error executing "HeadBucket" on "https://s3.us-west-004.backblazeb2.com/<bucket>":
   (client): 403 Forbidden (Request-ID: ...)

nextcloud.log:

files_external · OCP\Files\StorageNotAvailableException
Creation of bucket "<bucket>" failed. Error executing "CreateBucket" on "https://s3.us-west-004.backblazeb2.com/<bucket>":
AWS HTTP error: AccessDenied (client): not entitled

(The CreateBucket attempt is S3ConnectionTrait::getConnection()'s autocreate branch, separately fixable via verify_bucket_exists=false. After fixing that, the Availability wrapper still fails on test() -> HeadBucket and permanently marks the storage unavailable.)

Root cause

apps/files_external/lib/Lib/Storage/AmazonS3.php::test():

public function test(): bool {
    $this->getConnection()->headBucket([
        'Bucket' => $this->bucket,
    ]);
    return true;
}

lib/private/Files/Storage/Wrapper/Availability.php calls test() and records unavailability on exception. RECHECK_TTL_SEC = 600.

On Backblaze B2, the S3-compatible API returns 403 Forbidden for HeadBucket on any application key with a bucket restriction. Verified on a real B2 account with five key shapes (master, MBAK 4-bucket, SBAK 1-bucket full caps, SBAK 1-bucket without listAllBucketNames, SBAK 1-bucket with only readBuckets); HeadBucket returned 403 in all cases. The same keys returned 200 for ListObjectsV2 (MaxKeys=1), GetBucketLocation, PutObject, GetObject, DeleteObject on the same bucket.

Proposed fix

Replace HeadBucket in test() with a check that does not require the HeadBucket privilege but that any working external-storage credential already has. ListObjectsV2(MaxKeys=1) works across AWS S3, MinIO, Wasabi, and Backblaze B2:

public function test(): bool {
    $this->getConnection()->listObjectsV2([
        'Bucket' => $this->bucket,
        'MaxKeys' => 1,
    ]);
    return true;
}

GetBucketLocation is also a viable equivalent on B2; ListObjectsV2 is more universally accepted across S3-compatible services and matches how the existing getFolderContents() and unlink() paths in the same file already exercise the bucket.

Workaround in user environment

Until this is fixed upstream, B2-backed external storage mounts are unusable in Nextcloud. The B2 bucket can still be used as the primary object store (config.php objectstore section) because lib/private/Files/ObjectStore/S3ConnectionTrait uses a separate verify_bucket_exists flag for the bucket-existence check and does not invoke headBucket from a Storage::test()-equivalent path.

Additional references

• apps/files_external/lib/Lib/Storage/AmazonS3.php (line ~594, current test())
• lib/private/Files/Storage/Wrapper/Availability.php (RECHECK_TTL_SEC)
• lib/private/Files/ObjectStore/S3ConnectionTrait.php (parallel pattern for object store, uses verify_bucket_exists rather than calling HeadBucket unconditionally)
• Backblaze B2 application key documentation (bucket restriction limits HeadBucket on S3-compatible API)

Steps to reproduce

1. use external strage. backblaze. B2
2. setup external strage

Expected behavior

can upload files.

Nextcloud Server version

32

Operating system

None

PHP engine version

None

Web server

None

Database engine version

None

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

List of activated Apps

Nextcloud Signing status

Nextcloud Logs

Additional info

No response

[jg-shinji-ueda]

Related but not duplicate, I think.. :

• S3: High connectivity check rate #16623 (closed/stale) reports the same HeadBucket call as a cost concern, but did not identify the structural issue with bucket-restricted keys.
• [Bug]: Nextcloud tries to CreateBucket even when auto-create is disabled using S3 as primary storage #36427 covers the parallel CreateBucket / autocreate problem for primary S3 objectstore, which is a related but distinct code path.
• [Bug]: Can't add S3 external storage #39650 (closed/not planned) describes the symptom (B2 external storage unusable) without root-cause analysis; this Issue provides the missing analysis + fix.

[joshtrichards]

listBuckets + listAllBucketNames maybe?

[jg-shinji-ueda]

Thanks @joshtrichards — we tested that approach on a clean B2 contract and it works, but with a strict prerequisite that probably needs to be documented in any PR. Sharing the evidence in case it's useful for the patch.

Test environment

• Backblaze B2 S3-compatible endpoint: s3.us-west-004.backblazeb2.com
• Per-contract Master Application Key (MBAK) restricted to a fixed bucketIds set (4 contract-scoped buckets out of 6 total in the B2 account)
• Nextcloud 32.0.9 (snap, current upstream code path)
• AWS CLI used for raw S3 API verification, plus Nextcloud files_external:verify + WebDAV PUT for the real-world behavior on top

4-pattern comparison

#	Call	MBAK capabilities	Result	Conclusion
1	aws s3api list-buckets --endpoint-url <b2-endpoint>	readBuckets,listAllBucketNames,...	HTTP 200, returned **	The proposed code path works, and listAllBucketNames does not leak unrelated buckets — the restriction is honored
2	aws s3api head-bucket --bucket <contract-scoped-bucket>	same as #1	HTTP 403 Forbidden	Reproduces the current upstream failure (the reason AmazonS3::test() falls into the CreateBucket fallback chain)
3	aws s3api list-buckets	MBAK without listAllBucketNames	AccessDenied "not entitled to read bucket list"	The proposed path fails without this capability — see prerequisite below
4	Nextcloud files_external:verify <id> (current headBucket → CreateBucket path)	same MBAK as #1	Creation of bucket "<name>" failed. ... InvalidArgument (client): Location constraint specifies another region, cross-region requests are not supported	The current failure mode confirmed end-to-end
4'	WebDAV PUT /remote.php/dav/files/<user>/<mount>/probe.txt	same	HTTP 503 Storage with mount id <id> is not available	User-visible symptom that #4 produces
4''	WebDAV PUT /remote.php/dav/files/<user>/probe.txt (primary storage with verify_bucket_exists=false)	same	HTTP 201	Confirms the underlying S3 read/write actually works against B2 — the breakage is purely in test()'s reliance on HeadBucket

Prerequisite the PR should document

The proposed listBuckets path only works for B2 MBAKs that include listAllBucketNames in their capabilities (pattern #3 above). Default b2 create-key invocations don't include it, so the requirement is non-obvious.

Example MBAK creation that satisfies the proposed test():

b2 key create \
  --bucket sutenu-prd-<hash>-{configdbbackups,logs,primary,shared00,...} \
  <key-name> \
  readFiles,deleteFiles,listFiles,readBuckets,listAllBucketNames,writeBuckets,writeFiles

(writeFiles is for read-write contracts; drop it for read-only.)

The bucketIds restriction stays in effect, so listBuckets returns exactly the contract-scoped subset and the mount-time existence check passes without ever calling HeadBucket or CreateBucket.

Suggested wording for the S3 backend docs

"When using Backblaze B2 with a per-contract Application Key, the key must include the listAllBucketNames capability in addition to the per-bucket read/write capabilities. Without it, AmazonS3::test() cannot validate the bucket exists and the mount will fail."

Happy to help

If you'd like, we can rerun the same test matrix against a draft PR (we have the contract + bucket fixtures ready and the test harness is automated). Just ping this thread.