Encrypted external WebDAV storage not working in NC 13.0.3, OCP\Encryption\Exceptions\GenericEncryptionException: Missing Signature

[lje]

Steps to reproduce

1. Enable server side encryption module (Default encryption module)
2. Leave all settings at default (external storage encrypted, local storage not encrypted)
3. Mount an external WebDAV folder using a normal user, allow sharing for that folder. In my example WebDAV provider is https://webdav.magentacloud.de/

Expected behaviour

Folder should work

Actual behaviour

Upload to the remote folder works, download doesnt work. NC's Error log has lot of entries stating
OCP\Encryption\Exceptions\GenericEncryptionException: Missing Signature (complete error message will be at the end)
Looking directly at the WebDAV storage (using an WebDAV client) reveals that the file was uploaded successfully and is encrypted, but there are no hidden signature files

Server configuration

Operating system:
Linux 2.6.32 x86_64

Web server:
Apache 2.2

Database:
MySQL 5.1.73

PHP version:
7.0.29

Nextcloud version: (see Nextcloud admin page)
13.0.3 (same issue with 13.0.2)

Updated from an older Nextcloud/ownCloud or fresh install:
Updated from 12.x

Where did you install Nextcloud from:
Web based installer https://download.nextcloud.com/server/installer/setup-nextcloud.php

Signing status:

No errors have been found.

Login as admin user into your Nextcloud and access 
http://example.com/index.php/settings/integrity/failed 
paste the results here.

List of activated apps:

All default + encryption module

If you have access to your command line run e.g.:
sudo -u www-data php occ app:list
from within your Nextcloud installation folder

No sudo (shared host)

Detailled error message:
OCP\Encryption\Exceptions\GenericEncryptionException: Missing Signature

/var/www/virtual/<path>/apps/encryption/lib/Crypto/Crypt.php - line 529: OCA\Encryption\Crypto\Crypt->hasSignature('---------------...', 'AES-256-CTR')
/var/www/virtual/<path>/apps/encryption/lib/Crypto/Crypt.php - line 462: OCA\Encryption\Crypto\Crypt->splitMetaData('---------------...', 'AES-256-CTR')
/var/www/virtual/<path>/apps/encryption/lib/Crypto/Encryption.php - line 380: OCA\Encryption\Crypto\Crypt->symmetricDecryptFileContent('---------------...', 'D\x08!\x8E\x99\x99$\x08`fw\xA8\xDF\xA0H...', 'AES-256-CTR', 1, 0)
/var/www/virtual/<path>/lib/private/Files/Stream/Encryption.php - line 464: OCA\Encryption\Crypto\Encryption->decrypt(*** sensitive parameters replaced ***)
/var/www/virtual/<path>/lib/private/Files/Stream/Encryption.php - line 295: OC\Files\Stream\Encryption->readCache()
[internal function] OC\Files\Stream\Encryption->stream_read(8192)
/var/www/virtual/<path>/apps/files_external/3rdparty/icewind/streams/src/Wrapper.php - line 83: fread(Resource id #55, 8192)
/var/www/virtual/<path>/apps/files_external/3rdparty/icewind/streams/src/CallbackWrapper.php - line 91: Icewind\Streams\Wrapper->stream_read(8192)
[internal function] Icewind\Streams\CallbackWrapper->stream_read(8192)
/var/www/virtual/<path>/3rdparty/sabre/http/lib/Sapi.php - line 80: stream_copy_to_stream(Resource id #58, Resource id #59, '5961980')
/var/www/virtual/<path>/3rdparty/sabre/dav/lib/DAV/Server.php - line 498: Sabre\HTTP\Sapi sendResponse(Object(Sabre\HTTP\Response))
/var/www/virtual/<path>/3rdparty/sabre/dav/lib/DAV/Server.php - line 254: Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))
/var/www/virtual/<path>/apps/dav/lib/Server.php - line 287: Sabre\DAV\Server->exec()
/var/www/virtual/<path>/apps/dav/appinfo/v2/remote.php - line 35: OCA\DAV\Server->exec()
/var/www/virtual/<path>/remote.php - line 164: require_once('/var/www/virtua...')
{main}

[lje]

Issue still present in Nextcloud 13.0.4

[wiserweb]

Issue still present in Nextlcoud 13.0.6 and confirming this exact error message occurs with S3 External Storage as well when encryption is enabled.

[lje]

Issue still present in NC 14.0.3. Apparently, no one cares.

[globalcow]

Same for me

[globalcow]

Is there someone who can take a look at this issue or give us any feedback? Perhaps @rullzer ?

[cortopy]

Experiencing same issue. All folders work, except for the one I shared. Issue happens when syncing the folder with a device that didn't have that folder. So like others, seems to be about downloading files within shared folder

[martink-p]

Please see my post/issue regarding ownCloud here:
owncloud/core#34599
I bet it is the same problem.

[kettnsaeg]

Issue is sill present in 15.05.3.

BUT:

I - please note that I am a noob in anything regarding coding - can confirm that martink-p's solution for owncloud as presented at #34599 refers to the same problem and is a fix for this issue in the nextcloud family.

Setup:
Nextcloud 15.0.5.3
fresh install via Web Based Installer, updated,
PHP 7.3.2,
External Storage is a IONOS HiDrive connected via WebDAV,
serverside encryption for the external storage enabled

[micha-09]

Issue is sill present in 16.00.0

[lje]

So, this bug is unfixed for a year now. Yay...
Should we just give up on Nextcloud encryption?

[martink-p]

Uhm, Why would you give up?! I came up with a solution in February, which was even confirmed by another user (kettnsaeg).
I think you guys should finally merge my code into your project and close that weird bug...

Best regards,
Martin.

[kesselb]

@martink-p mind to open a pull request with your changes?

[martink-p]

@kesselb done. I've opened a pull request: #15946

[Woosah]

Sorry in advance for commenting on a closed issue!

If I'm right, the fix by @martink-p is merged into the master branch for NC17. At the moment I am on NC16.04 (production channel) and I have many users with files on encrypted external storages. Since yesterday, some are now facing several "bad-sig"-errors for about 20-25 random files, and I'd like to know if I can do something NOW to get these decrypted and accessable again?

Any help at the present moment would be appreciated to solve the issue until an update of NC will be available that fixes this once and for all...

Thanks and kind regards!

[wisemonkey]

With NC 17 and S3 storage as primary, I see following error
OCP\Encryption\Exceptions\GenericEncryptionException: Bad Signature
Are these related? Does NC17 still has issue with encryption with external primary storage?