occ user:delete does not remove storage and files where using Swift as primary store

[vpecinka]

Steps to reproduce

1. Set up Nextcloud using Swift as a primary storage
2. Login as admin and create user (e.g. tester)
3. Login as tester and add some files (for sure, there are the default ones)
4. As www-data user run cmd: php occ user:delete tester

Expected behaviour

The user and its oc_storage and oc_filecache should be deleted

Actual behaviour

The users is deleted, however the oc_storages and oc_filecache stil remain, together with files in the object storage.

Pre conditions:

MySQL [nextcloud_v13]> select * from oc_storages;
+------------+---------------------------------------------------+-----------+--------------+
| numeric_id | id                                                | available | last_checked |
+------------+---------------------------------------------------+-----------+--------------+
|          1 | object::store:nc0                                 |         1 |         NULL |
|          2 | object::user:admin                                |         1 |         NULL |
|          4 | object::user:775DA7E6-9157-48EE-92AD-DE87D45E22F1 |         1 |         NULL |
|          5 | object::user:tester                               |         1 |         NULL |
+------------+---------------------------------------------------+-----------+--------------+
5 rows in set (0.01 sec)

Delete user:

www-data@9264da61a860:/www/nextcloud$ php occ user:delete tester
The specified user was deleted

After delete:

MySQL [nextcloud_v13]> select * from oc_storages;
+------------+---------------------------------------------------+-----------+--------------+
| numeric_id | id                                                | available | last_checked |
+------------+---------------------------------------------------+-----------+--------------+
|          1 | object::store:nc0                                 |         1 |         NULL |
|          2 | object::user:admin                                |         1 |         NULL |
|          4 | object::user:775DA7E6-9157-48EE-92AD-DE87D45E22F1 |         1 |         NULL |
|          5 | object::user:tester                               |         1 |         NULL |
+------------+---------------------------------------------------+-----------+--------------+
5 rows in set (0.00 sec)

MySQL [nextcloud_v13]> select count(*) from oc_filecache where storage=5;
+----------+
| count(*) |
+----------+
|       22 |
+----------+
1 row in set (0.00 sec)

Server configuration

Operating system: Linux Debian 9.4
Web server: Nginx 1.10.3
Database: MySQL
PHP version: 7.0
Nextcloud version: (see Nextcloud admin page) 13.0.1.1
Updated from an older Nextcloud/ownCloud or fresh install: either
Where did you install Nextcloud from: tarfile from nextcloud.com
Are you using external storage, if yes which one: Swift as primary
Are you using encryption: no
Are you using an external user-backend, if yes which one: LDAP but same problem with local user

Nextcloud log (data/nextcloud.log)

Nextcloud log

{"reqId":"6RvewTtNcUFNNlCSEPaR","level":3,"time":"2018-05-28T14:35:45+00:00","remoteAddr":"","user":"--","app":"PHP","method":"--","url":"--","message":"Undefined offset: 1 at \/www\/nextcloud\/apps\/admin_audit\/lib\/Actions\/Console.php#32","userAgent":"--","version":"13.0.1.1"}

[vpecinka]

Well, when removing record from oc_storage for the given numeric_id to make stubs in the oc_filecache, and running php occ files:cleanup, I sucessfuly delete oc_filecache entries. However, the file objects remain in the Swift storage...

MySQL [nextcloud_v13]> select *from oc_storages where numeric_id=11986;
+------------+---------------------------------------------------+-----------+--------------+
| numeric_id | id                                                | available | last_checked |
+------------+---------------------------------------------------+-----------+--------------+
|      11986 | object::user:00A4CC06-1CA0-49C5-96AC-45CB77035129 |         1 |         NULL |
+------------+---------------------------------------------------+-----------+--------------+
1 row in set (0.00 sec)

MySQL [nextcloud_v13]> select * from oc_ldap_user_mapping where owncloud_name="00A4CC06-1CA0-49C5-96AC-45CB77035129";
Empty set (0.00 sec)

MySQL [nextcloud_v13]> select * from oc_filecache where storage=11986 limit 5;
+----------+---------+-------------------------------------------------+----------------------------------+----------+-------------------------------+----------+----------+------------+------------+---------------+-----------+------------------+----------------------------------+-------------+----------+
| fileid   | storage | path                                            | path_hash                        | parent   | name                          | mimetype | mimepart | size       | mtime      | storage_mtime | encrypted | unencrypted_size | etag                             | permissions | checksum |
+----------+---------+-------------------------------------------------+----------------------------------+----------+-------------------------------+----------+----------+------------+------------+---------------+-----------+------------------+----------------------------------+-------------+----------+
|  5275256 |   11986 | files/Documents                                 | 0ad78ba05b6961d92f7970b2b3922eca |  5275251 | Documents                     |        2 |        1 |      36227 | 1462869856 |    1462869856 |         0 |                0 | 57319f60813a3                    |          31 | NULL     |
| 17091289 |   11986 | files/Krasty-data/url-to-product-structure.json | 140b3fb419e3481f5ffa1513dd598de7 | 17089329 | url-to-product-structure.json |       56 |        3 |   15146492 | 1496658351 |    1496658351 |         0 |                0 | df0474aa3511ac8bc1ea9571f0a1625e |          27 |          |
| 17089334 |   11986 | files/Krasty-data/products.json                 | 306a7f2e0a4413e0704300ef97addeb8 | 17089329 | products.json                 |       56 |        3 |     281641 | 1496649090 |    1496649090 |         0 |                0 | fb67da264ebc8f43a5f91447d47fcd97 |          27 |          |
| 17089437 |   11986 | files/Krasty-data/get-offers.json               | 454308e8d65561c0b41747219536befe | 17089329 | get-offers.json               |       56 |        3 | 2862520434 | 1496649525 |    1496649525 |         0 |                0 | 172548f93ab1b69c2ec7bfe21fc4504c |          27 |          |
|  5275251 |   11986 | files                                           | 45b963397aa40d4a0063e0d85e4fe7a1 |  5275249 | files                         |        2 |        1 | 2882299436 | 1496658394 |    1496649162 |         0 |                0 | 593531da9bf1c                    |          31 | NULL     |
+----------+---------+-------------------------------------------------+----------------------------------+----------+-------------------------------+----------+----------+------------+------------+---------------+-----------+------------------+----------------------------------+-------------+----------+
5 rows in set (0.00 sec)

MySQL [nextcloud_v13]> delete from oc_storages where numeric_id=11986;
Query OK, 1 row affected (0.02 sec)

then

www-data@nextcloud-web-5864c5968c-zhx4p:/www/nextcloud$ php occ files:cleanup -vvv 
13 orphaned file cache entries deleted

MySQL [nextcloud_v13]> select * from oc_filecache where storage=11986;
Empty set (0.01 sec)

# swift -Axxx -U xxx -K xxx stat nc_19 id:17091289
       Account: AUTH_s_64
     Container: nc_19
        Object: id:17091289
  Content Type: application/octet-stream
Content Length: 15146492
 Last Modified: Mon, 14 May 2018 19:29:54 GMT
          ETag: 14dab0a0ea48c076ceae7438eaa8f266
 Accept-Ranges: bytes
        Server: nginx
    Connection: keep-alive
   X-Timestamp: 1526326193.63821
    X-Trans-Id: txc61f79f4f5ee4a2fbe8d8-005b0e971c

[MorrisJobke]

Set up Nextcloud using Swift as a primary storage

I guess that is missing some parts.

The user and its oc_storage and oc_filecache should be deleted

Usually not.

This is how to set up a Nextcloud instance with object storage:

• add a second config file (only needs to end in .config.php so s3.config.php)
• put in there the normal config for the object storage:

<?php

$CONFIG = [
	'objectstore' =>
		array (
			'class' => 'OC\\Files\\ObjectStore\\S3',
			'arguments' =>
				array (
					'bucket' => 'abc',
					'key' => '123',
					'secret' => 'abc',
					'hostname' => '127.0.0.1',
					'port' => '4567',
					'use_ssl' => false,
					'use_path_style' => true,
				),
		),
];

• then run the installation (either via occ maintenance:install or via the web UI)
• then you should have a working Nextcloud with the object storage as primary storage

I hope this helps to resolve your issues.

[vpecinka]

I`m not sure if we understand each other. The Swift (not S3) object primary storage works fine. Until user deletion. The problem is that Nextcloud leaves object in database and storage as well.

During normal life cycle, when user creates or deletes files, everything is ok - in the database and in the swift storage (=object are created and deleted as well). Only when I delete user (via php occ user:delete), the user disappears, but the object in storage and database not.

I do not understand the code to easily find the bug but I think there is something wrong with deleting user when having object storage.... Because user is removed e.g. from oc_users or oc_preferences but not from oc_storages and oc_filecache tables.

[MorrisJobke]

I`m not sure if we understand each other. The Swift (not S3) object primary storage works fine. Until user deletion. The problem is that Nextcloud leaves object in database and storage as well.

During normal life cycle, when user creates or deletes files, everything is ok - in the database and in the swift storage (=object are created and deleted as well). Only when I delete user (via php occ user:delete), the user disappears, but the object in storage and database not.

I do not understand the code to easily find the bug but I think there is something wrong with deleting user when having object storage.... Because user is removed e.g. from oc_users or oc_preferences but not from oc_storages and oc_filecache tables.

Okay - that one I clearly misunderstood.

@icewind1991 @rullzer Does this ring a bell for you?

[fdoumet]

The bug is in OC\User::delete line 229:

// Delete user files in /data/
if ($homePath !== false) {
   // FIXME: this operates directly on FS, should use View instead...
   // also this is not testable/mockable...
   \OC_Helper::rmdirr($homePath);
}

Looks like files are only actually removed when using local storage(!)

[MorrisJobke]

Looks like files are only actually removed when using local storage(!)

cc @icewind1991 @rullzer @blizzz