Mounted ftps doesn't show all the files and directories

[cdamken]

Expected behaviour

Mounting a ftps file system. put and erase files/directories and can be used with ownCloud

Actual behaviour

Mounting a ftps file system. put and erase files and ownCloud do not refresh the files or directories

Steps to reproduce

1. Mounting an external ftps, can be seen in ownCloud
2. Actual files and directories are shown
3. Put new files and/or directories in the ftps with an external application
4. Files and directories are not shown
   1. Nothing gets better if I execute 'php console.php files:scan --all' (It doesn't refresh)

Server configuration

ownCloud version: 6.0.4

00001186

[PVince81]

With FTPS you mean FTP + secure checkbox ?

There's trouble with some FTP servers that they don't return any mtime, so update detection cannot work.

Related: #6395

@icewind1991 maybe we need to put return false; in the hasUpdated as well ?

[cdamken]

@PVince81 yes: ftp with secure checkbox. The FTP server is ProFTPD

[MTRichards]

Have you set up a cron job (admin page > set background jobs to cron > configure an OS cron job to execute the cron).
Probably the issue is that ownCloud doesn't know that files have been added to the FTP server, since the FTP server doesn't tell this to ownCloud. Therefore, you need to set the cron job to occasionally scan for updates.

[karlitschek]

http://doc.owncloud.org/server/7.0/admin_manual/configuration/background_jobs.html documentation here

[cdamken]

@MTRichards what should the cron job says?, As I wrote before: php console.php files:scan --all' (It doesn't refresh)
Even unmouning and remounting does not help.

[MTRichards]

@karlitschek @DeepDiver1975 Any help on the specific syntax of the cron job? @cdamken has an example above, but it doesn't seem to work.

[DeepDiver1975]

@cdamken are you executing the console call with the web server user?

[cdamken]

@DeepDiver1975 @MTRichards

I just mounted a local directory with my ftps server (using curlftpfs) and added to OwnCloud as an external storage. I can see that ownCloud can access to the files in the FTP server, and it refreshes the content although the files have been uploaded through an FTP client. There are some problems to create files trough the ownCloud client in that directory, but it seems it's a problem with curlftpfs (the files cannot be created in the file system either)

When I use "Local" as an external storage, I see that if I delete the storage, the oc_filecache table keeps the old entries, but once I create again the local external storage, this table get refreshed with the new files (I deleted directly some files in the meantime). It doesn't happen if I create an external storage as FTP, since the files in the oc_filecache table that belong to this external storage don' get
refreshed (after recreating the storage, I can see some files that don't exist anymore) Nothing gets better if I execute 'php console.php files:scan --all'

You can see attached tree files: the ftp external storage definition (ac.png, where the indicator is green), the files not refreshed that I can see through the web interface (edw.png) and the files that actually exist in the ftp server (Ftp_folder.png). In order to reproduce it, you just have to create the FTPS external storage and delete some FTP files through the ftp client, not the owncloud client.

Attached in: S3-ownCloud\Shared\owncloud\support\github-issues\core\9630

[PVince81]

@icewind1991 please help. Thanks.

[cdamken]

@DeepDiver1975 From the web server, as administrator

[icewind1991]

It might be caused by the filecache entries not having valid hashes of the path, which makes it impossible for the scanner to remove the files. Can you try running

SELECT * from oc_filecache WHERE path_hash != md5(path);

(this is mysql specific, you probably need to replace the oc_ prefix)
(see also #10705)

You can also try re-running the scanner and checking the output it generates, if the output of the scanner lists the removed files it means it's a problem with the storage backend

[cdamken]

@icewind1991 I saved the results here: S3-ownCloud\Shared\owncloud\support\github-issues\core\9630\New\

[cdamken]

@icewind1991 Any news?

[cdamken]

@butonic @PVince81 @icewind1991 http://www.proftpd.org/docs/howto/Timestamps.html

Question: Does ProFTPD support changing the timestamp on a file using the MDTM command? That's what my client does.
Answer: No.

Why not? For several reasons:

The MDTM command is not one of the commands mandated by RFC959.
Some FTP clients use MDTM for retrieving the modification time, others try to use it to set the modification time, and still others do both.
Among those FTP clients that do use MDTM, there is no consistent format of the timestamp used.

In short, it is a royal mess. ProFTPD supports MDTM for retrieving the modification time, in GMT. Period.
So it seems that you can get the mtime date for a file using the MDTM command. I don't know if the mechanism used by OwnCloud involves using this command or some other. I tried myself sending the MDTM command against my Proftpd server and got a timestamp returned. I guess that Owncloud doesn't need to set the mtime itself, because probably we would need to recompile the FTP server with additional modules.

[PVince81]

ownCloud is using PHP's FTP stream wrapper. So if that one isn't using that command then it won't work.
At some point I suggested rewriting the FTP external storage to use the real client API: #6468

[ghost]

done

[PVince81]

please test with the files_external_ftp app from the marketplace and see if the problem persists

[cdamken]

I will test it and reopen (or create a new issue) if the problem still appears.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.