Owncloud Server Unsyncs all data from client and then resyncs

[johanbt]

Good Day Sir

Thanks in advance for your support.

I implemented owncloud for one of my customers.

Currently we have the following challenge:

The server unsyncs all data from the clients and the resyns the data back, this causes mayor delays in productivity.

I have seen to scenarios

When the storage unit goes down it removes all data

or when authentication is lost to the smb share for any reason

I have also seen the smb share is up but owncloud re-links to it.

Please find below all the config data.

I really hope you can assist me urgently.
Server configuration

Operating system: Ubuntu 16.04 lts

Web server:

apache2 -v
Server version: Apache/2.4.18 (Ubuntu)
Server built: 2018-06-07T19:43:03

Database:

dpkg -l | grep mariadb
ii mariadb-client-10.0 10.0.36-0ubuntu0.16.04.1 amd64 MariaDB database client binaries
ii mariadb-client-core-10.0 10.0.36-0ubuntu0.16.04.1 amd64 MariaDB database core client binaries
ii mariadb-common 10.0.36-0ubuntu0.16.04.1 all MariaDB common metapackage
ii mariadb-server 10.0.36-0ubuntu0.16.04.1 all MariaDB database server (metapackage depending on the latest version)
ii mariadb-server-10.0 10.0.36-0ubuntu0.16.04.1 amd64 MariaDB database server binaries
ii mariadb-server-core-10.0 10.0.36-0ubuntu0.16.04.1 amd64 MariaDB database core server files

PHP version:

php -v
PHP 7.0.30-0ubuntu0.16.04.1 (cli) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
with Zend OPcache v7.0.30-0ubuntu0.16.04.1, Copyright (c) 1999-2017, by Zend Technologies

php -i | grep -i '^libxml'
libxml Version => 2.9.3
libxml
libXML support => active
libXML Compiled Version => 2.9.3
libXML Loaded Version => 20903
libXML streams => enabled
libxml2 Version => 2.9.3

ownCloud version:

System Status
installed 1
maintenance
needsDbUpgrade
version 10.0.8.5
versionstring 10.0.8
edition Community
productname ownCloud

Storage backend (external storage):

Freenas ixsystem storage solution

accessing Data over smb/cifs share

Also

Add additional drive to owncloud server owncloud accessing drive over smb/cifs share

redis-server
29831:C 30 Aug 10:08:38.185 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
29831:C 30 Aug 10:08:38.185 # Redis version=4.0.10, bits=64, commit=00000000, modified=0, pid=29831, just started
29831:C 30 Aug 10:08:38.185 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
29831:M 30 Aug 10:08:38.185 * Increased maximum number of open files to 10032 (it was originally set to 1024).
29831:M 30 Aug 10:08:38.185 # Creating Server TCP listening socket *:6379: bind: Address already in use

What I have seen is that if the storage unit or any smb share is down or the owncloud server cannot connect to it,it removes the files from the clients desktops, when the storage unit is online or the smb share the it resyncs

Can we config the server that it does not remove the files when the remote share or online storage is off line

Owncloud

Runs in virtual environment vmware 6.5 on hp server

when the share is down and shows 0 kb

the data is removed from the clients

image

[johanbt]

owncloud.zip

[johanbt]

I have uploaded the log file of owncloud for review

[PVince81]

When an external storage like SMB becomes unavailable, the code should throw StorageNotAvailableException and this is propagated to the desktop clients as a 503 HTTP status code. This tells the clients to not delete the contents.

The question here is why this behavior was not observed on your setup.

In what way did your SMB server "go down" ? Was it still possible to connect to it during downtime ? Was it off the network ? (connection refused)

[PVince81]

@cdamken @jvillafanez FYI

[johanbt]

Good Day

1. If the storage nas unit reboots or any form of maintenance need to be done, and the owncloud server is up and the nas is down it removes all data from the desktops. Its shows the smb share at 0kb in the file manager viewer.

2. if the power goes of and the owncloud server is up and running before the nas it does the same

3. Yesterday all units where up and it smb link showed 0kb for 2 min removed all data from the desktops the smb link showed again its correct size and started resyncing the data to the desktops

[johanbt]

4. Even with a smb share to my linux server it does exactly the same think.

[johanbt]

[johanbt]

[johanbt]

[johanbt]

[johanbt]

My original owncloud config all data was in the www data folder
and we did not have any problems, since the move to the smb share we had allot of challenges

[PVince81]

do you have php7-libsmbclient enabled on your server ?

if not you might try with it

[johanbt]

Here is the detailed setup of my owncloud server

apt install -y apache2
apt install -y mariadb-server
apt install -y libapache2-mod-php7.0
apt install -y php7.0-gd
apt install -y php7.0-json
apt install -y php7.0-mysql
apt install -y php7.0-curl
apt install -y php7.0-intl
apt install -y php7.0-mcrypt
apt install -y php-imagick
apt install -y php7.0-zip
apt install -y php7.0-xml
apt install -y php7.0-mbstring
apt install -y php-apcu
apt install -y php-redis
apt install -y redis-server
apt install -y php7.0-ldap
apt install -y php-smbclient

wget https://download.owncloud.org/community/owncloud-10.0.4.tar.bz2
wget https://download.owncloud.org/community/owncloud-10.0.6.tar.bz2
wget https://download.owncloud.org/community/owncloud-10.0.8.tar.bz2

tar -xjf owncloud-10.0.8.tar.bz2
tar -xjf owncloud-10.0.6.tar.bz2

cp -r owncloud /var/www/html

nano /etc/apache2/sites-available/owncloud.conf

Alias /owncloud "/var/www/html/owncloud/"

<Directory /var/www/html/owncloud/>
Options +FollowSymlinks
AllowOverride All

Dav off

SetEnv HOME /var/www/html/owncloud
SetEnv HTTP_HOME /var/www/html/owncloud

ln -s /etc/apache2/sites-available/owncloud.conf /etc/apache2/sites-enabled/owncloud.conf

a2enmod rewrite
a2enmod headers
a2enmod env
a2enmod dir
a2enmod mime

service apache2 restart

chown -R www-data:www-data /var/www/html/owncloud/

sudo mysql --user=root mysql
CREATE USER 'beathan'@'localhost' IDENTIFIED BY '72670306';
GRANT ALL PRIVILEGES ON . TO 'beathan'@'localhost' WITH GRANT OPTION;
FLUSH PRIVILEGES;
exit

nano /etc/apache2/ports.conf
nano /etc/apache2/sites-enabled/000-default.conf

service apache2 restart

nano /etc/apache2/sites-enabled/owncloud.conf

Alias / "/var/www/html/owncloud/"

nano /var/www/html/owncloud/config/config.php

'overwrite.cli.url' => 'http://105.212.61.43:8090',

#Install the Build and Test Dependencies
#In order to get the latest version of Redis, we will be compiling and installing the software from source. Before we download the code, we need to satisfy the build dependencies so that we can compile the software.
#To do this, we can install the build-essential meta-package from the Ubuntu repositories. We will also be downloading the tcl package, which we can use to test our binaries.
#We can update our local apt package cache and install the dependencies by typing:

sudo apt-get update
sudo apt-get install build-essential tcl
#Download, Compile, and Install Redis
#Next, we can begin to build Redis.

#Download and Extract the Source Code
#Since we won't need to keep the source code that we'll compile long term (we can always re-download it), we will build in the /tmp directory. Let's move there now:

cd /tmp
#Now, download the latest stable version of Redis. This is always available at a stable download URL:

curl -O http://download.redis.io/redis-stable.tar.gz
#Unpack the tarball by typing:
tar xzvf redis-stable.tar.gz
#Move into the Redis source directory structure that was just extracted:

cd redis-stable
#Build and Install Redis
#Now, we can compile the Redis binaries by typing:
make
#After the binaries are compiled, run the test suite to make sure everything was built correctly. You can do this by typing:
make test
#This will typically take a few minutes to run. Once it is complete, you can install the binaries onto the system by typing:
sudo make install
#Configure Redis
#Now that Redis is installed, we can begin to configure it.
#To start off, we need to create a configuration directory. We will use the conventional /etc/redis directory, which can be created by typing:
sudo mkdir /etc/redis
#Now, copy over the sample Redis configuration file included in the Redis source archive:
sudo cp /tmp/redis-stable/redis.conf /etc/redis
#Next, we can open the file to adjust a few items in the configuration:
sudo nano /etc/redis/redis.conf
#In the file, find the supervised directive. Currently, this is set to no. Since we are running an operating system that uses the systemd init system, we can change this to systemd:

/etc/redis/redis.conf
. . .

If you run Redis from upstart or systemd, Redis can interact with your

supervision tree. Options:

supervised no - no supervision interaction

supervised upstart - signal upstart by putting Redis into SIGSTOP mode

supervised systemd - signal systemd by writing READY=1 to $NOTIFY_SOCKET

supervised auto - detect upstart or systemd method based on

UPSTART_JOB or NOTIFY_SOCKET environment variables

Note: these supervision methods only signal "process is ready."

They do not enable continuous liveness pings back to your supervisor.

supervised systemd

. . .
#Next, find the dir directory. This option specifies the directory that Redis will use to dump persistent data. We need to pick a location that Redis will have write permission and that isn't viewable by normal users.

#We will use the /var/lib/redis directory for this, which we will create in a moment:

/etc/redis/redis.conf
#. . .

The working directory.

The DB will be written inside this directory, with the filename specified

above using the 'dbfilename' configuration directive.

The Append Only File will also be created inside this directory.

Note that you must specify a directory here, not a file name.

dir /var/lib/redis

#. . .
#Save and close the file when you are finished.

#Create a Redis systemd Unit File
#Next, we can create a systemd unit file so that the init system can manage the Redis process.

#Create and open the /etc/systemd/system/redis.service file to get started:

sudo nano /etc/systemd/system/redis.service
Inside, we can begin the [Unit] section by adding a description and defining a requirement that networking be available before starting this service:

/etc/systemd/system/redis.service
[Unit]
Description=Redis In-Memory Data Store
After=network.target
In the [Service] section, we need to specify the service's behavior. For security purposes, we should not run our service as root. We should use a dedicated user and group, which we will call redis for simplicity. We will create these momentarily.

To start the service, we just need to call the redis-server binary, pointed at our configuration. To stop it, we can use the Redis shutdown command, which can be executed with the redis-cli binary. Also, since we want Redis to recover from failures when possible, we will set the Restart directive to "always":

/etc/systemd/system/redis.service
[Unit]
Description=Redis In-Memory Data Store
After=network.target

[Service]
User=redis
Group=redis
ExecStart=/usr/local/bin/redis-server /etc/redis/redis.conf
ExecStop=/usr/local/bin/redis-cli shutdown
Restart=always
Finally, in the [Install] section, we can define the systemd target that the service should attach to if enabled (configured to start at boot):

/etc/systemd/system/redis.service
[Unit]
Description=Redis In-Memory Data Store
After=network.target

[Service]
User=redis
Group=redis
ExecStart=/usr/local/bin/redis-server /etc/redis/redis.conf
ExecStop=/usr/local/bin/redis-cli shutdown
Restart=always

[Install]
WantedBy=multi-user.target
#Save and close the file when you are finished.

#Create the Redis User, Group and Directories
#Now, we just have to create the user, group, and directory that we referenced in the previous two files.

#Begin by creating the redis user and group. This can be done in a single command by typing:

sudo adduser --system --group --no-create-home redis
#Now, we can create the /var/lib/redis directory by typing:

sudo mkdir /var/lib/redis
#We should give the redis user and group ownership over this directory:

sudo chown redis:redis /var/lib/redis
#Adjust the permissions so that regular users cannot access this location:

sudo chmod 770 /var/lib/redis
#Start and Test Redis
#Now, we are ready to start the Redis server.

#Start the Redis Service
#Start up the systemd service by typing:

sudo systemctl start redis
#Check that the service had no errors by running:

sudo systemctl status redis
#You should see something that looks like this:

Output
? redis.service - Redis Server
Loaded: loaded (/etc/systemd/system/redis.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2016-05-11 14:38:08 EDT; 1min 43s ago
Process: 3115 ExecStop=/usr/local/bin/redis-cli shutdown (code=exited, status=0/SUCCESS)
Main PID: 3124 (redis-server)
Tasks: 3 (limit: 512)
Memory: 864.0K
CPU: 179ms
CGroup: /system.slice/redis.service
+-3124 /usr/local/bin/redis-server 127.0.0.1:6379

. . .
#Test the Redis Instance Functionality
#To test that your service is functioning correctly, connect to the Redis server with the command-line client:

redis-cli
#In the prompt that follows, test connectivity by typing:

ping
You should see:

Output
PONG
Check that you can set keys by typing:

set test "It's working!"
Output
OK
Now, retrieve the value by typing:

get test
You should be able to retrieve the value we stored:

Output
"It's working!"
Exit the Redis prompt to get back to the shell:

exit
As a final test, let's restart the Redis instance:

sudo systemctl restart redis
Now, connect with the client again and confirm that your test value is still available:

redis-cli
get test
The value of your key should still be accessible:

Output
"It's working!"
Back out into the shell again when you are finished:

exit
Enable Redis to Start at Boot
If all of your tests worked, and you would like to start Redis automatically when your server boots, you can enable the systemd service.

To do so, type:

sudo systemctl enable redis
Output
Created symlink from /etc/systemd/system/multi-user.target.wants/redis.service to /etc/systemd/system/redis.service.

Owncloud Redis Configuration
This example config.php configuration uses Redis for the local server cache:

nano /var/www/html/owncloud/config/config.php

'memcache.local' => '\OC\Memcache\Redis',
'redis' => [
'host' => 'localhost',
'port' => 6379,
],
'memcache.locking' => '\OC\Memcache\Redis',

#Setting up Cron Jobs
#Adding a Cron User
www-data
nano /etc/cron.allow
#Adding a Cron Job
crontab -u www-data -e
*/15 * * * * php-cli -f /var/www/html/owncloud/cron.php

#Verify if cornjob is added
crontab -u www-data -l

[johanbt]

I will verify the version

[johanbt]

apt install -y php-smbclient
Reading package lists... Done
Building dependency tree
Reading state information... Done
php-smbclient is already the newest version (0.8.0~rc1-2build1).
The following packages were automatically installed and are no longer required:
linux-headers-4.4.0-128 linux-headers-4.4.0-128-generic
linux-headers-4.4.0-131 linux-headers-4.4.0-131-generic
linux-headers-4.4.0-87 linux-headers-4.4.0-87-generic
linux-image-4.4.0-128-generic linux-image-4.4.0-131-generic
linux-image-4.4.0-87-generic linux-image-extra-4.4.0-128-generic
linux-image-extra-4.4.0-131-generic linux-image-extra-4.4.0-87-generic
Use 'apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

[johanbt]

php --version
PHP 7.0.30-0ubuntu0.16.04.1 (cli) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
with Zend OPcache v7.0.30-0ubuntu0.16.04.1, Copyright (c) 1999-2017, by Zend Technologies

[jvillafanez]

You might want to try #32416 patch and check if the situation improves. It's expected to come with OC 10.0.10

Nevertheless, there are other errors you might want to investigate, specially the ones like

"message":"fread(): \\Icewind\\SMB\\NativeStream::stream_read - read -8193 bytes more data than requested (-1 read, 8192 max) - excess data will be lost at \/var\/www\/html\/owncloud\/apps\/files_external\/3rdparty\/icewind\/streams\/src\/Wrapper.php#83"

You might also want to update to 10.0.9 because it includes an update to the smb library that ownCloud is using. It might help with the above.

[johanbt]

Thanks for the update
Please send me the detailed process to upgrade to 10.0.9

[johanbt]

How do i apply the patch above 32416

[johanbt]

Thanks jvillafanez and PVince81 for your support

[jvillafanez]

https://doc.owncloud.org/server/10.0/admin_manual/maintenance/upgrade.html should contain enough information to upgrade your server

[johanbt]

Thanks and the Patch how do I apply that

[jvillafanez]

Patch code can be found in https://patch-diff.githubusercontent.com/raw/owncloud/core/pull/32416.patch . You can download it and apply with the "patch" utility. In any case, before attempting to patch something, know what you're doing as it can break the system, so have a backup ready in case something goes wrong.

[johanbt]

Thanks jvillafanez

[johanbt]

@jvillafanez Just to verify I must to the upgrade first and then apply the patch

[jvillafanez]

Yes, otherwise the patch will likely be removed after the upgrade since it isn't available in 10.0.9.
In any case, verify that both the upgrade and the patch have been run without any errors.

[johanbt]

Thanks @jvillafanez
I am writing the upgrade process in detail step by step to test first
My owncloud prod environment runs on vmware esxi 6.5 on a hp server
So I can clone or snapshot the server test the upgrade and roleback if needed

[johanbt]

@jvillafanez
How does the patch utility work?

[patrickjahns]

Related to:

"message":"fread(): \\Icewind\\SMB\\NativeStream::stream_read - read -8193 bytes more data than requested (-1 read, 8192 max) - excess data will be lost at \/var\/www\/html\/owncloud\/apps\/files_external\/3rdparty\/icewind\/streams\/src\/Wrapper.php#83"

You should have a look into using SMBv2 ( or higher) as protocol. This might also require, that you use a more recent samba library (4.7/4.8)

[johanbt]

@patrickjahns
Hi I verified my samba version
smbstatus --version
Version 4.3.11-Ubuntu

https://www.samba.org/samba/history/

14 August 2018
Samba 4.8.4, 4.7.9 and 4.6.16 Security Releases Available for Download

These are security releases in order to address
CVE-2018-10858 (Insufficient input validation on client directory listing in libsmbclient.),
CVE-2018-10918 (Denial of Service Attack on AD DC DRSUAPI server.),
CVE-2018-10919 (Confidential attribute disclosure from the AD LDAP server.),
CVE-2018-1139 (Weak authentication protocol allowed.) and
CVE-2018-1140 (Denial of Service Attack on DNS and LDAP server.).

I will upgrade to latest stable release

[johanbt]

@jvillafanez
Please see below the upgrade process I want to test, any remarks
Manual ownCloud Upgrade
Backup Your Existing Installation
Backup the following items:
Step 1 The ownCloud server data directory
cp -rv /var/www/html/owncloud /opt/backup/owncloud
Step 2 The ownCloud server database
mysqldump > /opt/backup/database/beathan.sql
Step 3 The config.php file
cp -rv /var/www/html/owncloud/config/config.php /opt/backup/config
Step 4 All 3rd party apps
none installed
Step 5 Enable Maintenance Mode Using The occ command
sudo -u www-data php occ maintenance:mode --on
Step 6 Disable Cron jobs
sudo service cron stop
step 7 Stop the Webserver
sudo service apache2 stop
Step 8 Download the latest owncloud version
cd tmp
wget https://download.owncloud.org/community/owncloud-10.0.9.tar.bz2
Step 9 Extract the .tar.bz2 archive
tar -jxf owncloud-10.0.9.tar.bz2 -C
step 10 Remove Current Owncloud Directory
rm /var/www/html/owncloud
Step 11 Move Extracted copy to correct owncloud path
mv /tmp/owncloud /var/www/html/
Step 12 Copy the config.php from the backup
cp -rv /opt/backup/config.php to /var/www/html/owncloud/config/
Step 13 Copy back owncloud data folder from backup
cp -rv /opt/backup/owncloud/data /var/www/html/owncloud/data
Step 14 Disable Core Apps
sudo -u www-data php occ app:disable activity
sudo -u www-data php occ app:disable files_pdfviewer
Step 15 Start the Webserver
sudo service apache2 start
Step 16 Upgrade ownCloud
sudo -u www-data php occ upgrade
Step 17 Disable Maintenance Mode Using The occ command
sudo -u www-data php occ maintenance:mode --off
Step 18 Restart the Webserver
sudo service apache2 restart

[jvillafanez]

https://linux.die.net/man/1/patch

patch -p1 < "patchfile"

If you're using a VM I think it's easier to take a snapshot of the VM and use it as backup instead of making the backups of the DB and ownCloud separately.
In addition, taking advantage of the VM, you might want to remove the owncloud installation except for the ownCloud's data and config directory and copy the new ownCloud's files there. There is no need to copy the config and data folders out and in the ownCloud's folder. As long as the "old" data and config directories are there, there shouldn't be problems

[johanbt]

@jvillafanez

I will do that.

So i do not need to run the upgrade command as well?

[johanbt]

[johanbt]

I delete everything accept data and config folder

[johanbt]

Please review my revised process
Manual ownCloud Upgrade
Step 1 Take VM snapshot
Step 2 Enable Maintenance Mode Using The occ command
sudo -u www-data php occ maintenance:mode --on
Step 3 Disable Cron jobs
sudo service cron stop
step 4 Stop the Webserver
sudo service apache2 stop
Step 5 Download the latest owncloud version
cd tmp
wget https://download.owncloud.org/community/owncloud-10.0.9.tar.bz2
Step 6 Extract the .tar.bz2 archive
tar -jxf owncloud-10.0.9.tar.bz2 -C
Step 7 Remove all files and folder from /var/www/html/owncloud accept data and config folders
Step 8 Move Extracted copy to correct owncloud path
mv /tmp/owncloud /var/www/html/
Step 9 Disable Core Apps
sudo -u www-data php occ app:disable activity
sudo -u www-data php occ app:disable files_pdfviewer
Step 10 Start the Webserver
sudo service apache2 start
Step 11 Disable Maintenance Mode Using The occ command
sudo -u www-data php occ maintenance:mode --off
Step 12 Restart the Webserver
sudo service apache2 restart

[jvillafanez]

You have to run the upgrade command at step 9-10 in order to run the upgrade routines. It's recommended to have the webserver disabled (maybe just the ownCloud site, not the whole webserver) in order to prevent unwanted interactions with ownCloud during the upgrade.

[johanbt]

I gather we need the webservice running for the upgrade to work

[johanbt]

Manual ownCloud Upgrade of Existing Owncloud Vmware Virtual Server
Step 1 Take VM snapshot
Step 2 Enable Maintenance Mode Using The occ command
sudo -u www-data php occ maintenance:mode --on
Step 3 Disable Cron jobs
sudo service cron stop
step 4 Stop the Webserver
sudo service apache2 stop
Step 5 Download the latest owncloud version
cd tmp
wget https://download.owncloud.org/community/owncloud-10.0.9.tar.bz2
Step 6 Extract the .tar.bz2 archive
tar -jxf owncloud-10.0.9.tar.bz2 -C
Step 7 Remove all files and folders from /var/www/html/owncloud accept data and config folders
Step 8 Move Extracted copy to correct owncloud path
mv /tmp/owncloud /var/www/html/
Step 9 Disable Core Apps
sudo -u www-data php occ app:disable activity
sudo -u www-data php occ app:disable files_pdfviewer
Step 10 Upgrade ownCloud
sudo -u www-data php occ upgrade
Step 11 Start the Webserver
sudo service apache2 start
Step 12 Disable Maintenance Mode Using The occ command
sudo -u www-data php occ maintenance:mode --off
Step 13 Restart the Webserver
sudo service apache2 restart

[johanbt]

Do you want me to start the webservice before the upgrade

[johanbt]

@jvillafanez the patch process
how to the patch utility know for which application the patch must be applied for.
can you please talk me through the process

[johanbt]

@jvillafanez
This system is a prod system and I want to apply the upgrade an patch as quick as possible

[PVince81]

@johanbt please note that this is a bug tracker, not a support portal.

See http://owncloud.org/support

[johanbt]

@PVince81
Thanks, i just wanted to know how to apply your bug patch.
I will figure it out
Thanks for the support

[davitol]

Seems like I had the same problem:

1 - As admin user I configured a general WND mountpoint (Win10 as backend located in a Virtual Machine)
2 - I synced the files and the mountpoint using Desktop Client Version 2.4.3. Files were downloaded to my local.
3- I shutdown the VM that contains the WND backend
4 - oC Desktop Client tries to reach to the WND backend, and it deletes the files already saved in local.

[jvillafanez]

Works for me.

Folders are kept in the desktop.

Checked with ownCloud 10.0.10RC2, linux desktop client 2.4.3 (build 10035).
ubuntu 16.04, PHP 7.0.22-0ubuntu0.16.04.1

[johanbt]

Good Day Owncloud Gurus

Thanks in Advance for your support.

After the upgrade my customers data still re-syncs, what I did then is to setup a lab and mirrored the complete setup of the customer site, as a technologist I always want to know way something happens what is the root cause.

The customer setup:

Linux ubuntu server 16.04lts
latest version of owncloud 10.0.9
Redis also installed
mariadb database

Then two devices that store data:

1. File server with smb share (also Ubuntu 16.04 Lts)
2. Freenas storage device smb share

From both devices the data will re-sync.

This will happen during server or nas maintenance or when the network goes down or power loss.

In owncloud you link to the external storage and then you can share the data in two ways

1. Directly from the storage link you configure you can share the data to group or user
   

With this sharing method the data shared to groups or users, the data that have synced stays on the clients till the storage is up again, if its a large file and its in the process of syncing it will be removed and re-synced when the storage its up again. So the users can work and when access to the storage is online all is fine.

The changes the users made syncs up and the remainder of the data that did not sync down syncs down.

2. Sharing the data from storage to groups and users in the webconcole, shares the data and it syncs to the groups or user. If connectivity is lost for any reason from the owncloud server to storage it removes this data from all the clients within seconds and then re-syncs the data back when connectivity is resorted to the storage

Here you can see the data is syncing

If I shutdown the file server all data will be removed from the agents

All data shared from the storage removed

If i turn on the storage it re-syncs

If it can work in the manner of the data that is already synced stays on the clients and then syncs all changes when the storage is up again it will be excellent.

Thanks in Advance for your support with this matter.

Best Regards

Johan

[jvillafanez]

Directly from the storage link you configure you can share the data to group or user

Just a minor correction: you're not sharing anything there. You're allowing access to the files in that storage to the members of that group (in that particular case)

Anyway, this is what I tested, and I think it's working fine

Sharing the data from storage to groups and users in the webconcole, shares the data and it syncs to the groups or user

This is a different scenario. I haven't tested this one, so there might be problems. Maybe @davitol can help to reproduce the problem.

I can't say for sure, but it looks really problematic. Taking a guess, the shared folder might disappear because of the error thrown by the underlying storage. The problem might be in the sharing, as it might be not properly handling this error case.

As a workaround for now, I'd recommend to avoid using sharing over external storages if you can ensure a stable connection to the storage (to prevent this connection problem). You can either set a proper external storage connection for all the users, or let the user connect via SMB to other servers. If people want to share files, they can use the external storage for that, although it won't really be private.

[johanbt]

Hi Juan

Thanks for the reply.

Sorry I agree its giving access will amend my post

I tested links to 3 types of storage and they all give the same problem removing the data.

I found an intrem solution so that the data will not be removed its not a long term solution but at least the customer can work.

If you give access to the storage to the full resolved path of the sub-folder that you want to share and you add a group or user and an then loose connectivity to the storage the data is then not removed.

You are going to sit with multiple storage configs towards the same storage platform but at least the data will remain on the clients and users can work will the storage are being brought online again.

I cannot move the data into the owncloud server just because of the vast size of the data. Storage can be offline for maintenance adding volumes, network switches in maintenance on Sundays or power loss due to power supply maintenance and ups upgrades.

Please see my config below:

Proof of the sync state before i shutdown the storage all data not removed

After shutting down the storage the data still on client pc

I hope it helps.

I also hope you can change the owncloud code to amend this challenge, I thank You in advance for your support.