Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subfolders of moved folders not synced #1000

Closed
eboth opened this issue Jan 6, 2019 · 41 comments

Comments

@eboth
Copy link

commented Jan 6, 2019

Expected behaviour

When on your local Windows system, you move a folder into your Nextcloud folder within the same NTFS partition (did't test other file systems), the Nextcloud client should recursively synchronize all directory depths, i.e. all files and folders in all nested subfolders.

Actual behaviour

Moved folders are only synchronized up to the first level.
This is the directory test as copied into the Nextcloud folder:

+-test/
  |
  +-level1/
  | |
  | +-level2/
  | |
  | +-level2.txt
  |
  +-level1.txt

This is the directory structure as copied by the Nextcloud client to the server:

+-test/
  |
  +-level1/
  |
  +-level1.txt

Steps to reproduce

  1. Create a directory test with at least one subfolder level1 outside your Nextcloud folder hierarchy, but on the same partition as your Nextcloud folder.
  2. Place content (files/folders) in level1.
  3. Move (not copy) test into your Nextcloud folder (Windows default C:\Users\username\Nextcloud).
  4. Wait for Nextcloud client to finish synchronization.
  5. Check contents of test on the server; confirm that the subfolder level1 is empty on the server.

Client configuration

2.5.1final (build 20181204)

Operating system:
Windows 7 SP 1 (6.1)

File system:
NTFS

Installation path of client:
c:\Program Files (x86)\Nextcloud\

Server configuration

13.0.8.2

Logs

  1. Client logfile:
    https://gist.github.com/eboth/49d061cd66724e3e9dc699cfc4d1591f

While the log file was captured, the folder test_copied (with two levels of subdirectories as shown in the directory listing above) was copied to the Nextcloud folder, and the folter test_moved (same contents) was moved to the Nextcloud folder. The log seems to show that in the copied folder, all nodes were recognized correctly, while in the moved folder, the Nextcloud client never became aware of the nodes below level1.

@eboth eboth changed the title Subfolders of folders moved into Nextcloud folder not synced Subfolders of moved folders not synced Jan 6, 2019

@packi

This comment has been minimized.

Copy link

commented Jan 11, 2019

I can confirm that on Linux, script to create a test folder structure can be found here: https://gist.github.com/packi/1361912055bd03490bbb6a8906d9800e

I've initially created a bug for that on the owncloud client repos with more details as I didn't realize there was a separate repos for Nextcloud. owncloud/client#6949

@abysso2

This comment has been minimized.

Copy link

commented Feb 8, 2019

Same here with linux client 2.5.1 ans server 15.0.4 ...

@dude2k5

This comment has been minimized.

Copy link

commented Feb 9, 2019

Same here, have the same issue. Main folder syncs, but subfolder did not.

@estepix

This comment has been minimized.

Copy link

commented Feb 15, 2019

Same problem here:

Client

  • Version: 2.5.1final (build 20181204)
  • OS: Ubuntu 16.04.5 LTS xenial
  • File system: ext4

Server

  • Versions: 14 and 15.0.4
  • OS: Ubuntu 16.04.5 LTS
  • Web Server: Nginx 1.15.8-8.6.0.1

Using PPA:

Packages version:

ii  libnextcloudsync0                                    2.5.1-20181204.111806~xenial1                                               amd64        Nextcloud sync library
ii  nextcloud-client                                     2.5.1-20181204.111806~xenial1                                               amd64        Nextcloud desktop sync client
ii  nextcloud-client-l10n                                2.5.1-20181204.111806~xenial1                                               all          Nextcloud client internatialization files

Log shows that _csync_detect_update only detects first level:

[_csync_detect_update 	Checking for rename based on inode # 11010575
[_csync_detect_update 	file: nextcloud/myclient/test, instruction: INSTRUCTION_NEW <<=
[_csync_detect_update 	Checking for rename based on inode # 11010705
[_csync_detect_update 	file: nextcloud/myclient/test/level1, instruction: INSTRUCTION_NEW <<=
@hkiang01

This comment has been minimized.

Copy link

commented Feb 17, 2019

workaround: quit and restart the desktop client. Tested with the nextcloud-client Ubuntu ppa on Ubuntu 18.10.

@estepix

This comment has been minimized.

Copy link

commented Feb 18, 2019

I have read about that "workdaround" before and also tested and yes, it works, however restarting the client every time we add a few folders to nextcloud does not seem viable on a daily work basis

@eboth

This comment has been minimized.

Copy link
Author

commented Feb 18, 2019

@hkiang01, thanks! Confirmed on Windows. This makes the bug a lot less lethal as you can hope that sooner or later the files will end up on the server.

@SonRiab

This comment has been minimized.

Copy link

commented Feb 25, 2019

Possible duplicate #1083?

@bjoernv

This comment has been minimized.

Copy link

commented Feb 27, 2019

I can not reproduce this. All files from file structure #1000 (comment) are synchronized.

Client

* Version: 2.5.1

* OS: openSUSE Tumbleweed 20190225

* File system: ext4

Server

* Version: 14.0.7

* OS: Ubuntu 14.0.4 LTS

* Web Server: Apache 2.4.7

* PHP 7.2
@aaronpk

This comment has been minimized.

Copy link

commented Mar 3, 2019

Same problem here on MacOS. I was trying to migrate files from Dropbox and noticed that any time I dragged a folder with lots of subfolders over, none of the subfolders had any contents synced.

Confirmed that quitting and re-launching the desktop client causes a complete sync.

@dnv

This comment has been minimized.

Copy link

commented Mar 16, 2019

I can confirm this happening to me on Windows 10 with Nextcloud sync client 2.5.1final (build 20181204). Let's just say it made me quite vary of using Nextcloud to begin with.

@driph

This comment has been minimized.

Copy link

commented Mar 17, 2019

Confirming the same issue, 2.5.1final (build 20181204) on macOS 10.13.6.
Restarted the client resolved it for the bugged sync, but any further additions of nested directories after the sync on start result in the same behavior and workaround.

@wpoet

This comment has been minimized.

Copy link

commented Mar 23, 2019

Confirming the inability of the client to sync all subfolders and files: Ubuntu 18.10, Version 2.5.1final (build 20181204). This is really a killerbug. Hopefully this will be fixed soon. We are trying to introduce Nextcloud to students of our school, but this is makes nextcloud a real no go.

It is indeed a showstopper, and shouldn't really be so difficult to fix. There are two additional workarounds (in addition to restarting the desktop client).

One is to add a file to the empty directory on the lowest level. E.g., if you do mkdir -p dir1/dir2/dir3/dir3 on the nextcloud client root directory, dir3 and dir4 will not be synced to the server, leaving dir2 without a subdirectory. If you do touch dir1/dir2/dir3/dir4/somefile, then all dirs will be synced, and somefile will exist on the server.

The second workaround is to use the command line client and do
nextcloudcmd -s -u <nextcloud user> -p <nextcloud password> <nextcloud rootdir> <nextcloud url>

This does a complete re-sync.

I hope this helps anyone who suffers under the deficiencies of the desktop client.

@abysso2

This comment has been minimized.

Copy link

commented Apr 8, 2019

I just upgraded to the linux-client 2.5.2, still the same issue. I can not believe it. The server ist such an excellent piece of software, but the client really can not be trusted. That is a real showstopper - why does the owncloud client work as expected?

@adrianriedo

This comment has been minimized.

Copy link

commented Apr 18, 2019

Spent a full day looking on Windows 10 machine what I made wrong (because I did not see expected files on the Server) and finally found this issue.
I can confirm the problem (moving folder structure, sync not done 100%).
This is a showstopper to move all our data from Seafile (which has a perfect working client) to Nextcloud.
The Server Side of Nextcloud is quite nice - but the very most fundamental thing (sync files with the Desktop Client) can not be trusted all the times...
I hope someone who is experienced with the Desktop Client can correct this.

@dothebakedpotato

This comment has been minimized.

Copy link

commented Apr 18, 2019

Also have this issue (with Linux desktops/laptops). Can occur when copying directories as well as moving.

Unfortunately Nextcloud is not fit for enterprise or business without a working and robust client. It makes me extremely nervous about the prospect of losing critical data.

@ziggr

This comment has been minimized.

Copy link

commented Apr 22, 2019

I just got bit by this bug, too.

$ nextcloud --version
Nextcloud version 2.5.2git
Using Qt 5.5.1, built against Qt 5.5.1
Using 'OpenSSL 1.0.2g  1 Mar 2016'

$ uname -a
Linux gemini 4.15.0-47-generic #50~16.04.1-Ubuntu SMP Fri Mar 15 16:06:21 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

$ mount | grep nvme
/dev/nvme0n1p2 on / type ext4 (rw,noatime,discard,errors=remount-ro,data=ordered)

A whole .git repo and its subdirectories, along with the contents of a lot of subdirectories of other files, all missing from server. Upon restarting the desktop client, the files synced to server (phew!).

This is really heartbreaking after spending a whole weekend creating this server and getting everything to work. How can I use Nextcloud to sync my source code if my projects all have subdirectories (and git repos!).

Edit: I triggered the bug by just moving a large nested directory structure from one location to ~/Nextcloud/ . Creating or touching individual files in nested subdirectories seems to sync just fine. It's just large file copies/moves that escape Nextcloud's gaze:

$ mkdir -p a/b/c
$ touch a/b/c/empty.txt
$ mv a ~/Nextcloud

# ~/Nextcloud/a/ exists on server
# ~/Nextcloud/a/b/ exists on server
# ~/Nextcloud/a/b/c/ does not exist on server
# ~/Nextcloud/a/b/c/empty.txt does not exist on server
@adrianriedo

This comment has been minimized.

Copy link

commented Apr 22, 2019

I really hope someone who codes for the desktop client reads this and will improve it.
Fact is that we used Owncloud some years ago and had very similar issues.
Then we used Seafile for the last 2-3 years and it worked fine with its desktop client.
Nextcloud has quite a momentum right now, so we thought to give it a try again - unfortunately the desktop sync is still a problem as it seems - I can not tell all out users to close and restart the client every day to make shure the files are synced...

Nextcloud has some nice features on the Server - but when it comes to syncing with desktop client it seems to have real issues.
Can it really be that so many users use it without even noticing that files are missing on the server?
Strange

@Kambrian

This comment has been minimized.

Copy link

commented May 5, 2019

I cannot believe such a HUGE MAJOR bug exists with this software and not get fixed (or even noticed?) for such a long time till today.

I am regretting the efforts I spent trying to set up my personal cloud with nextcloud. This should not be anything from a commercial company. Just hard to believe.

@Germano0

This comment has been minimized.

@adrianriedo

This comment has been minimized.

Copy link

commented May 5, 2019

Worked with Nextcloud now for 2 weeks after some years of Seafile.
Now after this (and some other bugs in the Desktop Client that we found) we had to move back to Seafile again. The Desktop Client has also problems with multiple Sync Directories when you use Standby on your Computer - after wakeup syncing does not start on all Sync Folder (if you have more than a hand full of them).
The Web UI is nice and fancy - but syncing with Desktop is sadly a risk.
I would have loved to move from Seafile to Nextcloud - but not at this stage...

@jospoortvliet

This comment has been minimized.

Copy link
Member

commented May 6, 2019

I can reproduce this. And yes, restarting the client finds and syncs the files, so I don't think this leads to data loss - you just might not have files synced as quickly as you'd expected in the rare case that you move a complex and deep set of folders into a synced folder. Bad enough and the problem should be found and fixed, but not a disaster, I think. You don't do this action often, most users do restart their computers once a week if not more often, and you rarely need all those files available everywhere right away.

@bill-mcgonigle

This comment has been minimized.

Copy link

commented May 6, 2019

most users do restart their computers once a week if not more often

This might be true on Windows, but I have hard enough time getting linux users to reboot for kernel security updates - uptimes over 90 days are quite typical. It's not a safe assumption for a design guideline.

@dnv

This comment has been minimized.

Copy link

commented May 6, 2019

@kleinph

This comment has been minimized.

Copy link

commented May 6, 2019

I agree with @dnv, the handling of this bug looks unprofessional. There are two issues open (this and #876) since at least end of November and there seems to be no response from anyone of Nextcloud.

@dothebakedpotato

This comment has been minimized.

Copy link

commented May 6, 2019

Assuming that users will restart their computers is a bad excuse, and not reasonable in many environments. I have workstation uptimes on the order of months.

I need users' files to be synced reliably onto ZFS file servers. The ZFS pools are then backed up conventionally. This is a bug that is occurring regularly, many times a week. If I can't rely on the sync client then data loss is a very real and worrying possibility.

The way this has been handled by Nextcloud is faintly ridiculous. It is not showing itself to be a serious or competent company.

@jospoortvliet

This comment has been minimized.

Copy link
Member

commented May 6, 2019

@dothebakedpotato your company is not a customer of us, are you? So we have no SLA or any other contract in place with your company?

I believe that serious companies run software with a SLA from a vendor. Not doing that is indeed risking user data, and if my IT team would do that without telling me, I would not be happy.

Please note that github is for contributors to coordinate their work. We handle bugs from home users and plan our work. If your company uses Nextcloud and you look or ask for help here, you're in the wrong place. Customers can contact our team through https://portal.nextcloud.com and there they get prompt help.

(Edit: I updated the comment a few times to clarify that I'm clearly talking to a paid employee from another company who expects free support services. We're very happy to help home users with Nextcloud and we want to fix this bug, but having employees from other companies using their paid time to tell us we're not working fast enough for them for free is a little disappointing.)

@dnv

This comment has been minimized.

Copy link

commented May 6, 2019

@dothebakedpotato your company is not a customer of us, are you? So we have no SLA or any other contract in place with your company?

I believe that serious companies run software with a SLA from a vendor. Not doing that is indeed risking user data, and if my IT team would do that without telling me, I would not be happy.

Public relations tip: you are not making how this looks for Nextcloud any better with this post.

Have you provided a fix for this issue to those who actually ARE paying you?

@jospoortvliet

This comment has been minimized.

Copy link
Member

commented May 6, 2019

@dnv that might very well be, I haven't checked. If a fix was provided to our customers, it will end up in the publicly released version soon. Your company can then update once that is done, or become a customer so they can help pay for the work we do.

I will lock this now for contributors, this discussion is not helping anyone. If you suffer from this as home user - please, restart your client regularly until we or any volunteer has had time to fix it. I'm sorry for this issue, we're working as much as we can.

If this bites you as enterprise user, please contact our sales team.

@nextcloud nextcloud locked as too heated and limited conversation to collaborators May 6, 2019

@jancborchardt

This comment has been minimized.

Copy link
Member

commented May 6, 2019

I’d like to remind everyone that we have a Code of Conduct.

  • 🔥 Aggressive behavior does not make anyone more likely to fix anything. It makes it less likely.
  • 🤖 The people working on this project are, in fact, real people. Be friendly.
  • 💸 Rather than wasting time writing an inflammatory comment, donate to fix this issue on Bountysource.
  • ❤️ Remember that the software is fully free & open source. Treat the people working on it with respect.

Also thanks to everyone who has been constructive in this issue.

@jancborchardt

This comment has been minimized.

Copy link
Member

commented May 6, 2019

Also, you might have seen that we are looking for a C++/Qt developer to join our desktop client team. Please spread the word: https://twitter.com/camiladotcodes/status/1115523047031808001 – that’s another thing you can do to actually help with this issue.

❤️

@nextcloud nextcloud unlocked this conversation May 9, 2019

@camilasan

This comment has been minimized.

Copy link
Member

commented May 9, 2019

I am sorry for the misunderstanding - but please be assured that we are working on this issue. You might not get a reply but it doesn't mean I am not aware of the issue. I am actively looking to hire an additional person to join me on the desktop client. Please be patient and understand that we are very busy and doing everything we can to make Nextcloud work perfectly. Comments like the one we replied to can be hurtful when you're doing your best.

joshua-sterner added a commit to joshua-sterner/desktop that referenced this issue May 9, 2019

Fixed Issue nextcloud#1000 - Subfolders of moved folders not synced
Signed-off-by: Joshua Sterner <joshua.s.sterner@gmail.com>
@jancborchardt

This comment has been minimized.

Copy link
Member

commented May 9, 2019

And huge thanks to @joshua-sterner for contributing a fix! Pull request is open at #1245 – everyone please help testing and review! 😊

@camilasan

This comment has been minimized.

Copy link
Member

commented May 9, 2019

The PR #1245 works for me. Could you all run https://transfer.sh/X46OK/Nextcloud-PR-1245-bd9fcd88d85f48889e5e6fd55456cec7b123a22c-x86_64.AppImage to verify that too? Thanks.

@camilasan camilasan added this to the 2.5.3 🐛 Bug fixes milestone May 9, 2019

@dothebakedpotato

This comment has been minimized.

Copy link

commented May 9, 2019

@dothebakedpotato your company is not a customer of us, are you? So we have no SLA or any other contract in place with your company?

I believe that serious companies run software with a SLA from a vendor. Not doing that is indeed risking user data, and if my IT team would do that without telling me, I would not be happy.

Please note that github is for contributors to coordinate their work. We handle bugs from home users and plan our work. If your company uses Nextcloud and you look or ask for help here, you're in the wrong place. Customers can contact our team through https://portal.nextcloud.com and there they get prompt help.

(Edit: I updated the comment a few times to clarify that I'm clearly talking to a paid employee from another company who expects free support services. We're very happy to help home users with Nextcloud and we want to fix this bug, but having employees from other companies using their paid time to tell us we're not working fast enough for them for free is a little disappointing.)

You're making an awful lot of assumptions. Let me explain the situation.

We're a mid enterprise of about 300 users. We have worked hard over a long time to build our business to where we are today and we are careful about how we spend our money. I have been looking for a "cloud" technology to replace our NFS/Kerberos/Samba file shares. Nextcloud looked most promising and so, since you release your product under the AGPL, I decided to try it out. I have been running a trial with a very small number of users to see whether it is a viable option. If it was suitable, of course we would buy an SLA.

Unfortunately for both of us (given the effort and time I have invested in trialling it), my conclusion is that it is not suitable. Your desktop client is buggy and puts data at risk. This is a long-standing bug and clearly the client is not your priority. I posted on this ticket to see what would happen. What happened is that I was told it was not a real issue and data was not at risk. I was attacked and then the ticket was locked so I could not give any response. My advice would be not to tell users that they are wrong: they know their use-cases better that you do. It is also not reasonable to demand that people buy an SLA before you will fix critical bugs.. I do not buy SLA's so that I can ask people to fix buggy software. It's a very bizarre argument to me and I have never heard it before. If you don't want commercial enterprises to know about your bugs, don't release your product under the AGPL.

Unfortunately, the level of professionalism and customer-focus shown (by no one individual, but as a whole) means that I could never recommend Nextcloud. I have torn-down my Nextcloud VM and am working on trialling an alternative solution.

For what it's worth, I found the server to be quite impressive, the rest not so much. Your Android client is OK but needs a lot more work. Your mistake is to assume that any critique on your product are a critique on the developers, but this is not the case. It could be a lack of resources, problems with upstream libraries, anything... but frankly your customers or potential customer do not care, you are offering a product here. Nextcloud needs to look at itself and decide what it is... are you are a for-profit company or an open-source charity? Are you employees or volunteers? If you are employees then your job is to serve customers, not insult them, and to take honest feedback on your product on-board and do what you need to do to make it better. That is what it means to be in business.

@jancborchardt

This comment has been minimized.

Copy link
Member

commented May 10, 2019

@dothebakedpotato please note @camilasan’s reply and apology above at #1000 (comment) – no need to further inflame this conversation.

@Germano0

This comment has been minimized.

Copy link

commented May 10, 2019

I have just tested #1245 and it seems to fix the problem. I have just released a new updated version into Fedora testing repositories.

This bug affected also owncloud client 2.5.0 owncloud/client#6804 You may want to check how they fixed it https://github.com/owncloud/client/pull/6806/files

@bill-mcgonigle

This comment has been minimized.

Copy link

commented May 10, 2019

paging @ivaradi for Debian (FYI, István).

camilasan added a commit that referenced this issue May 10, 2019

@camilasan

This comment has been minimized.

Copy link
Member

commented May 10, 2019

PR merged.

@camilasan camilasan closed this May 10, 2019

Desktop Client 2.5 Series automation moved this from To do to Merged | Done | Release! May 10, 2019

@tomharristech

This comment has been minimized.

Copy link

commented May 18, 2019

I have folders that have been moved around previously that haven't been synced - any ideas on how I can sort this out?

@tobiasKaminsky

This comment has been minimized.

Copy link
Member

commented May 20, 2019

Restart the client should help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.