Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

client 2.2.3-nightly stop sync on large shared folder - #5092

Closed
PenIv opened this issue Jul 29, 2016 · 14 comments
Closed

client 2.2.3-nightly stop sync on large shared folder - #5092

PenIv opened this issue Jul 29, 2016 · 14 comments
Labels
Discussion Server Involved sev2-high
Milestone
2.2.3

Comments

@PenIv
Copy link

PenIv commented Jul 29, 2016 •
edited
Loading

Expected behaviour

Client should sync all files

Actual behaviour

Client start "Checking for Changes" in large folder and stop with error "An error occurred while opening a folder"

Steps to reproduce

  1. Upgrade to client 2.2.3 (build 6279)
  2. Start client
  3. Wait for client to rescan large folder (7.75 GB with 11 325+ files)

Server configuration

Operating system:
Ubuntu 14.04
Web server:
Nginx/1.4.6
Database:
MySQL
PHP version:
5.5.9
ownCloud version:
9.1.0

Client configuration

Client version: 2.2.3 (build6279)

Operating system:
Windows 10
OS language:
English
Installation path of client:
C:\Program Files (x86)\ownCloud

Logs

  1. Output of owncloud --logwindow or owncloud --logfile log.txt
    ocLogwindow.zip

  2. Web server error log:
    owncloud.log.txt

  3. ownCloud log (data/owncloud.log):
    ...
    |0|Attach|INST_SYNC|Down|1469102824|5790baea1823c|0|00386230ocaf51bd2515|4||0|0|1409387492|||INST_NONE|
    |0|Attach/Oudtshoorn|INST_SYNC|Down|1469102824|5790baea07d75|0|00386268ocaf51bd2515|4||0|0|1467468787|||INST_NONE|
    |0|Attach/Oudtshoorn/Photos|INST_SYNC|Down|1469102824|5790bae9ebe22|0|00388822ocaf51bd2515|4||0|0|1469028218|||INST_NONE|
    12:12:15|7945|Attach/Oudtshoorn/Photos/10550_0 DSCN6019.JPG|INST_NEW|Down|1469102795|5156d4494e74949b7f9513b12e656d56|406135|00616484ocaf51bd2515|4||0|0|0|||INST_NONE|
    12:12:24|4949|Attach/Oudtshoorn/Photos/10550_0 DSCN6020.JPG|INST_NEW|Down|1469102789|eb784ebf1ad1d9e55084d958fec49b87|252649|00616485ocaf51bd2515|4||0|0|0|||INST_NONE|

    =#=#=# Syncrun started until (0 msec)

    =#=#=# Syncrun started until (0 msec)

    =#=#=# Syncrun started until (0 msec)

    =#=#=# Syncrun started until (0 msec)

    ...
@PenIv
Copy link
Author

PenIv commented Jul 29, 2016 •
edited
Loading

If I fall back to version 2.2.2 all folders sync successfully.
(I am using the nightly because of unrelated issue #5019 in 2.2.2.)
This is a similar scenario as described in issue #5070 with 507 errors for User B and increasing the quotas to unlimited does stop the 507 errors on the server but the error that I encounter with owncloud client 2.2.3 is experienced by User A on the same shared folder.

@dragotin
Copy link
Contributor

dragotin commented Aug 2, 2016

might be related: owncloud/core#25582

@guruz
Copy link
Contributor

guruz commented Aug 2, 2016 

07-29 13:35:08:893 0x6292278 OCC::DiscoveryJob::remote_vio_opendir_hook: OCC::DiscoveryJob(0xabe01d8) Attach/Dysselsdorp/Plans Calling into main thread...
07-29 13:35:08:893 0x9d76f0 OCC::AbstractNetworkJob::start: !!! OCC::LsColJob created for "https://***/owncloud" + "/MASS/Oudtshoorn 2011/Attach/Dysselsdorp/Plans" "OCC::DiscoverySingleDirectoryJob"
07-29 13:35:28:081 0x9d76f0 OCC::Folder::slotRunEtagJob: * Trying to check "https://***/owncloud/remote.php/webdav/MASS/RKN" for changes via ETag check. (time since last sync: 144 s)
07-29 13:35:28:083 0x9d76f0 OCC::Folder::slotRunEtagJob: * Trying to check "https://***/owncloud/remote.php/webdav/MASS/MunsActive" for changes via ETag check. (time since last sync: 141 s)
07-29 13:35:28:084 0x9d76f0 OCC::FolderMan::slotRunOneEtagJob: Scheduling "https://***/owncloud/remote.php/webdav/MASS/RKN" to check remote ETag
07-29 13:35:28:086 0x9d76f0 OCC::AbstractNetworkJob::start: !!! OCC::RequestEtagJob created for "https://***/owncloud" + "/MASS/RKN" "OCC::Folder"
07-29 13:35:30:458 0x9d76f0 OCC::FolderMan::slotRunOneEtagJob: Scheduling "https://***/owncloud/remote.php/webdav/MASS/MunsActive" to check remote ETag
07-29 13:35:30:458 0x9d76f0 OCC::AbstractNetworkJob::start: !!! OCC::RequestEtagJob created for "https://***/owncloud" + "/MASS/MunsActive" "OCC::Folder"
07-29 13:35:38:895 0x6292278 OCC::DiscoveryJob::remote_vio_opendir_hook: OCC::DiscoveryJob(0xabe01d8) Attach/Dysselsdorp/Plans ...Returned from main thread
07-29 13:35:38:895 0x6292278 OCC::DiscoveryJob::remote_vio_opendir_hook: 5 when opening Attach/Dysselsdorp/Plans msg= ""
07-29 13:35:38:895 0x6292278 OCC::DiscoveryJob::remote_vio_closedir_hook: static void OCC::DiscoveryJob::remote_vio_closedir_hook(csync_vio_handle_t*, void*) OCC::DiscoveryJob(0xabe01d8) "/MASS/Oudtshoorn 2011/Attach/Dysselsdorp"
07-29 13:35:38:895 0x6292278 OCC::DiscoveryJob::remote_vio_closedir_hook: static void OCC::DiscoveryJob::remote_vio_closedir_hook(csync_vio_handle_t*, void*) OCC::DiscoveryJob(0xabe01d8) "/MASS/Oudtshoorn 2011/Attach"
07-29 13:35:38:895 0x6292278 OCC::DiscoveryJob::remote_vio_closedir_hook: static void OCC::DiscoveryJob::remote_vio_closedir_hook(csync_vio_handle_t*, void*) OCC::DiscoveryJob(0xabe01d8) "/MASS/Oudtshoorn 2011"
07-29 13:35:38:895 0x9d76f0 OCC::SyncEngine::handleSyncError:  #### ERROR during  csync_update :  "An error occurred while opening a folder "

5 is EIO.. checking more.

I know that @jturcotte and @ogoffart played around with the discovery mutexes for 2.2.3 to fix a crash/deadlock..

@guruz
Copy link
Contributor

guruz commented Aug 2, 2016

Isn't this a timeout if it is exactly 30 sec? 35:08 vs 35:38..

@jturcotte
Copy link
Member

jturcotte commented Aug 2, 2016 •
edited
Loading

That would be 622017a yeah. Let's revert it.

This means that our main thread is sometimes blocked for more than 30 seconds, and that the GUI isn't responsive in that case, so this is bad. But I should have guessed, we'll have to fix that deadlock some other way.

@guruz
Copy link
Contributor

guruz commented Aug 2, 2016

@PenIv What kind of folder is /MASS/Oudtshoorn 2011/Attach/Dysselsdorp/Plans for you?
Is it a folder shared to you?
Or a subfolder of a folder shared to you?
Are you using external/remote storage on your oC server?

@guruz guruz modified the milestones: 2.3.0, 2.2.3 Aug 2, 2016
@guruz
Copy link
Contributor

guruz commented Aug 2, 2016

@dragotin Let's wait for @PenIv 's input to decide if this needs to be changed for 2.2.3 or 2.3.0

@danimo I wonder how many people downloaded/tried 2.2.3beta1..

@PenIv
Copy link
Author

PenIv commented Aug 2, 2016 •
edited
Loading

I (UserA) Share the parent folder /MASS/Oudtshoorn 2011 with 2 other users.
User B as Read/Write and user C as read-only.
User B caused the 507 errors in the system log which was fixed by increasing his quota to Unlimited (on 2.2.2).
User C sync without any issues (also on 2.2.3-nightly).
No remote storage.
Not sure if relevant but new files are currently added by User B and if I fall back to 2.2.2 it sync the new files with no errors.

@guruz
Copy link
Contributor

guruz commented Aug 2, 2016

@PenIv But the sync log was from UserA? So userA gets the An error occurred while opening a folder?

Why would something cause a PROPFIND to take so long @PVince81 @DeepDiver1975 ?

@PenIv
Copy link
Author

PenIv commented Aug 2, 2016

Yes. Sync log with error is from User A.

@jturcotte jturcotte mentioned this issue Aug 3, 2016
Merged
jturcotte added a commit that referenced this issue Aug 3, 2016
@jturcotte
Revert "Fix a deadlock when shutting down during discovery" (#5100)
b307f2b 
Reverts commit 622017a

Could be the cause of #5092 and the cost is higher than the benefit if this is the case.
A network request taking more than 30 seconds isn't something unlikely in this world
and shouldn't be a good reason to abort. We should try to untangle the threads
dependencies to properly fix this if possible instead.
@PenIv
Copy link
Author

PenIv commented Aug 4, 2016

@jturcotte Thank you for the fix fix. No more errors on large folders since 2.2.3-nightly20160804 (build 6294)

@PenIv PenIv closed this as completed Aug 4, 2016
@jturcotte
Copy link
Member

You're welcome, thanks for proactively verifying 🙂

@guruz
Copy link
Contributor

guruz commented Aug 4, 2016

@PenIv How many folders/files are directly under /MASS/Oudtshoorn 2011/Attach/Dysselsdorp/Plans? (not subfolders/files)

@PenIv
Copy link
Author

PenIv commented Aug 4, 2016

0 folders, 3442 files under /MASS/Oudtshoorn 2011/Attach/Dysselsdorp/Plans.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Discussion Server Involved sev2-high
Projects
None yet
Milestone
2.2.3
Development

No branches or pull requests
4 participants
@jturcotte @guruz @dragotin @PenIv