Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Up/download stalls on iMac #152
I am using Dropbox embedded in an application and I have observer a very strange behaviour. On almost all of my devices, up/downloading files with
It works on all my iOS devices and on my Macbook Pros. But it stalls on my quadcore iMac.
I have traced the problem and added debug statements to the
This is not a network problem, because at the same time all my other devices in the same network all work fine.
When I force the
Any idea why this happens. I am only starting at most 10 simultaneous downloads at a time, so it cannot be a problem with thread explosion or anything.
This is all tested with 3.0.18
added a commit
May 26, 2017
@pascalfribi: Any update here? Are you still seeing this issue?
Can you also confirm that your app is not in the background when the download finishes? It won't execute your handlers until the app is in the foreground again.
Would you mind updating to 3.1.0 as well?
Given that this is an issue only on your quadcore iMac and not other apple platforms, I'm more inclined to think it's an
I'll see if I can get a hold of an iMac to test this out. Any more specifics you can give about in which cases this problem arises?
Hi, since I force the foreground session I did not see the problem anymore. Be aware that foregound/background sessions on the macOS have a different meaning than on the iOS. macOS has no such thing as running in the background with limited resources. This is a thing introduced on iOS to prevent battery drain. On a normal unix system you are allowed to run as much in the background as you want to. As for NSURLSession the docs from Apple explicitly state that there is a difference on what it means on macOS. On macOS a background NSURLSession continues to run and eventually finishes even if you application has been terminated. A foreground session is of course stopped when the process terminates. So there is no real difference whether the app is in the foreground or in the background. Still I made those tests with the same result. I have some doubts that the problem really is in the Apple code. NSURLSession is such a basic functionality that I would expect that my iMac would not run properly at all, if the issue is in this library. I rather suspect it has to do with the 4 cores where some race condition is more likely to happen than on 2 cores. But this is really just a wild guess. As for the 3.1.0 version. I made the tests as well on 3.1.0 with the same result. How it all happens is as follows: - my software uses an excellent library for CoreData syncing across devices. This is called Ensembes (you should try it if you have a need for this). This library can sync data with iCloud, dropbox and other backends. It has support built in for dropbox V1 and V2 API. - I first thought the problem is in this library and I worked together with the developer to track down the issue. At the end of the day this is what this framework does: - it checks on dropbox if there are new files to sync (with the longpoll stuff) - if there is new stuff around it downloads the data. It does this in blocks of 10 files at a time. Only when these 10 files have been downloaded, a new block of 10 files is downloaded (or less of course if there is not more) - Sometimes it of course also uploads data to dropbox with the same approach (max 10 files) - It does not do both at the same time - Now the problem of stall might happen at any moment. Sometimes everything works fine, sometimes some files are properly down/uploaded and then suddenly it stalls and nothing works anymore. As the framework waits for the completion of this up/download, the complete sync then stalls. - What I have seen is that when it stalls all consecutive downloads session stall (==> first 4 out of 10 could work and then 5 to 10 would stall indefinitely). So really very basic. Hope that explains it a little. If there is something I need to test, please let me know. Regards Pascal…
On 1 Jun 2017, at 02:36, Stephen Cobbe ***@***.***> wrote: @pascalfribi <https://github.com/pascalfribi>: Any update here? Are you still seeing this issue? Can you also confirm that your app is not in the background when the download finishes? It won't execute your handlers until the app is in the foreground again. Would you mind updating to 3.1.0 as well? Given that this is an issue only on your quadcore iMac and not other apple platforms, I'm more inclined to think it's an NSURLSession bug. I'll see if I can get a hold of an iMac to test this out. Any more specifics you can give about in which cases this problem arises? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#152 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AF4HdOnZpTlxoV-9XaL5CqUnTqnt86K-ks5r_gedgaJpZM4NgMC4>.
This seems to still exist as of 3.9.1.
I noticed occasional difficulty with sync on an iOS app. Certain files would seem to fail to download under various circumstances. I couldn't find a clear reason.
So I built a test app on macOS (my dropbox handling code is cross-platform.) I discovered that I could reliably upload files, but occasionally had trouble downloading a file.
I could reliably reproduce this by creating two test files, which are then uploaded. I then duplicated one of the files in my regular Dropbox folder, so that it is uploaded by the macOS Dropbox app. When my test app received notification of the new file, it attempts to download the file, but fails:
No matter how much I tried, the response block was never called.
After finding this thread, I used this in my setup:
And at this point syncing seemed to work flawlessly during my tests. I have not tried this on iOS yet.
My test mac environment is a Late 2012 Mac Mini running 10.12.6. (4 cores).
If there is something else useful that I can offer, please let me know.
The bridging code between the test app and the Dropbox SDK is proprietary so I can’t share that at this time. I can see if I can strip it down to just the bare bones SDK calls rather than the full sync library and post that. (Assuming that is enough to still trigger the problem.)…
Sent from my iPhone
On Sep 11, 2018, at 12:38 PM, Greg K ***@***.***> wrote: @fletcher Thanks for the information! I don't believe we were ever able to reproduce this here. Would you be able to provide the test project for reproducing this? Thanks in advance! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
This seems to replicate it. Create a new Xcode macOS Cocoa App project with the following
Run the program in Xcode and watch the output window. The program will upload two fake files to your Dropbox folder. It then checks for updates repeatedly. It should successfully upload the two files, and then successfully download them (though it doesn't save them anywhere.)
Then use the Finder to duplicate one of the files in the Dropbox folder so that the Dropbox client uploads it. You should then get a message attempting to download the new file, but no message indicating successful download.
If you change
@fletcher Thanks for putting this together. I just tried this, but the issue initially didn't reproduce for me (even without the fix):
I notice you're not doing any error handling for the
This is a stripped down version of the original code that does not include all of the original debugging I tried to use to narrow down the issue. You can, of course, add whatever debug logging you like. But when the problem occurs, the response block is not called, so there is nowhere from which to print out
More interestingly, I just rebooted my machine (after it was powered down for several hours for unrelated reasons) and now the test fails to demonstrate the problem (in other words the download works successfully.) Restarting the app doesn't fix it. Which fits with my iOS experience where sometimes I successfully sync many files from multiple devices without issue, and other times it gets "stuck" and unable to download new files, even after force quitting the app.
Without being familiar with the underlying dropbox code, it seems that the issue would fit into 1 of 2 scenarios:
The irregular nature of the issue makes me think about a threading issue. I run the test app and my real app with a variety of diagnostics, but most frequently leave the thread debugging on because that is one of the things I have been focused on. I have not found any thread issues.
I do occasionally get exceptions that occur within the dropbox SDK. Descending the stack trace does not identify any obvious parameter issues in my code (though it's not impossible.) I have not noticed any particular pattern to the exceptions. They are infrequent but not exceptionally rare (perhaps 2 or 3 in the hundred times or so of running the test up while creating and debugging it??) I have not noticed any on-device crashes attributable to Dropbox, only in the simulator.
Perhaps importantly, when I rebooted my mac I had also powered down my network (storm preparations for hurricane). So perhaps the overall network is running a bit "cleaner" in some way that prevents a small issue from occurring that was triggering the dropbox issue?
PS> By the timestamps, it appears that you waited 9 minutes between running the app and duplicating the file (at least on this attempt). That may or may not play a role, but I usually wait a few seconds after successfully downloading the first two files and then proceed.
@fletcher Thanks for the detailed information! I was just checking in case it only appeared that the response block wasn't getting called because you were only printing something in the successful case. It's good to know you already checked that.
Anyway, I wouldn't expect scenario 2; any error should get bubbled back to the response block. Scenario 1 is expected if the network connection is lost (see note), but that wouldn't seem to apply here since we've only been discussing attempts while using a working network connection. That said, given the result you described here, it does seem possible there was a network issue leading to this.
Unfortunately, it will be hard to investigate without a reliable way to reproduce it, but I'll leave this open for the team.
Yeah -- I was intrigued that the test failed so consistently and then worked after rebooting Mac and my network.
Also, in case it isn't obvious. When the test failed, I would run it again and the first two uploads/downloads would work. It was only the 3rd download that failed to work. Every time.
So perhaps something about uploading "cleared something" (e.g. a buffer or a boolean) that then allowed the download to work properly. But an "unexpected" download, didn't work.
But again, all of this is in the context of a sync program that seems to work properly 95%+ of the time, and a test suite that managed to fail 95%+ of the time (it worked once or twice). But after the reboot it works 100% so far.
If this is ever solved, I hope to hear an explanation. I'd love to understand what set of circumstances causes the failure!