-
-
Notifications
You must be signed in to change notification settings - Fork 861
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bug: Issues with onedrive 2.5.0-rc1 #2666
Comments
@gitbls
Firstly, when enabling logging I would always also include
This will be 100% correct for a first run with v2.5.0 as there has been lots that has changed, including the DB schema which is not backwards compatible. This is essentially also the same as doing a You can redo this test with either dropping back to v2.4.25 (the DB schema will be incompatible, and perform a full scan) - to which you can then also determine the time difference between v2.5.0 and v2.4.25 for that --resync scenario. IMPORTANT POINT: Because you are also using So this is not a bug or issue.
This is 100% correct because it has no record of these as being stored online at this point in time - because of the DB Schema change which is the same as doing a So this is not a bug or issue, but is one of the application output changes.
OK .. this essentially means that the file already exists online. Due to you not using If the size or hash (meaning the content of the file online does not match) is different that is when the local file is backed up so that you can then determine what is the correct file ... maybe you edited or cropped the file locally ... who knows ... but a difference was detected between online and local and action was taken to preserve your data. So this is not a bug or issue. To assist with your understanding all of the above 3 items, please read the Client Architecture Documentation and potentially change your usage to include / add
This is a genuine error, but without any context as to where this is occurring or why. A HTTP Status Code 502 is a standard HTTP response status code that indicates a gateway or proxy server received an invalid response from an upstream server. In the context of calling an API, this usually means that the OneDrive service is acting as a gateway or proxy and attempted to forward your request to another server but received a bad response from that server. A 502 error is usually indicative of issues on the server side or with the network, not with the API requests the 'onedrive' application is making. To dig into this actual error, a full debug log would be required to dig into where and how this is being generated and what the 'onedrive' application did when this was encountered - was the request retried correctly .. ?? Please read: https://github.com/abraunegg/onedrive/wiki/Generate-debug-log-for-support to generate a debug log. Once you have got this log available, please generate password protected archive and email the debug logfile through. If you are concerned about data sensitivity, please read: |
Thanks @abraunegg for the detailed response. I'll read through the links you've provided and run another test as suggested. ETA Sunday US west coast time. |
Please also remember - any use of |
@gitbls |
Sadly, I'm not exactly clear how to do this. What's the most efficient way to pull down that version? I've been wrestling with a problematic upload of one file, just got a log with Thx EDIT: can I just download the zip file from https://github.com/abraunegg/onedrive/tree/onedrive-v2.5.0-release-candidate-1? |
I use a script titled
Depending on how you have installed the compiler (you are using Debian 12, so you could just add
No .. as that is the initial code drop for RC1 .. not where the PR is now with documentation fixes, a couple of other items, your spelling error catch etc. |
Sadly, I'm not a git wizard. Whats the best
Thx! So just to confirm...PR 2661 is what I want, or do I want 2668? |
PR 2661 is correct: #2661 |
Running test now. Thx! |
OK, here's where I'm at. Using the latest onedrive, syncing upload of the whole tree fails to upload HomeV3.QDF. I've attached that log (log9.1), which i trimmed down to the relevant stuff. However, if I do a single directory (the one that contains HomeV3.QDF,) it worked fine, Well, it worked twice in a row. (log11) On the released version (v2.4.25-1+np1 installed via apt from download.opensuse.org) it failed more frequently, but then, I didn't have the ip_protocol_version setting (see below) Also, when I started seeing the error as demonstrated in log9.1, a couple of other things:
Thx! |
@gitbls As the log indicates, using 'ip_protocol_version = 1' in your config file is the right approach. This option is available in v2.4.25 as well: https://github.com/abraunegg/onedrive/blob/master/docs/USAGE.md#ip_protocol_version This option was put in as it has been proven beyond all doubt that libcurl itself gets hung up with some IPv6 stuff, which is why forcing it to use IPv4 only stops this resolution snafu from occurring. This is a libcurl bug unfortunately that it will flip/flop between IPv4 and IPv6 for resolution if not explicitly told to use IPv4. Additionally, you are using Debian - thus are locked into using old packages - your version of libcurl was released in February 20 2023 and there are a number of resolution IPv4/IPv6 fixes post then. I would attempt to use Also - non of your logs are showing all the actual HTTPS Debug logs either. The actual libcurl parts will only display to the console (for data sensitivity reasons), which is why the logging when using that option is slightly different. Please read: https://github.com/abraunegg/onedrive/wiki/Generate-https-debug-log-for-support When you review your console generated debug log, you will see then all the HTTPS parts, and see where libcurl is trying to resolve against IPv6 .. and this is where it then fails - hence why then IPv4 should be used to exclude IPv6 resolution. Now .. you mention you changed the IP handling and it didnt resolve the issue ... if you add |
Yep, I was today day's old when I discovered that
But using ip_protocol_version=1 should prevent these curl problems?
I'll do that.
That's weird. The only thing that I did differently than that page was leaving off the
I added the ip_protocol_version = "1" long before these last 2 runs. I mistakenly left the quotes off and noticed that it wasn't being set. Will add |
Here's the log output for
|
Correct - it should work around the IPv6 issues
Great .. so it is being set and used .. thus, libcurl should not be using IPv6 for all operations (DNS, API access etc) |
Just ran the single directory 4 times and it worked every time. I did see this fail more frequently with the released version (even doing a single directory) so this is an improvement. Will add some additional files to the directory to see if that helps stimulate the error. If that doesn't do it, I'll try it a few times without all the logging, perhaps that is slowing things down enough such that it doesn't fail? Also, you mentioned there was no debug-https output. What should that look like (so I can ensure that I'm getting it going forward)? Thanks for your help with this! |
@gitbls
and so on. This sort of logging is only sent to the console / ssh session which is why redirection of console using For your testing at the moment I would remove
Looking back at the prior comment .. this also stands out. HTTP/2 introduces a binary framing layer at its core, which is a departure from the newline-delimited text format of HTTP/1.x. This binary framing layer is designed to improve performance over HTTP/1.x by allowing multiplexing of requests and responses, stream prioritisation, and more efficient use of network resources. If the CPU is peaking, it means something is not working right in libcurl. As a result. the next piece of advice I have is that you drop down to using pure HTTP/1.1 rather than HTTP/2 + some potential HTTP/1.1 communication streams. Use either Your resulting end up configuration might be:
And this will be down to the fact of Debian using a old curl version ....... |
Giving it a total of 4 QDFs each about 200MB or so causes it to fail in single-directory mode. Here's the latest log: |
Great .. so this confirms somthing odd going on with libcurl |
Yep, and I forgot to mention: if you look at log-17, search for 17:04:28, and notice that the previous timestamp is four minutes earlier. I think I mentioned this previously. Also, onedrive was consuming 300% CPU during that period. NUC fan was really humming! |
Yes - however as the logfile provided has all the dates - this is the '--enable-logging' log .. not the console log, which is missing the other curl outputs .... Also this is all HTTP/2 failures. Suggest you try:
and go from there. |
Using force_http_11="true" in the config file works for all 3 files. 🥂 Is this relatively "safe" to use as a workaround while I putz around to see if I can update curl and company? |
Using As this client 100% relies on curl/libcurl to operate - having a relative bug free curl/libcurl version is key to having 'onedrive' operate smoothly. Now that you know the cause, this issue can be potentially resolved as your issue is with libcurl and HTTP/2 and not 'onedrive' |
Indeed. I'll sort curl out, and I need to get back to project, sdm. (RasPi tool) I have some cool features just waiting for me to test them. Not a onedrive bug, thanks for your help in sorting this out! |
Closing issue as this is not an issue with 'onedrive' |
@gitbls |
Thx! |
@abraunegg Thanks again for your help in sleuthing this out. I haven't yet installed the libcurl backport, but it's on my todo list. I wanted to mention that this new version is most definitely faster! My nightly backup used to take about 13-15 minutes, and now takes less than 2 minutes for a similar update (running through the whole directory tree and uploading a single 200+MB file that changed. |
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Describe the bug
I built 2.5.0-rc1 on Debian 12 X64. Did
which took almost 4 hours to complete. (Using the current released version without
--dry-run
takes 15-20 minutes).It also reported:
which surprised me given that a sync was done a few hours prior and everything in onedrive cloud was up to date.
There were several of these errors:
and a couple of this error:
Operating System Details
Debian 12 x64 Linux mondo 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64 GNU/Linux
Client Installation Method
From Source
OneDrive Account Type
Personal
What is your OneDrive Application Version
2.5.0-rc1
What is your OneDrive Application Configuration
What is your 'curl' version
Where is your 'sync_dir' located
Local
What are all your system 'mount points'
What are all your local file system partition types
How do you use 'onedrive'
onedrive is run in a systemd timer-enabled script nightly. The script does the command:
Steps to reproduce the behaviour
I'm sure I can easily reproduce it here by switching to the new verison of onedrive with the above command.
Complete Verbose Log Output
I could provide this by doing another run, but there is a huge amount of personal information that I would need to redact (directory names, file names, etc).
Please let me know if you need this after you've reviewed the above information.
Screenshots
No response
Other Log Information or Details
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: