Files will not download when using curl 7.62.0 (Arch Linux)

[sim31]

Bug Report Details

After some packages were updated on my system, onedrive is unable to sync remote changes. I know it's because of updates, because I was able to reproduce on another machine (same OS). Uploading changes works fine.

During update curl and dmd were updated, among other packages. But I think only those two are relevant. No other dependencies were upgraded. I suspect dmd is only used to build and I didn't rebuild, so I suspect it's curl?

curl version: curl 7.62.0

This app was installed from AUR: https://aur.archlinux.org/packages/onedrive-abraunegg-git/

Application and Operating System Details:

• OS: Arch Linux
• Application version: onedrive v2.1.4-6-geb2cbfd
• DMD or LDC compiler version: DMD64 D Compiler v2.083.0

To Reproduce
Probably:
1.Update curl to 7.62.0
2.Try to download remote changes
3. See error

Complete Verbose Log Output
A clear and full log of the problem when running the application in the following manner (ie, not in monitor mode):

onedrive --synchronize --verbose
Loading config ...
Using Config Dir: /home/tadas/.config/onedrive
Initializing the OneDrive API ...
Opening the item database ...
All operations will be performed in: /home/tadas/OneDrive
Initializing the Synchronization Engine ...
Account Type: personal
Default Drive ID: 5dc48a7432385a6d
Default Root ID: 5DC48A7432385A6D!103
Remaining Free Space: 4664290569
Fetching details for OneDrive Root
OneDrive Root exists in the database
Syncing changes from OneDrive ...
Applying changes of Path ID: 5DC48A7432385A6D!103
The local item is unsynced, renaming
Downloading file notes-ts/test.md ... done.
notes-ts/test.md: No such file or directory
Syncing changes from OneDrive ...
Applying changes of Path ID: 5DC48A7432385A6D!103
The local item is unsynced, renaming
Downloading file notes-ts/test.md ... done.
notes-ts/test.md: No such file or directory
Syncing changes from OneDrive ...
Applying changes of Path ID: 5DC48A7432385A6D!103
The local item is unsynced, renaming
Downloading file notes-ts/test.md ... done.
std.file.FileException@/usr/include/dlang/dmd/std/file.d(1556): notes-ts/test.md: No such file or directory
----------------
??:? @trusted bool std.file.cenforce!(bool).cenforce(bool, scope const(char)[], scope const(char)*, immutable(char)[], ulong) [0x817ff68c]
??:? @safe void std.file.setTimes!(immutable(char)[]).setTimes(immutable(char)[], std.datetime.systime.SysTime, std.datetime.systime.SysTime) [0x8180fe76]
??:? void sync.SyncEngine.downloadFileItem(itemdb.Item, immutable(char)[]) [0x8184353f]
??:? void sync.SyncEngine.applyNewItem(itemdb.Item, immutable(char)[]) [0x81842eab]
??:? void sync.SyncEngine.applyDifference(std.json.JSONValue, immutable(char)[], bool) [0x81842d41]
??:? void sync.SyncEngine.applyDifferences(immutable(char)[], const(char)[]) [0x8184222f]
??:? void sync.SyncEngine.applyDifferences() [0x81841261]
??:? void main.performSync(sync.SyncEngine, immutable(char)[], bool, bool, bool) [0x818265ea]
??:? _Dmain [0x81825e80]

[abraunegg]

@sim31

During update curl and dmd were updated, among other packages. But I think only those two are relevant. No other dependencies were upgraded. I suspect dmd is only used to build and I didn't rebuild, so I suspect it's curl?

Curl is used during the build process as well - so if it has changed, then you will need to rebuild the client to ensure that the updated libraries are used.

This being said I am able to replicate the issue above after performing a full update via pacman -Syu which updated curl and dmd & rebuilding the client, so it looks like something in the updated curl library is breaking the client.

[abraunegg]

@sim31
The workaround for the moment is to downgrade the curl version to the prior one until I can work out why this is breaking the download of files.

[simvux]

thank god there's an easy solution. I was worried Microsoft had changed their API and breaking everything

[abraunegg]

I suspect (yet to confirm as rebuilding packages on Arch is not my knowledge area) that this issue potentially stems from this default behaviour change in curl 7.62 (https://curl.haxx.se/mail/archive-2018-10/0033.html):

• url: default to CURL_HTTP_VERSION_2TLS if built h2-enabled
• multiplex: enable by default

[abraunegg]

Listed issue here for analysis as I think this is a curl issue given when looking at https debug details the requests are 'flip flopping' between HTTP 1.1 and HTTP 2

curl/curl#3253

The attempt of the file download is occurring under HTTP 2 & this is where it fails. Under curl 7.61 this call is a HTTP 1.1 call and it succeeds .

[abraunegg]

Users of Arch Linux are advised to roll back to curl 7.61.x version to prevent any data loss scenario's where files are not being downloaded correctly and potentially being deleted from OneDrive.

[abraunegg]

Can someone who is running Arch Linux & is affected by this issue rebuild libcurl 7.62.0 on Arch Linux with ./configure --enable-debug and then run a sync with the following command:

onedrive --synchronize --verbose --debug-https

I am not an Arch Linux user, thus rebuilding a package & enabling debug is not my area of knowledge - CentOS / RHEL no problem.

This will assist in hunting down this problem.

[abraunegg]

For those users affected by this issue, please can you test this PR:

git clone https://github.com/abraunegg/onedrive.git
cd onedrive
git fetch origin pull/225/head:pr225
git checkout pr225
make
make install

[A1TV]

The PR works on manjaro
libcurl 7.62.0

onedrive --synchronize --verbose        
Loading config ...
Using Config Dir: /home/a1/.config/onedrive
Initializing the OneDrive API ...
Opening the item database ...
All operations will be performed in: /data/Onedrive/
Initializing the Synchronization Engine ...
Account Type: personal
Default Drive ID: 47ea50cd3c718f61
Default Root ID: 47EA50CD3C718F61!101
Remaining Free Space: 1037333676894
Fetching details for OneDrive Root
OneDrive Root exists in the database
Syncing changes from OneDrive ...
Applying changes of Path ID: 47EA50CD3C718F61!101
Uploading differences of .
Processing root
Uploading new items of .
Uploading file ./test ... done.
Remaining free space: 1037333676780
Uploading file ./test.png ... done.
Remaining free space: 1037333669378
Applying changes of Path ID: 47EA50CD3C718F61!101

[sim31]

PR fixes it for me too.

[virtualflyer]

works here as well

[abraunegg]

PR #225 merged

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.