Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

computeQuickXorHash generates a different hash when files are > 64Kb #23

Closed
abraunegg opened this issue Jun 16, 2018 · 7 comments
Closed
Labels
Bug Something isn't working Fixed

Comments

@abraunegg
Copy link
Owner

When looking at the solution for #2, it has been identified that the function which generates the quickXorHash hash for comparison with what is returned by OneDrive, generates a different value to what is expected when the file size is > 64Kb.

When the file is under 64Kb, the quickXorHash that is generated matches what is returned from OneDrive.

Simply 'touching' a local file > 64Kb will trigger this hash difference:

The file last modified time has changed
file mtime = 2018-Jun-16 00:52:06Z
item mtime = 2018-Jun-16 00:51:30Z
item hash = tJIwyksN/FCJfJML1uY05oBbHXY=
file hash = tJIwyksN/FBrg2z0KRnL+6hbHXY=
The file content has changed
Uploading file hcw85prop.ax ... done.

In reality if a local file has been modified, the content 'may' have changed, thus the modified file will be uploaded as expected, however when simply 'touching' the file this should not occur (upload modified file due to contents changing - when it hasnt).

Seeking some assistance to look at qxor.d to work out where things are potentially going wrong as it is not immediately obvious.

OneDrive API developer documentation around quickXorHash is here:

https://docs.microsoft.com/en-us/onedrive/developer/code-snippets/quickxorhash

@abraunegg
Copy link
Owner Author

This issue is probably one of the causes of the 'ETag does not match current item's value' issue that has plagued this client since the early days.

@robertjf
Copy link

looking at the reference implementation (https://github.com/OneDrive/onedrive-api-docs/blob/live/docs/code-snippets/quickxorhash.md), there are a few slight differences - will see how I go if I have time this morning

@abraunegg
Copy link
Owner Author

Thanks - any assistance is greatly appreciated

abraunegg added a commit that referenced this issue Jul 2, 2018
…Issue #2) (#40)

* Resolve multiple versions of file shown on website after single upload - however Issue #23 & OneDrive API bug (OneDrive/onedrive-api-docs#877) will still create 2 file versions on OneDrive Business. The work around is to disable file versions until #877 is resolved.
@robertjf
Copy link

robertjf commented Jul 10, 2018

well, I got this one licked. PR incoming, just got to fix up my repository first.
Actually, might be easier if I just put the necessary changes in here:

onedrive/src/qxor.d

Lines 35 to 37 in c0d2874

byte low = cast(byte) (bitsInVectorCell - vectorOffset);
byte xoredByte = 0;

change to

				ubyte low = cast(ubyte) (bitsInVectorCell - vectorOffset);

				ubyte xoredByte = 0;

that's it. problem solved :)

I also made a few other minor cosmetic changes to my local copy, but they shouldn't have any impact.

robertjf pushed a commit to robertjf/onedrive that referenced this issue Jul 10, 2018
Fixes QuickXorHash issue
@abraunegg
Copy link
Owner Author

Thanks so much - will validate the PR later today

abraunegg pushed a commit that referenced this issue Jul 10, 2018
* Resove computeQuickXorHash generates a different hash when files are > 64Kb
@abraunegg
Copy link
Owner Author

Resolved by PR #60

@lock
Copy link

lock bot commented Jan 6, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Jan 6, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Bug Something isn't working Fixed
Projects
None yet
Development

No branches or pull requests

2 participants