Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix GoogleDrive and Dropbox getFile returning corrupted binary data #1155

Merged
merged 4 commits into from Jan 31, 2019

Conversation

3 participants
@iLiviu
Copy link
Member

commented Jan 20, 2019

Fixes #1116
Dropbox and Googledrive backends first read the response as text and then tried to convert it back to raw value, however not all bytes can be encoded as text, and are lost during the conversion to text and back. Now, they request the response as ArrayBuffer and encode it to text and if they detect binary data, they return the unprocessed ArrayBuffer.
Also made a fix to shouldBeTreatedAsBinary function because it was treating white space characters (horizontal tab, vertical tab, CR, LF) as binary specific characters and added the Unicode replacement character (\uFFFD) as binary specific because the decoder inserts this character when it detects an invalid code sequence for the text encoding provided.

iLiviu added some commits Jan 19, 2019

fix binary data detection
exclude whitespace characters from binary specific characters and add unicode replacement character
fix get binary files for Dropbox and GoogleDrive #1116
Dropbox and Googledrive backends first read the response as text and then tried to convert it back to raw value, however not all bytes can be encoded as text, and are lost during the conversion to text and back. Now, they request the response as ArrayBuffer and encode it to text and if they detect binary data, they return the unprocessed ArrayBuffer.

moved getTextFromArrayBuffer from wireclient to util to be used by Dropbox and GoogleDrive

fixed tests for the new change.
remove unused readBinaryData from utils
readBinaryData is no longer used and should not be used anyway, as binary data should not be encoded as text in the first place.
fix indentation
eslint failed due to incorrect indentation
@galfert

This comment has been minimized.

Copy link
Member

commented Jan 23, 2019

Nice one. The code looks good to me and I confirmed it to be working.

Thanks a lot for this fix.

👍

@skddc

This comment has been minimized.

Copy link
Member

commented Jan 24, 2019

+1

@galfert galfert merged commit 0056679 into remotestorage:master Jan 31, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@galfert

This comment has been minimized.

Copy link
Member

commented Jan 31, 2019

Added a changelog entry for this in 53e7271.

@iLiviu iLiviu deleted the iLiviu:bugfix/1116-read_binary_file_dropbox_googledrive branch Feb 2, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.