Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dropbox-Api-Arg needs encoding in the headers #54

Closed
ncw opened this issue Oct 10, 2019 · 1 comment · Fixed by #56
Closed

Dropbox-Api-Arg needs encoding in the headers #54

ncw opened this issue Oct 10, 2019 · 1 comment · Fixed by #56

Comments

@ncw
Copy link
Contributor

ncw commented Oct 10, 2019

An rclone user reported file name corruptions when uploading things to Dropbox with rclone - see rclone/rclone#3609 for background.

Greg K in the Dropbox developer forum was kind enough to explain the problem: https://www.dropboxforum.com/t5/API-Support-Feedback/Possible-bug-corrupting-file-names-with-certain-unicode-code/m-p/370538

In summary this was a result of:

  1. A change of code at Dropbox, now rolled back
  2. The Dropbox-Api-Arg header not being properly encoded

When rclone does an upload using the Upload method from the SDK an HTTP transaction like this gets sent:

2019/10/10 12:40:16 DEBUG : HTTP REQUEST (req 0xc000463900)
2019/10/10 12:40:16 DEBUG : POST /2/files/upload HTTP/1.1
Host: content.dropboxapi.com
User-Agent: rclone/v1.49.5
Transfer-Encoding: chunked
Authorization: XXXX
Content-Type: application/octet-stream
Dropbox-Api-Arg: {"path":"/tdir/Русский.txt","mode":{".tag":"overwrite"},"autorename":false,"client_modified":"2000-01-01T00:00:00Z","mute":false,"strict_conflict":false}
Accept-Encoding: gzip

0

Note how the UTF-8 characters are not encoded in the Dropbox-Api-Arg header.

In the documentation it states

If you use the Dropbox-API-Arg header, you need to make it "HTTP header safe". This means using JSON-style "\uXXXX" escape codes for the character 0x7F and all non-ASCII characters.

This is clearly not being done, hence the problem.

For example here the JSON encoded arguments are put straight into the header as far as I can see.

func (dbx *apiImpl) Upload(arg *CommitInfo, content io.Reader) (res *FileMetadata, err error) {
cli := dbx.Client
dbx.Config.LogDebug("arg: %v", arg)
b, err := json.Marshal(arg)
if err != nil {
return
}
headers := map[string]string{
"Content-Type": "application/octet-stream",
"Dropbox-API-Arg": string(b),
}
if dbx.Config.AsMemberID != "" {

ncw added a commit to ncw/dropbox-sdk-go-unofficial that referenced this issue Oct 22, 2019
As part of investigating a filename corruption issue, it was noted
that this SDK doesn't encode the Dropbox-API-Arg to make it "HTTP
header safe" according to the specification here:
https://www.dropbox.com/developers/reference/json-encoding

This fixes the encoding and updates the generator to apply it
everywhere.

Fixes dropbox#54
@ncw
Copy link
Contributor Author

ncw commented Oct 22, 2019

I've made a pull request #56 to fix this - PTAL :-)

ncw added a commit to ncw/dropbox-sdk-go-unofficial that referenced this issue Nov 15, 2019
As part of investigating a filename corruption issue, it was noted
that this SDK doesn't encode the Dropbox-API-Arg to make it "HTTP
header safe" according to the specification here:
https://www.dropbox.com/developers/reference/json-encoding

This fixes the encoding and updates the generator to apply it
everywhere.

Fixes dropbox#54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant