New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use multipart/mixed for attachments #2832

Closed
nolanlawson opened this Issue Oct 5, 2014 · 9 comments

Comments

Projects
None yet
4 participants
@nolanlawson
Copy link
Member

nolanlawson commented Oct 5, 2014

multipart/mixed is more efficient for sending attachments around, and plus Sync Gateway only supports that format (#2676 and #2681).

nolanlawson added a commit that referenced this issue Oct 5, 2014

@jchris

This comment has been minimized.

Copy link
Contributor

jchris commented Jan 9, 2015

I think this is the place to discuss which implementation to test and how to test it. I think Apache CouchDB will use this if we support it so we maybe it's an easy first place to work on Sync Gateway interop.

I'd like to learn how to interact with the test automation, maybe this is a good chance to do it?

@nolanlawson

This comment has been minimized.

Copy link
Member

nolanlawson commented Jan 9, 2015

Yep, currently non-org members won't have their PRs tested in Saucelabs, but all the non-Saucelabs tests will be run. Probably the most important thing here is to verify support for PouchDB Server as well as CouchDB 1.6, so it should be fine for now. After that, we can run the Sauce tests to verify the proprietary browsers.

@jchris

This comment has been minimized.

Copy link
Contributor

jchris commented Jan 9, 2015

@nolanlawson

This comment has been minimized.

Copy link
Member

nolanlawson commented Jan 9, 2015

Yep, that looks right. TBH can't remember why I stopped working on it.

@nolanlawson

This comment has been minimized.

Copy link
Member

nolanlawson commented Jan 9, 2015

jchris pushed a commit to jchris/pouchdb that referenced this issue Jan 12, 2015

@gabrielnau

This comment has been minimized.

Copy link
Contributor

gabrielnau commented Jan 14, 2015

Sorry for the late reply, I didn't see the Github notification.
Yes indeed, I just rebased it against master.

@nolanlawson

This comment has been minimized.

Copy link
Member

nolanlawson commented May 24, 2015

Started looking into multipart again. It's gonna be a mess.

First off, it appears CouchDB returns the content as gzip-encoded. Meaning I guess we'll need to bundle a gzip decoder with PouchDB so that we can actually decode that content?? Unless there's a good Blob-related hack we can use to make the browser do it (or something...).

Second, I'm stuck right now trying to actually decode multipart/mixed. If anybody can figure out how I'm supposed to decode a string like 'H4sIAAAAAAAAAwMAAAAAAAAAAAA=\n' (which should be a base64-encoded, gzip-encoded string "This is a base64 encoded text"), then I'd be happy to hear how.

If anyone is curious, here is an example of a full multipart/mixed response from CouchDB, completely base64 encoded: https://gist.github.com/nolanlawson/5ecdce6ea62ecb7ded4c

@nolanlawson

This comment has been minimized.

Copy link
Member

nolanlawson commented May 24, 2015

So in retrospect, the only reason we care about this is for CSG compatibility.

For CouchDB, the benefit is increased efficiency when replicating attachments. We lose those efficiency gains (imo) if we need to bundle a multipart parser and a gzip decoder with PouchDB.

We could solve this problem a lot more simply if we just avoided GETing a doc with attachments=true in the replicator, and instead just fetched each attachment in a separate request. All the modern browsers we support can do that for us, and give us the data in a nice simple arraybuffer format.

nolanlawson added a commit that referenced this issue May 24, 2015

nolanlawson added a commit that referenced this issue May 24, 2015

nolanlawson added a commit that referenced this issue May 24, 2015

(#2832) - implement broken multipart/related
This commit doesn't work. It's just to demonstrate
a CouchDB bug.

nolanlawson added a commit that referenced this issue May 24, 2015

(#2832) - implement multipart/related
If a document contains non-stub attachments,
then upload them as multipart-related instead
of application/json.

nolanlawson added a commit that referenced this issue May 25, 2015

(#2832) - implement multipart/related
If a document contains non-stub attachments,
then upload them as multipart-related instead
of application/json.

nolanlawson added a commit that referenced this issue May 25, 2015

(#2832) - implement multipart/related
If a document contains non-stub attachments,
then upload them as multipart-related instead
of application/json.

nolanlawson added a commit that referenced this issue May 25, 2015

(#2832) - implement multipart/related
If a document contains non-stub attachments,
then upload them as multipart-related instead
of application/json.

nolanlawson added a commit that referenced this issue May 27, 2015

nolanlawson added a commit that referenced this issue May 27, 2015

(#2832) - implement multipart/related
If a document contains non-stub attachments,
then upload them as multipart-related instead
of application/json.

nolanlawson added a commit that referenced this issue May 29, 2015

nolanlawson added a commit that referenced this issue May 29, 2015

(#2832) - implement multipart/related
If a document contains non-stub attachments,
then upload them as multipart-related instead
of application/json.

nolanlawson added a commit that referenced this issue May 30, 2015

nolanlawson added a commit that referenced this issue May 30, 2015

(#2832) - implement multipart/related
If a document contains non-stub attachments,
then upload them as multipart-related instead
of application/json.

nolanlawson added a commit that referenced this issue May 30, 2015

(#2832) - implement multipart/related
If a document contains non-stub attachments,
then upload them as multipart-related instead
of application/json.

nolanlawson added a commit that referenced this issue May 31, 2015

(#2832) - implement multipart/related
If a document contains non-stub attachments,
then upload them as multipart-related instead
of application/json.
@daleharvey

This comment has been minimized.

Copy link
Member

daleharvey commented May 31, 2015

Fixed by c6059d2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment