New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

100 use blobs for attachments #502

Merged
merged 25 commits into from Mar 1, 2013

Conversation

Projects
None yet
2 participants
@jo
Copy link
Member

jo commented Feb 28, 2013

All tests run through. Jeah!

Initial issue: #100

  • Special handling for Chrome, which does not support Blobs (use base64 encoded strings)
  • Use Buffers for node
  • Introduce attachment-binary-store for leveldb
@daleharvey

This comment has been minimized.

Copy link
Member

daleharvey commented Mar 1, 2013

Wow epic commit, huge props getting this working across adapters and browsers, not an easy thing at all.

looks like the node tests are dying though

Could I ask for one more bit to this, updated documentation to cover the various ways in which you can store and read attachments, the files are in https://github.com/daleharvey/pouchdb/blob/master/docs/api.md and you can test them with

~/src/pouchdb/docs$ jekyll --server --auto

and visiting 127.0.0.1:4000

@jo

This comment has been minimized.

Copy link
Member

jo commented Mar 1, 2013

Sure, the docs are on my todo list for today, it was too late yesterday ;)

Error: Value argument cannot be an empty Buffer

Will see if I can fix this, don't know why this wasn't failing local.

@jo

This comment has been minimized.

Copy link
Member

jo commented Mar 1, 2013

Had an older levelup version locally, which allowed to store empty values.

Green Travis Yippie!

@jo

This comment has been minimized.

Copy link
Member

jo commented Mar 1, 2013

rebased to master and force pushed

jo added some commits Feb 21, 2013

(#100) - Use Blobs for attachments: IndexedDB
Add Blob storage support for IndexedDB.
(#100) - Use Blobs for attachments: HTTP
Add Blob storage support for HTTP adapter.
(#100) - Fix empty attachment blobs: HTTP
Chrome reads `new Blob([null])` as "null", whereas FF reads it as "".
(#100) - Use Blobs: normalize FileReader
Always use `reader.onloadend` and access data via `this.result`.
(#100) - Use Blobs (IndexedDB): Improvements
Improve `decode` option behaviour in `getAttachment`
and prevent base64 detour for inline attachments in Chrome.
(#100) - Use Blobs for attachments: WebSQL
Add Blob storage support for WebSQL.
(#100) - Use Blobs for attachments: add Test
Add a test saving an attachment together with an invalid doc.
(#100) - Use Blobs: Fix invalid docs
Fix error when trying to save invalid docs.
(#100) - Use Blobs for attachments: update Test
Update test saving an attachment together with an invalid doc.
Now receives error.
(#100) - Blobs: abstract Blob creation and reading
introduce `makeBlob` and `readBlob` functions in test.utils.js and use
them in test.attachments.js to be able to switch blob handling inside
node context.
(#100) - Blobs: expose blob test utils
expose `makeBlob` and `readBlob` test utils for node context and use
appropriate Buffers.
(#100) - Blobs: adapt support to node ajax version
set encoding to null which forces request to return a buffer.
(#100) - Blobs: LevelDB support
add support for blob attachments for LevelDB using buffers.
(#100) - Blobs: adapt pouch.adapter
to use encode instead of decode for attachments.
Also remove attachments: true for update attachment.
(#100) - Blobs: test content type
Test whether the correct content type was stored.
(#100) - Blobs: fix content type
and use explicit type for tests.
(#100) - Blobs: introduce binary store
Use a seperate store for attachment blobs.
This should significantly improve performance for big attachments as
they do not have to be json stringified and encoded on every read.
Also, this enables attachment streaming possibilities.
(#100) - Blobs: respect processData (node http)
if set to false data is not stringified before send to server.
(#100) - Blobs: fix empty attachments (leveldb)
Current levelup (0.6) does not allow to store empty buffers or strings.
Fix this by simply not storing empty binaries.
@@ -304,8 +304,8 @@ var HttpPouch = function(opts, callback) {
var parts = id.split('/');
if ((parts.length > 1 && parts[0] !== '_design' && parts[0] !== '_local') ||
(parts.length > 2 && parts[0] === '_design' && parts[0] !== '_local')) {
// Nothing is expected back from the server
options.json = false;
// Binariy is expected back from the server

This comment has been minimized.

@daleharvey

daleharvey Mar 1, 2013

Member

typo :)

@daleharvey

This comment has been minimized.

Copy link
Member

daleharvey commented Mar 1, 2013

\o/

daleharvey added a commit that referenced this pull request Mar 1, 2013

@daleharvey daleharvey merged commit e051a9a into master Mar 1, 2013

1 check passed

default The Travis build passed
Details

@daleharvey daleharvey deleted the 100-use-blobs-for-attachments branch Mar 1, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment