Permalink
Switch branches/tags
Nothing to show
Find file
Fetching contributors…
Cannot retrieve contributors at this time
271 lines (191 sloc) 11.1 KB
::::: CURRENT/NEXT ::
- [FEATURE]: `Storage#get('post/*')` - get all documents of type 'post' (...using some kind of pagination if possible?)
- [FEATURE]: `Storage#types` - export all document types.
- [FEATURE]: `Storage#keys` - export all document keys.
- [FEATURE]: Count records by type: `Storage#count(<TYPE>)`
- Memory: `Object.keys`
- FileSystem: http://nodejs.org/docs/v0.3.1/api/fs.html#fs.readdir
- Memcache: https://gist.github.com/1365005 (HACK)
- Redis: http://redis.io/commands/keys
- MongoDB: http://docs.mongodb.org/manual/reference/method/db.collection.count
- ElasticSearch: http://www.elasticsearch.org/guide/reference/api/count.html
- AmazonS3: http://docs.amazonwebservices.com/AmazonS3/2006-03-01/API/RESTBucketGET.html (HACK - bad one)
- [FEATURE]: Clear records by type: `Storage#clear(<TYPE>)`
- Memory: `Object.keys`
- FileSystem: http://nodejs.org/docs/v0.3.1/api/fs.html#fs.readdir
- Memcache: https://gist.github.com/1365005 (HACK)
- Redis: http://redis.io/commands/keys
- MongoDB: http://docs.mongodb.org/manual/reference/method/db.collection.remove
- ElasticSearch: http://www.elasticsearch.org/guide/reference/api/delete-by-query.html
- AmazonS3:
http://docs.aws.amazon.com/AmazonS3/latest/API/multiobjectdeleteapi.html
http://docs.amazonwebservices.com/AmazonS3/2006-03-01/API/RESTBucketGET.html (HACK - bad one)
* [ADAPTER/FEATURE]: Generic client-side HTTP GUI: `node-document-gui`
- Example: https://github.com/tldrio/mongo-edit
- Example: https://github.com/hij1nx/levelweb
::::: HIGH ::
- [FEATURE/API/PERFORMANCE]: Enhance `Collection` DB operations to not run as single operations, rather merge them and use bulk DB operation.
1. Operations
Post.set(1, data, callback) // Post.storage.set('Post', 1, data, callback)
Article.set(1, data, callback) // Article.storage.set('Article', 1, data, callback)
2. Convert to Bulk Operation
Document([Post, Article]).set(1, data, callback)
- [ENHANCEMENT/REVIEW/DISCUSS]: Event hooks both before and after (...and middle?):
- Document.Model.prototype.emit('save:before') - before (first)
- Document.Model.prototype.emit('save:each') - for each item (if many)
- Document.Model.prototype.emit('save:after') - after (last)
- [FEATURE]: `#monitor` - shortcut for listening on DB operations (like Redis MONITOR).
- [FEATURE]: Review meta fields: `_updated_at`, `_created_at`. Adapter and hook into events, e.g. "before save", etc.? Middleware API?
- Example:
var TimestampsMiddleware = require('node-document-middleware-timestamps');
Post.use(TimestampsMiddleware);
post.use(TimestampsMiddleware);
[Post, Article].use(TimestampsMiddleware);
[post, article].use(TimestampsMiddleware);
- Original *deprecated* idea - middleware could do this without bloating the API:
// Will be saved as `_created_at` (on first write) - if meta prefix is '_'.
Post.meta('created_at', false, function() {
// Access to instance (doc) and klass (doc.klass) if needed.
return new Date();
});
// Will be saved as `_fingerprint` (on every write) - if meta prefix is '_'.
Post.meta('fingerprint', true, function() {
return [this.klass.type, encodeURIComponent(this.url)].join('-');
});
- [FEATURE]: Connection pool for storage connections - not only per instance, per storage adapter
- https://github.com/coopernurse/node-pool
- Example:
SomeStorage.pool = poolModule.Pool({...});
SomeStorage.prototype.__defineGetter__('pool', function() { return this.klass.pool; });
SomeStorage.aquire(callback);
(new SomeStorage).aquire(callback);
- Issue/Review: Needs to close connections manually to make this behave as intended. Make connection pools optional?
- [BUG]: See `test/index.js` comment. Probably related to the swallowed errors issue.
- [FEATURE/API]: Extend the Collection API with "Perform collection of operations". Possible enhancement: Merge it into bulk-request under the hood.
- Example: Merge different operations on different document types into single bulk operation.
[(new Post).save, (new Article).fetch, (new Post).fetch].exec(function(err) {
assert.typeOf (this, 'array');
assert.instanceOf (this[0], Post);
assert.instanceOf (this[1], Article);
assert.instanceOf (this[2], Post);
});
- [ISSUE]: Reserved attribute keywords (e.g. CouchDB); meta attribute field prefix should be overrideable by storage adapter ("mapper light").
- [REFACTOR]: Make `Differ#diff` and `Validator#validate` synchronous? - worst case drop support for `Amanda` or "fake it".
- [FEATURE]: Support "expires" for documents (Storage)
- Memory: Invalidate on fetch
- FileSystem: Invalidate on fetch
- Redis: http://redis.io/commands/expire
- MongoDB: http://docs.mongodb.org/manual/tutorial/expire-data
- ElasticSearch: http://www.elasticsearch.org/guide/reference/mapping/ttl-field.html
- Memcache: http://code.google.com/p/memcached/wiki/NewCommands#Standard_Protocol
- AmazonS3: http://aws.typepad.com/aws/2011/12/amazon-s3-object-expiration.html (i.e. note really => invalidate on fetch)
* Always set `_expires` property.
- [FEATURE]: Migrate from one DB to another via storage instances: `Storage#export(storage || Stream, ...)` + `Storage#import(storage || Stream, ...)`, and maybe Storage#sync(storage, ...
- Inherit Stream?
- http://maxogden.com/node-streams
- http://blog.nodejs.org/2012/12/20/streams2/
- Example:
function copy (srcdb, dstdb, callback) {
srcdb.readStream().pipe(dstdb.writeStream()).on('close', callback)
}
- [API]: Subclassing for simpler use of `instanceof` keyword checking:
EventEmitter
Adapter
Validator
Differ
Stream
Storage
::::: LOW ::
- [FEATURE]: Seeds (`node-document-seed`) - simple seed generator (interface)
- https://github.com/acatl/datafixture.js
- [DOC]: In each storage README; list support table, connection-URL/key mapping table, connection URL format, etc.
- [FEATURE]: "Mappers" (`node-document-mapper`) - define how data should map between different databases. Major feature, will require a quite lot of work.
- [FEATURE]: Generic CLI for interacting with/between storages: `node-document-cli` - requires `Storage` to be ported to `Stream` first.
- [ENHANCEMENT]: Make `Document.storage` a property, to later support more advanced shortcuts like in the constructor:
Post.storage = 'file:///app'
- [ADAPTER]: Google Spreadsheet
- https://github.com/samcday/node-google-spreadsheets
- [ADAPTER]: OrientDB
- https://github.com/gabipetrovay/node-orientdb
- http://www.orientdb.org
- [ADAPTER]: ArangoDB
- https://github.com/kaerus/arango-client
- http://www.arangodb.org
- [ADAPTER]: AmazonSimpleDB
- https://github.com/SaltwaterC/aws2js/wiki/SDB-Client
- https://github.com/rjrodger/simpledb
- http://aws.amazon.com/simpledb/
- [ADAPTER]: AmazonElastiCache
- https://github.com/SaltwaterC/aws2js/wiki/ElastiCache-client
- https://console.aws.amazon.com/elasticache/home?region=us-east-1
- [ADAPTER]: AmazonDynamoDB
- https://github.com/SaltwaterC/aws2js/wiki/DynamoDB-Client
- https://github.com/jed/dynamo
- https://console.aws.amazon.com/dynamodb/home?region=us-east-1
- [ADAPTER]: Dropbox
- https://github.com/sintaxi/node-dbox
- [ADAPTER]: RavenDB
- https://github.com/tchype/node-ravendb
- [ADAPTER]: HBase
- https://github.com/wdavidw/node-hbase
- [ADAPTER]: Cassandra
- https://github.com/racker/node-cassandra-client
- https://github.com/simplereach/helenus
- [ADAPTER]: Google Cloud Datastore
- https://github.com/GoogleCloudPlatform/google-cloud-datastore
- [ADAPTER]: Google Storage
- https://github.com/nodejitsu/pkgcloud#storage
- [ADAPTER]: Rackspace Storage
- https://github.com/nodejitsu/pkgcloud#storage
- [ADAPTER]: Azure Storage
- https://github.com/nodejitsu/pkgcloud#storage
- [REFACTOR]: Switch to lower-lever driver for `AmazonS3` storage adapter
- https://github.com/SaltwaterC/aws2js/wiki/S3-Client
- [FEATURE]: "Middleware" API for storages, and such - e.g. `Post.use(storage_1).use(storage_2)` or `Post.storage.use(storage_1, storage_2)`.
- [REFACTOR]: var MyStorageAdapter = Storage.adapter('MyStorageAdapter', function() { /* ... */ })
- [ADAPTER/ENHANCEMENT]: Support SCP in `File` adapter, e.g. `fs://localhost:1234/tmp/{db}-{env}`
- [REFACTOR/HALTED]: Rewrite `Memcache` storage to driver that support SASL (auth) - tried MemJS but it contains serious race-condition bug. =S
- [REFACTOR]: Rewrite `ElasticSearch` storage using vanilla HTTP agent.
- [FEATURE/DISCUSS]: Support "versioning" for documents (Storage).
* Always set `_version` property.
- [FEATURE/DISCUSS]: Wherever it makes sense; `Storage#index` + `Storage#unindex`
- [DOC]: Proper function documentation (JSDoc-style).
- http://en.wikipedia.org/wiki/JSDoc
- https://developers.google.com/closure/compiler/docs/js-for-compiler
- [FEATURE]: Support multiple endpoints - where applicable - for all storage adapters/clients.
- [REVIEW]: Support advanced URL convention (multiple hosts, options, etc.)
- http://docs.mongodb.org/manual/reference/connection-string
- [ADAPTER]: SQLite3
- https://github.com/developmentseed/node-sqlite3
- [ADAPTER]: PostgreSQL (HStore)
- https://github.com/brianc/node-postgres
- https://github.com/nonuby/node-postgres-hstore
- [ADAPTER]: MySQL
- https://github.com/felixge/node-mysql
- http://www.igvita.com/2010/03/01/schema-free-mysql-vs-nosql
- https://github.com/igrigorik/em-proxy/blob/master/examples/schemaless-mysql/mysql_interceptor.rb
::::: MAYBE ::
- [ADAPTER/EXPERIMENT]: `Serializer` adapters support (...for storages where it makes sense)
- JSON - DONE
- BSON - DONE
- MsgPack - DONE
- [ADAPTER/EXPERIMENT]: `Compressor` adapter support (...for storages where it makes sense)
- Deflate: DONE
- LZF - DONE
- Snappy - DONE
- ZIP/TAR - https://github.com/ctalkington/node-archiver
- [FEATURE]: Detect and handle lost connections in a reliable and automatic way. Fallback to other storage?
- [ADAPTER]: CouchBase
- https://github.com/couchbase/couchnode
- https://github.com/membase/manifest
- https://github.com/clojurewerkz/spyglass/blob/master/.travis.yml
- [ADAPTER]: BerkelyDB
- https://github.com/mcavage/node-bdb
- OS X: './build/default/*' => './build/Release/*' (fork it)
- https://npmjs.org/package/bindings - to fix the above problem
- NOTE: Got it working standalone, but fails when installed via `npm install`
- [ADAPTER]: HyperDex
- https://github.com/rescrv/HyperDex/tree/master/hyperclient/nodejs
- [ADAPTER]: REST (storage adapter)
- https://github.com/danwrong/restler
- [REVIEW]: Get/Set/Del nested attribute values in some convenient way.