Skip to content

New S3 Store and some fixes
Compare
Choose a tag to compare
@shyamd shyamd released this 31 Mar 19:32
v0.5.0
2ee043f
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

We're starting to expand functionality. One of the keys for Maggma will be connecting to a lot of different data sources but leveraging the powerfull querying in MongoDB. The GridFS concept of an index + bucket storage patterns really well to other cloud storage platforms that we hope to support.

Changes:

  • Added an AmazonS3 Store
  • Added from_collection in mongostore to instantiate a MongoStore from a PyMongo Collection object
    Warning: This object will not serialize and deserialize properly so don't expect this to work with MPI building or saving to files
  • Fixed distinct and groupby to be applicable to all stores.
Assets 2