-
Notifications
You must be signed in to change notification settings - Fork 6
Dat Module Wishlist #5
Comments
re: elasticsearch https://github.com/geopipes/elasticsearch-backend |
Some other storage possibilities (in no particular order, and I am not working on any of these):
|
You're very welcome, @maxogden. |
Here's one uploader for ElasticSearch based on the module by @missinglink: https://github.com/astro/dat-elasticsearch-upload |
Support the CKAN API, in particular through http://datahub.io (part of CKAN - Open Knowledge Foundation) - Latest docs here: http://docs.ckan.org/en/latest/api/index.html |
pouch-dat is a connector for pouchdb/couchdb and dat |
rethinkdb might be a good one to add as well |
+1 on CKAN datastore API support, or even a CKAN extension that uses Dat instead of CKAN's datastore. |
Neo4j might be another possibility. +1 for CKAN. |
https://github.com/coleww/s3-blobs I make no promises, but i shall try! |
Not sure if this fits here, but I just created vinyl-dat to read/write dat records in a gulp workflow. It's basically just a way to convert dat records into vinyl files. I think it still needs some work and possibly a better way to just let dat do conversions on contents (like choosing csv, json, etc...) |
firebase - I'll update if I start on this. |
I think this should work for redis as a backend https://www.npmjs.org/package/redisdown |
I am going to have a look at CKAN as an import/export target as well as a possible storage back-end. ckanDOWN: https://github.com/finnp/ckanDOWN |
some project specific api's; google, osm note that another potential collection of service-formats is here: dat-ecosystem-archive/datproject-discussions#5
Would https://www.npmjs.org/package/gdal be a usefull lib for dat to connect to a huge number of formats (http://gdal.org/ogr_formats.html)? |
@pvgenuchten that is awesome, thanks for linking it. I'm not sure what the best place to plug gdal into dat would be at the moment but I'll think about it. It's also great that the gdal module is using node-pre-gyp to make it easy to install cross-platform |
We'd be interested in throwing http://figshare.com/ in the ring here, also have some devs who could help out :) |
So this is a pouchdb want originally but applies to dat. Leveldown can be tricky to get working on all platforms and while pre-built ones ones are helpful there are always going to be weirder platform combinations. It would be great to have a pure js leveldown backend (for node) that was not restricted to memory. Being able to install dat or pouchdb with no c deps would be super useful. Medea seems the best of what is out there but has a requirement that keys be kept in memory which combined with the node.js hard 2GB mem limit means that there are hard limits on the datasets you'd be able to put into it. So really the thing that would be super helpful is a pure js nodejs db with spill to disk. |
Check out http://geogig.org/ "GeoGig is an open source tool that draws inspiration from Git, but adapts its core concepts to handle distributed versioning of geospatial data." |
Has many of the basic one not been covered? :) As seen here https://github.com/rvagg/node-levelup/wiki/Modules#storage Or am I am wrong? I am a little confused because how is this dat related, isn't is more of Leveldb related, that is needs more adapters? Or is there a good example on a plugin there is written? |
Just published dat-mongo to handle realtime updates from MongoDB with the |
@calvinmetcalf If you're looking to build a pure JS levelDB this might be a good place to start -- https://github.com/rvagg/leveljs |
Ooo I like, thanks
|
any interest to RDF / Linked Data support? I can work on a |
|
@wikier RDF / Marmotta here, please! We'll need it for http://transformap.co |
so @almereyda, should we create an issue to organize the work, define the goals, etc? |
@wikier Yes, of course! We'll be squatting the CouchDB Meetup in Berlin next week, where I'm also inviting @finnp right now, to precise the requirements of the TransforMap data integration pipeline. Please feel free to get started with your perspective, but I will only then be able to respond meaningfully. |
In the dawn of tomorrow, I'm stumbling over dat browser. |
I can't believe nobody's mentioned SQLite. So I'll do it. |
How about JSON, RDF, and Atom feed URLs? :-D |
Yeah, then I could transform Discourse' API into RSS :) Or we just plug dat On 7 June 2015 at 16:10, Benjamin Lupton notifications@github.com wrote:
|
Guide links in the issue are broken. |
links to the guides are still broken |
@maxogden, perhaps edit the original comment on this issue to use [ ] [x] checkboxes instead of bullets? |
+1 on CKAN |
This issue is to discuss ideas for dat modules that would be awesome to have. If you want to write a module leave a comment to notify others that you are working on it (preferably also leave a link to the repo).
I'll keep this post updated to link to all of the modules (until we have too many to fit here).
To learn more about dat modules check out the Importing data guide
If a module doesn't have a link w/ it it means nobody is working on it yet
Read about Import Methods to determine which type of module should be used for each case.
List
(not in any particular order)
In Progress/Done
Up for grabs
The text was updated successfully, but these errors were encountered: