This repository has been archived by the owner on Aug 12, 2020. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 20
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #8 from noffle/readme
README and API improvements
- Loading branch information
Showing
7 changed files
with
110 additions
and
169 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,12 +1,90 @@ | ||
IPFS Data Importing | ||
=================== | ||
|
||
> Import data into an IPFS DAG Service. | ||
[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://ipn.io) | ||
[![](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs) | ||
[![Build Status](https://travis-ci.org/ipfs/js-ipfs-data-importing.svg?style=flat-square)](https://travis-ci.org/ipfs/js-ipfs-data-importing) | ||
![](https://img.shields.io/badge/coverage-%3F-yellow.svg?style=flat-square) | ||
[![Dependency Status](https://david-dm.org/ipfs/js-ipfs-data-importing.svg?style=flat-square)](https://david-dm.org/ipfs/js-ipfs-data-importing) | ||
[![js-standard-style](https://img.shields.io/badge/code%20style-standard-brightgreen.svg?style=flat-square)](https://github.com/feross/standard) | ||
|
||
> JavaScript implementation of the layout and chunking mechanisms used by IPFS | ||
## Example | ||
|
||
Let's create a little directory to import: | ||
```sh | ||
$ cd /tmp | ||
$ mkdir foo | ||
$ echo 'hello' > foo/bar | ||
$ echo 'warld' > foo/quux | ||
``` | ||
|
||
And write the importing logic: | ||
```js | ||
// Dependencies to create a DAG Service (where the dir will be imported into) | ||
var memStore = require('abstract-blob-store') | ||
var ipfsRepo = require('ipfs-repo') | ||
var ipfsBlocks = require('ipfs-blocks') | ||
var ipfsMerkleDag = require('ipfs-merkle-dag') | ||
|
||
var repo = new ipfsRepo('', { stores: memStore }) | ||
var blocks = new ipfsBlocks.BlockService(repo) | ||
var dag = new ipfsMerkleDag.DAGService(blocks) | ||
|
||
|
||
var ipfsData = require('ipfs-data-importing') | ||
|
||
// Import /tmp/foo | ||
ipfsData.import('/tmp/foo', dag, { | ||
recursive: true | ||
}, done) | ||
|
||
// A root DAG Node is received upon completion | ||
function done (err, rootStat) { | ||
if (err) { throw err } | ||
console.log(rootStat) | ||
} | ||
``` | ||
|
||
When run, the stat of root DAG Node is outputted: | ||
|
||
``` | ||
{ Hash: <Buffer 12 20 bd e2 2b 57 3f 6f bd 7c cc 5a 11 7f 28 6c a2 9a 9f c0 90 e1 d4 16 d0 5f 42 81 ec 0c 2a 7f 7f 93>, | ||
Size: 59843, | ||
Name: 'foo' } | ||
``` | ||
|
||
## API | ||
|
||
```js | ||
var importer = require('ipfs-data-importing') | ||
``` | ||
|
||
### importer.import(target, dagService, opts, cb) | ||
|
||
`target` can be a `string`, `Buffer`, or `Stream`. When it's a string, the file | ||
or directory structure rooted on the filesystem at `target` is imported, with | ||
the hierarchy preserved. If a Buffer or Stream, a single DAG node will be | ||
imported representing the buffer or stream's contents. | ||
|
||
Uses the [DAG Service](https://github.com/vijayee/js-ipfs-merkle-dag/) instance | ||
`dagService`. Accepts the following `opts`: | ||
|
||
- `recursive`: whether to recurse into directories. Defaults to `false`. | ||
|
||
Calls the callback `cb(err, stat)` on completion or error, where `stat` is an | ||
object with the `Hash`, `Size`, and `Name` of the root | ||
[`DAGNode`](https://github.com/vijayee/js-ipfs-merkle-dag/). | ||
|
||
## install | ||
|
||
With [npm](https://npmjs.org/) installed, run | ||
|
||
``` | ||
$ npm install ipfs-data-importing | ||
``` | ||
|
||
## license | ||
|
||
ISC |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,45 +1,5 @@ | ||
var through2 = require('through2') | ||
var chunker = require('block-stream2') | ||
|
||
exports = module.exports = FixedSizeChunker | ||
|
||
// The difference of this chunker compared to other fixed size chunkers | ||
// available, is that it doesn't add padding the last chunk | ||
|
||
function FixedSizeChunker (size) { | ||
var stream = through2(transform, flush) | ||
|
||
var buf = new Buffer(0) | ||
|
||
function transform (chunk, enc, cb) { | ||
var that = this | ||
|
||
buf = Buffer.concat([buf, chunk]) | ||
|
||
if (buf.length >= size) { | ||
slice() | ||
} | ||
|
||
function slice () { | ||
var chunk = new Buffer(size, 'binary') | ||
var newBuf = new Buffer(buf.length - size, 'binary') | ||
buf.copy(chunk, 0, 0, size) | ||
buf.copy(newBuf, 0, size, buf.length) | ||
buf = newBuf | ||
that.push(chunk) | ||
|
||
if (buf.length >= size) { | ||
return slice() | ||
} | ||
} | ||
|
||
cb() | ||
} | ||
|
||
function flush (cb) { | ||
// last chunk | ||
this.push(buf) | ||
cb() | ||
} | ||
|
||
return stream | ||
exports = module.exports = function (size) { | ||
return chunker({ size: size, zeroPadding: false }) | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.