New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

track: `Files API + MFS` #60

Closed
diasdavid opened this Issue Jan 26, 2016 · 11 comments

Comments

Projects
None yet
3 participants
@diasdavid
Member

diasdavid commented Jan 26, 2016

The files API offers an interface to work with unixfs, a Merklelized Data Structure that represents a typical FileSystem used in a Unix OS. with files, dirs and symlinks. This API is what enables IPFS to add and retrieve files without having to ask the users to convert their data into a graph data structure.

To learn more about the Files API, you can

Implementation roadmap

  • Files API spec ipfs/specs#88
  • Unixfs Spec
  • jsipfs files add (current ipfs add)
  • jsipfs files cat (current ipfs cat)
    • cat subfiles
  • jsipfs files get (current ipfs get)
  • jsipfs files mv - Move files.
  • jsipfs files cp - Copy files into mfs.
  • jsipfs files ls - List directories.
  • jsipfs files mkdir - Make directories.
  • jsipfs files stat - Display file status.
  • jsipfs files rm ... - Remove a file.
  • jsipfs files read - Read a file in a given mfs.
  • jsipfs files write - Write to a mutable file in a given filesystem.

@diasdavid diasdavid self-assigned this Jan 26, 2016

@diasdavid diasdavid changed the title from feature: files to files Feb 23, 2016

@diasdavid

This comment has been minimized.

Show comment
Hide comment
@diasdavid

diasdavid Feb 23, 2016

Member

Updated this issue

Member

diasdavid commented Feb 23, 2016

Updated this issue

@nginnever

This comment has been minimized.

Show comment
Hide comment
@nginnever

nginnever Mar 1, 2016

Member

jsipfs files get

Taking a note from @noffle and a stab at jsipfs files get

Relevant go-ipfs links:
https://github.com/ipfs/go-ipfs/blob/master/core/commands/get.go

ARGUMENTS:

<ipfs-path> - The path to the IPFS object(s) to be outputted

OPTIONS:

-o, --output            string - The path where output should be stored
-a, --archive           bool   - Output a TAR archive
-C, --compress          bool   - Compress the output with GZIP compression
-l, --compression-level int    - The level of compression (1-9)

STEPS

1. Get the gzip compression options, bail if err (TODO: Look into js gzip modules) - return NoCompression if compress bool is false - return DefaultCompression if compress bool is true but compress level is undefined - return NoCompression and error if bool is true and compress level is not between 1-9 - return compress level if none of the above conditions are met 2. Get the node from the command request context. 3. Get the path from the argument . - TODO: List what happens here 4. Resolve the path, bail if err 5. Set archive to the option --archive provided. 6. DagArchive (this is equivalent to `ipfs getdag $hash | maybe_tar | maybe_gzip`) - TODO: list what happens here 7. create a readable stream of the dagnode 8. Check the stream bail if error 9. Check the path option. If it does not exist, defaut path to the multihash 10. Get the compression level from options 11. Get the archive boolean from options 12. Pass the readable stream and path to a write() function - If archive or compression options != gzip.NoCompress then pass to writeArchive() function - else pass the stream and path to writeExtracted() function - **If writeArchive()** - check archive is set to true - if it is, check if the ending of the path is `path + '.tar'` or `path + '.tar.gz'` - if it does not then add .tar to path name - adjust path name to .gz if compression level != gzip.NoCompress - create the file on disk - print to console `Saving archive to $path` - show a progress bar - **if writeExtracted()** - print to console `Saving file(s) to $path` - show progress bar - TODO: figure out what extractor is
Member

nginnever commented Mar 1, 2016

jsipfs files get

Taking a note from @noffle and a stab at jsipfs files get

Relevant go-ipfs links:
https://github.com/ipfs/go-ipfs/blob/master/core/commands/get.go

ARGUMENTS:

<ipfs-path> - The path to the IPFS object(s) to be outputted

OPTIONS:

-o, --output            string - The path where output should be stored
-a, --archive           bool   - Output a TAR archive
-C, --compress          bool   - Compress the output with GZIP compression
-l, --compression-level int    - The level of compression (1-9)

STEPS

1. Get the gzip compression options, bail if err (TODO: Look into js gzip modules) - return NoCompression if compress bool is false - return DefaultCompression if compress bool is true but compress level is undefined - return NoCompression and error if bool is true and compress level is not between 1-9 - return compress level if none of the above conditions are met 2. Get the node from the command request context. 3. Get the path from the argument . - TODO: List what happens here 4. Resolve the path, bail if err 5. Set archive to the option --archive provided. 6. DagArchive (this is equivalent to `ipfs getdag $hash | maybe_tar | maybe_gzip`) - TODO: list what happens here 7. create a readable stream of the dagnode 8. Check the stream bail if error 9. Check the path option. If it does not exist, defaut path to the multihash 10. Get the compression level from options 11. Get the archive boolean from options 12. Pass the readable stream and path to a write() function - If archive or compression options != gzip.NoCompress then pass to writeArchive() function - else pass the stream and path to writeExtracted() function - **If writeArchive()** - check archive is set to true - if it is, check if the ending of the path is `path + '.tar'` or `path + '.tar.gz'` - if it does not then add .tar to path name - adjust path name to .gz if compression level != gzip.NoCompress - create the file on disk - print to console `Saving archive to $path` - show a progress bar - **if writeExtracted()** - print to console `Saving file(s) to $path` - show progress bar - TODO: figure out what extractor is
@nginnever

This comment has been minimized.

Show comment
Hide comment
@nginnever

nginnever Mar 1, 2016

Member

jsipfs files cat

A look at jsipfs files cat

Relevant go-ipfs links:
https://github.com/ipfs/go-ipfs/blob/master/core/commands/cat.go
https://github.com/ipfs/go-ipfs/blob/master/core/coreunix/cat.go

ARGUMENTS:

<ipfs-path>... - The path to the IPFS object(s) to be outputted

DESCRIPTION:

Retrieves the object named by <ipfs-or-ipns-path> and outputs the data
it contains.

STEPS

1. Get the node from the command request context. 2. If node.onlineMode = false, setupOfflineRouting() - TODO: List setupOfflineRouting steps 3. call cat() function passing in the context, the node, and the path(s) from the arguments - create read streams array for each path provided - for each path, call coreunix.Cat(ctx, node, fpath) - resolve the dagnode - return a readable stream - append the stream from resolve to the read streams array - collect the total length by adding each buffer length returned from resolve - return the readers array, length, and null or error 4. Do a conditional GC 5. set the length in the response 6. combine the readers streams into one read stream 7. check if the length < the smallest progress bar size 8. create the progress bar if not 9. set the res to the read stream.
Member

nginnever commented Mar 1, 2016

jsipfs files cat

A look at jsipfs files cat

Relevant go-ipfs links:
https://github.com/ipfs/go-ipfs/blob/master/core/commands/cat.go
https://github.com/ipfs/go-ipfs/blob/master/core/coreunix/cat.go

ARGUMENTS:

<ipfs-path>... - The path to the IPFS object(s) to be outputted

DESCRIPTION:

Retrieves the object named by <ipfs-or-ipns-path> and outputs the data
it contains.

STEPS

1. Get the node from the command request context. 2. If node.onlineMode = false, setupOfflineRouting() - TODO: List setupOfflineRouting steps 3. call cat() function passing in the context, the node, and the path(s) from the arguments - create read streams array for each path provided - for each path, call coreunix.Cat(ctx, node, fpath) - resolve the dagnode - return a readable stream - append the stream from resolve to the read streams array - collect the total length by adding each buffer length returned from resolve - return the readers array, length, and null or error 4. Do a conditional GC 5. set the length in the response 6. combine the readers streams into one read stream 7. check if the length < the smallest progress bar size 8. create the progress bar if not 9. set the res to the read stream.

@diasdavid diasdavid changed the title from files to files / add Mar 21, 2016

@diasdavid

This comment has been minimized.

Show comment
Hide comment
@diasdavid

diasdavid Mar 21, 2016

Member

merged #67

Member

diasdavid commented Mar 21, 2016

merged #67

@diasdavid diasdavid removed their assignment Mar 21, 2016

@diasdavid diasdavid changed the title from files / add to files API Mar 30, 2016

@diasdavid diasdavid modified the milestone: milestone 1 - js-ipfs on the browser Apr 4, 2016

@diasdavid

This comment has been minimized.

Show comment
Hide comment
@diasdavid

diasdavid Apr 22, 2016

Member

@nginnever I know you are working on this. Can you make a PR tagged with WIP from a branch under this project?

Member

diasdavid commented Apr 22, 2016

@nginnever I know you are working on this. Can you make a PR tagged with WIP from a branch under this project?

@diasdavid

This comment has been minimized.

Show comment
Hide comment
@diasdavid
Member

diasdavid commented Apr 22, 2016

-> #156

@diasdavid diasdavid removed this from the milestone 1 - js-ipfs on the browser milestone Jul 27, 2016

@diasdavid diasdavid changed the title from files API to track: `jsipfs files` -The files + mfs API Aug 15, 2016

@diasdavid diasdavid changed the title from track: `jsipfs files` -The files + mfs API to track: `jsipfs files` - The files + mfs API Aug 15, 2016

@diasdavid diasdavid added milestone 6 and removed milestone 1 labels Aug 20, 2016

@diasdavid

This comment has been minimized.

Show comment
Hide comment
@diasdavid

diasdavid Aug 20, 2016

Member

The remaining items (specific to mfs) go into the Milestone 6 - Feature parity

Member

diasdavid commented Aug 20, 2016

The remaining items (specific to mfs) go into the Milestone 6 - Feature parity

@diasdavid

This comment has been minimized.

Show comment
Hide comment
@diasdavid
Member

diasdavid commented Sep 7, 2016

@diasdavid diasdavid referenced this issue Dec 5, 2016

Closed

.cat subfiles #478

@diasdavid diasdavid added the ready label Dec 5, 2016

@diasdavid diasdavid changed the title from track: `jsipfs files` - The files + mfs API to track: `Files API + MFS` Jan 9, 2017

@diasdavid

This comment has been minimized.

Show comment
Hide comment
@diasdavid

diasdavid Jan 11, 2017

Member

Now Files Add work as expected with the option to do trickle-dag too! #707

Member

diasdavid commented Jan 11, 2017

Now Files Add work as expected with the option to do trickle-dag too! #707

@diasdavid

This comment has been minimized.

Show comment
Hide comment
@diasdavid

diasdavid Jun 16, 2017

Member

cat/get subtrees is now there too :)

Member

diasdavid commented Jun 16, 2017

cat/get subtrees is now there too :)

@diasdavid

This comment has been minimized.

Show comment
Hide comment
@diasdavid

diasdavid Sep 12, 2017

Member

Now tracked here https://github.com/ipfs/ipfs/blob/master/IMPLEMENTATION_STATUS.md

tl;dr; Files is done, MFS isn't

Member

diasdavid commented Sep 12, 2017

Now tracked here https://github.com/ipfs/ipfs/blob/master/IMPLEMENTATION_STATUS.md

tl;dr; Files is done, MFS isn't

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment