track: `Files API + MFS` #60

Open
diasdavid opened this Issue Jan 26, 2016 · 9 comments

Projects

None yet

3 participants

@diasdavid
Member
diasdavid commented Jan 26, 2016 edited

The files API offers an interface to work with unixfs, a Merklelized Data Structure that represents a typical FileSystem used in a Unix OS. with files, dirs and symlinks. This API is what enables IPFS to add and retrieve files without having to ask the users to convert their data into a graph data structure.

To learn more about the Files API, you can

Implementation roadmap

  • Files API spec ipfs/specs#88
  • Unixfs Spec
  • jsipfs files add (current ipfs add)
  • jsipfs files cat (current ipfs cat)
    • cat subfiles
  • jsipfs files get (current ipfs get)
  • jsipfs files mv - Move files.
  • jsipfs files cp - Copy files into mfs.
  • jsipfs files ls - List directories.
  • jsipfs files mkdir - Make directories.
  • jsipfs files stat - Display file status.
  • jsipfs files rm ... - Remove a file.
  • jsipfs files read - Read a file in a given mfs.
  • jsipfs files write - Write to a mutable file in a given filesystem.
@diasdavid diasdavid self-assigned this Jan 26, 2016
@diasdavid diasdavid changed the title from feature: files to files Feb 23, 2016
@diasdavid
Member

Updated this issue

@nginnever
Collaborator

jsipfs files get

Taking a note from @noffle and a stab at jsipfs files get

Relevant go-ipfs links:
https://github.com/ipfs/go-ipfs/blob/master/core/commands/get.go

ARGUMENTS:

<ipfs-path> - The path to the IPFS object(s) to be outputted

OPTIONS:

-o, --output            string - The path where output should be stored
-a, --archive           bool   - Output a TAR archive
-C, --compress          bool   - Compress the output with GZIP compression
-l, --compression-level int    - The level of compression (1-9)

STEPS

1. Get the gzip compression options, bail if err (TODO: Look into js gzip modules) - return NoCompression if compress bool is false - return DefaultCompression if compress bool is true but compress level is undefined - return NoCompression and error if bool is true and compress level is not between 1-9 - return compress level if none of the above conditions are met 2. Get the node from the command request context. 3. Get the path from the argument . - TODO: List what happens here 4. Resolve the path, bail if err 5. Set archive to the option --archive provided. 6. DagArchive (this is equivalent to `ipfs getdag $hash | maybe_tar | maybe_gzip`) - TODO: list what happens here 7. create a readable stream of the dagnode 8. Check the stream bail if error 9. Check the path option. If it does not exist, defaut path to the multihash 10. Get the compression level from options 11. Get the archive boolean from options 12. Pass the readable stream and path to a write() function - If archive or compression options != gzip.NoCompress then pass to writeArchive() function - else pass the stream and path to writeExtracted() function - **If writeArchive()** - check archive is set to true - if it is, check if the ending of the path is `path + '.tar'` or `path + '.tar.gz'` - if it does not then add .tar to path name - adjust path name to .gz if compression level != gzip.NoCompress - create the file on disk - print to console `Saving archive to $path` - show a progress bar - **if writeExtracted()** - print to console `Saving file(s) to $path` - show progress bar - TODO: figure out what extractor is
@nginnever
Collaborator

jsipfs files cat

A look at jsipfs files cat

Relevant go-ipfs links:
https://github.com/ipfs/go-ipfs/blob/master/core/commands/cat.go
https://github.com/ipfs/go-ipfs/blob/master/core/coreunix/cat.go

ARGUMENTS:

<ipfs-path>... - The path to the IPFS object(s) to be outputted

DESCRIPTION:

Retrieves the object named by <ipfs-or-ipns-path> and outputs the data
it contains.

STEPS

1. Get the node from the command request context. 2. If node.onlineMode = false, setupOfflineRouting() - TODO: List setupOfflineRouting steps 3. call cat() function passing in the context, the node, and the path(s) from the arguments - create read streams array for each path provided - for each path, call coreunix.Cat(ctx, node, fpath) - resolve the dagnode - return a readable stream - append the stream from resolve to the read streams array - collect the total length by adding each buffer length returned from resolve - return the readers array, length, and null or error 4. Do a conditional GC 5. set the length in the response 6. combine the readers streams into one read stream 7. check if the length < the smallest progress bar size 8. create the progress bar if not 9. set the res to the read stream.
@diasdavid diasdavid changed the title from files to files / add Mar 21, 2016
@diasdavid
Member

merged #67

@diasdavid diasdavid removed their assignment Mar 21, 2016
@diasdavid diasdavid changed the title from files / add to files API Mar 30, 2016
@nginnever nginnever was assigned by diasdavid Apr 2, 2016
@diasdavid
Member

@nginnever I know you are working on this. Can you make a PR tagged with WIP from a branch under this project?

@diasdavid
Member

-> #156

@RichardLitt RichardLitt referenced this issue in ipfs/ipfs May 8, 2016
Open

IPFS Weekly Updates #151

@diasdavid diasdavid changed the title from files API to track: `jsipfs files` -The files + mfs API Aug 15, 2016
@nginnever nginnever was unassigned by diasdavid Aug 15, 2016
@diasdavid diasdavid changed the title from track: `jsipfs files` -The files + mfs API to track: `jsipfs files` - The files + mfs API Aug 15, 2016
@diasdavid diasdavid added milestone 6 and removed milestone 1 labels Aug 20, 2016
@diasdavid
Member

The remaining items (specific to mfs) go into the Milestone 6 - Feature parity

@diasdavid
Member
@diasdavid diasdavid referenced this issue Dec 5, 2016
Closed

.cat subfiles #478

@diasdavid diasdavid added the ready label Dec 5, 2016
@diasdavid diasdavid changed the title from track: `jsipfs files` - The files + mfs API to track: `Files API + MFS` Jan 9, 2017
@diasdavid
Member

Now Files Add work as expected with the option to do trickle-dag too! #707

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment