[community contribution] Files API milestone

[schomatis]

Community contribution opportunity

Area: IPFS files.
Estimated time: 5-10 hours (just an estimate, you can dedicate as much time as you want/have).
Difficulty: Easy, aimed to new contributors.
What you can learn: anything within the IPFS Files API domain (e.g., interaction with files in IPFS, how is a DAG constructed, how is a file formatted inside the DAG, DAG and UnixFS layers).
Mentor: @schomatis (meaning I commit to guide the contributor throughout the entire resolution of this issue answering any question he/she may have).

Summary

This is kind of a generic entry point issue for any possible contribution regarding the IPFS Files API and the corresponding milestone which has as its main objective to make the Go implementation of IPFS Files as developer-friendly as possible. This may include (but it's not restricted to) adding comments, refactoring code, writing examples and documentation, reviewing the code to find its obscure points and think of ways to simplify it.

Where to start (work in progress)

The first place to start for anyone who wants to learn how is a file added to IPFS would be with the ipfs add command (I'm not sure if we have nice ipfs add examples, but adding anything to the repo is a good first step). After experimenting with it you can take a look at the issues #5059 and #5081 for a rough discussion of what is the code flow when adding a file. These are not the final product, just what I've got so far, the idea is to turn them into a comprehensive document that new developers (such as you) could use to get a better grasp of what's going on under the hood.

The tricky part is that most of the code being referenced there has been moved to the go-unixfs repo (the Go implementation of IPFS is scattered throughout many different repositories) and although the file structure has been maintained you won't find it here (go-ipfs) but there (go-unixfs). The extra tricky part: when you install go-ipfs you'll see that the other repos this one depends on won't get downloaded to the typical Go path, e.g., ~/go/src/github.com/ipfs/go-unixfs. IPFS will use its own package manager (gx) to download those repos through the IPFS system, which is pretty cool, but may not be the most friendly thing for beginners. If you're browsing the code with your IDE and you follow a reference to the go-unixfs repo you'll see that it will take you to a path like ~/go/src/gx/ipfs/Qm_long_hash/go-unixfs, which is where gx stores its dependencies. (More on gx in an upcoming issue.)

After that you can look at the balanced builder, the package that takes a Unix file and breaks it up into pieces and formats it as a DAG node. What is a DAG node you may ask? Simply speaking, a node in the tree that makes up the file, I'm not sure we have a nice document about it just yet (check our official docs, https://docs.ipfs.io/, work in progress). Just reading through those parts of the code and pointing out blind spots in how are we describing the IPFS model (what we're taking for granted as previous knowledge, or what we're mentioning but not defining) is already a great help.

After you get an idea of how a file is added take a look at the DAG reader to see how the same DAG is read back to the user. There's a PR (another work in progress) that may be of help,

• decouple the DAG traversal logic from the DAG reader go-unixfs#12

• Add a DAG walker with support for IPLD Nodes go-ipld-format#39

You're still here? Great, whatever you understood from the previous references we should add it to a document in this PR. You can also leave in this issue any feedback about it, also feel free to open new issues/PRs discussing pitfalls or improvement in the code. Any kind of help will be very much appreciated.

[schomatis]

@CharlieRoseMarie If you don't mind me using you as a guinea pig here this could be a good place for you to start. Let me know what you think or any doubts you may have.

[CharlieRoseMarie]

@schomatis Looks good to me. :)

[schomatis]

/cc @overbool (just checking if you may be interested in something here)

[overbool]

@schomatis Yes, I would like to do it, thx

[overbool]

@schomatis There are many issues in this milestone, could you give me some advice on choosing a issue to fix in the beginning?

[nitishm]

@schomatis Is this something that I could help with? I just started playing with IPFS and would love to contribute, but I am having trouble finding something to start with.

[schomatis]

Hey @nitishm! You can find all of the help wanted issues here:

https://waffle.io/ipfs/go-waffle?label=help%20wanted

although that is a very diverse list, you'll have to dig through a bit to find something that you like, is there any part of IPFS you'd want to work on?

---

If you're interested in the Files API side of IPFS the current work I'm doing is mainly focused on MFS (https://github.com/ipfs/go-mfs/milestone/1) but most of that is advanced stuff (it depends on your comfort level with Go), the plus side being that as I'm actively working on it the RTT of questions/answers is going to be rather low, you wouldn't need to solve anything in particular, just looking at the PRs/issues and the code in general and asking questions would be very useful (as it would be an additional check that would force me to rethink my work).

[nitishm]

@schomatis I am pretty comfortable with Go since I have been using it at work for 2 years now. I do however struggle with open-source contributions. I am especially hesitant to contribute because I feel I do not have enough domain specific knowledge. With IPFS I have started to browse through the code, while playing around with ipfs add to understand the code flow.

Not looking for anything in particular, just really like the ipfs project and would like be a part of the community.

Would really appreciate if you could guide me (just a little bit) and I think I could definitely be of some help with go-mfs.

[schomatis]

I do however struggle with open-source contributions. I am especially hesitant to contribute because I feel I do not have enough domain specific knowledge.

I think we all go through something similar at first, from my experience the most useful thing to do at first is try to familiarize oneself with the code and ask questions (a lot of them), it does sound like you're being a drag, but if you look at a piece of the code and ask (in an elaborate question) why and I can't just point you to the documentation/code/tutorial that explains/justifies it, and I need 5 paragraphs to explain it then I am the one at fault here, and you're helping me pinpoint the obscure points in the reasoning behind the code.

[nitishm]

That makes sense. I have been playing with the code + debugger to look at the flow of ipfs add and ipfs get to understand the layers (block, dag-nodes, etc).
With the MFS is there someway I could help out ? Are there any open issues (easy one) that I could tackle, so that I can familiarize myself with the code-base and the use cases.
I have been reading through your comments to understand the difference between UnixFS, MFS and files.