Skip to content

Commit

Permalink
first sketch of the primer
Browse files Browse the repository at this point in the history
  • Loading branch information
Matt Zumwalt committed Aug 20, 2016
0 parents commit 855779a
Show file tree
Hide file tree
Showing 10 changed files with 242 additions and 0 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
_book
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Introduction

# Concepts

* Cryptographic Hashes and Content Addressability
* Authenticated Graphs
* Turning Files into Trees
* Turning any Data into Trees
* Immutability: "Changes" as _additions_ to the tree
* Publishing hashes on the DHT
* Getting data from Peer to Peer Network
* CRDTs
* Pubsub
* Authenticated Streams (with pubsub)
20 changes: 20 additions & 0 deletions SUMMARY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Summary

* [Files on IPFS](files-on-ipfs/README.md)
1. [Tutorial: Adding Files to IPFS](/files-on-ipfs/1-add-files/README.md)
2. [Tutorial: Accessing files on IPFS](/files-on-ipfs/2-access-files/README.md)
3. [Tutorial: Updating Files on IPFS using IPNS](/files-on-ipfs/3-update-files/README.md)
4. [Tutorial: Making Sure Files Persist in IPFS](/files-on-ipfs/4-pin-files/README.md)
* The IPFS DAG
1. Tutorial: Create a cryptographic hash
2. [Tutorial: Build a tree of data in IPFS using cryptographic hashes to link the pieces (a Merkle DAG)](/ipfs-dag/blocks-from-scratch/README.md)
3. [Tutorial: Turn a file into a tree of hashes](/ipfs-dag/files-as-dags/README.md)
4. Tutorial: Explore the types of software that use hash trees to track data
* Dynamic Content on IPFS
1. Disclaimer: Dynamic content on IPFS is a Work in Progress
2. Tutorial: Add data to the DAG (locally)
3. Tutorial: Tell peers about your Changes
4. Tutorial: Use hashes to get someone's changes from IPFS
5. Tutorial: Use a pub/sub strategy to pass around messages about changes
6. Tutorial: Resolve conflicts with a merge strategy (CRDTs)
* Distributed Computation
41 changes: 41 additions & 0 deletions files-on-ipfs/1-add-files/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Adding a file into IPFS

## Goals (Learning Objectives)
After doing this tutorial you will be able to
* Add a file to IPFS
* Read a file out of IPFS

## Steps
### Step 1: Add the file to IPFS

```
ipfs add afile.txt
```
This will return a hash that looks like `qms2hjwx8qejwm4nmwu7ze6ndam2sfums3x6idwz5myzbn`. Copy that hash for the next step.

### Step 2: Confirm that the file is now in IPFS

Type the following into the command line, replacing `thathash` with the hash you copied from the last step.
```
ipfs ls thathash
```

The above command should print out something like:
```
ipfs@earth ~> ipfs ls qms2hjwx8qejwm4nmwu7ze6ndam2sfums3x6idwz5myzbn
qmv8ndh7ageh9b24zngaextmuhj7aiuw3scc8hkczvjkww 7866189
qmuvjja4s4cgyqyppozttssquvgcv2n2v8mae3gnkrxmol 7866189
qmrgjmlhlddhvxuieveuuwkeci4ygx8z7ujunikzpfzjuk 7866189
qmrolalcquyo5vu5v8bvqmgjcpzow16wukq3s3vrll2tdk 7866189
qmwk51jygpchgwr3srdnmhyerheqd22qw3vvyamb3emhuw 5244129
```

What this is showing is all of the immediate sub-blocks of your file, and the
size of them and their children on disk. If you want to know more about how these blocks and sub-blocks work, explore the [tutorials about the IPFS DAG](/ipfs-dag)

### Step 2: Read the File




## Explanation
9 changes: 9 additions & 0 deletions files-on-ipfs/2-access-files/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Accessing files on IPFS
## Goals (Learning Objectives)
After doing this tutorial you will be able to
* _todo_

## Steps
1. _todo_

## Explanation
52 changes: 52 additions & 0 deletions files-on-ipfs/3-update-files/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Update Files on IPFS using IPNS

## Goals (Learning Objectives)
After doing this tutorial you will be able to
* Point to a file in IPFS using IPNS and update that IPNS pointer over time as the file changes
* Track an entire website using IPFS and IPNS as it changes over time
* Map DNS to IPNS

## Conceptual Framework

Normally, updating content means replacing a file - for instance, if I update a blog post, then people will see the edited file, and not the new one.
However, with IPFS, both verisons of the file will be accessible in the network. It's not a matter of replacing: you add the new one, too. This raises the question: how do we actually update our links, so that people will see the new version of a file? They can't go to the file's location, because IPFS locates files by looking for their hashes (that's what content-addressed means). So, you need to have a way of pointing people to the new hash easily.

The trick is to add new the content, and then update a pointer to that content. So, there needs to be a way of having a mutable pointer.

This is where IPNS comes in, the InterPlanetary Naming System (Name Service?). IPNS is a simple service that uses your peer ID to point to a particular hash. This hash can change, but your peer ID doesn't. That means that you can point to content in IPFS that may also change, and people can still access it without needing to know the new hash before hand.

**Author Question**: Does IPNS point to a constant hash that is in your config, or does it just use your peerId? Does your IPNS hash ever change?

## Steps

_Here's where you list links to the activities in this module._

1. If you have not already done it, follow the tutorial to **[Add a file to IPFS](/files-on-ipfs/add-files)**
2. Set up IPNS on your node
3. Create an IPNS entry that points to your file
4. Modify your File and add the modified version to IPFS
5. Update the IPNS entry to point to the new version

(maybe put in another tutorial)
6. Map DNS to IPNS

(maybe put in another tutorial)
7. add multiple files (ie. an entire website) to IPFS
8. Use IPNS to link to the entire website, or any file in the website

## Notes

From IRC
```
??. Use the files API.
15:14:11 <•Kubuxu> you are limited to one IPNS per node
15:14:18 <•Kubuxu> but you can have directories in IPNS
15:14:26 <•Kubuxu> combine it with files API
15:15:19 <•Kubuxu> ipfs files mkdir /public
15:15:46 <•Kubuxu> ipfs files cp /ipfs/$HASH_OF_PAPER /public/my-faviourite-paper.pdf
15:16:16 <•Kubuxu> ipfs name publish $(ipfs files stat --hash /public)
15:16:41 ⇐ jaboja quit (~jaboja@2a00:f41:3875:fd4b:de85:deff:fe55:967a) Ping timeout: 264 seconds
15:16:42 <•Kubuxu> you can reference your paper with: /ipns/$PEERID/my-faviourite-paper.pdf
15:16:50 <•Kubuxu> and you can have many more files in there
```
8 changes: 8 additions & 0 deletions files-on-ipfs/4-pin-files/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Making sure that a file persists in the IPFS network

## Goals (Learning Objectives)
After doing this tutorial you will be able to
* _todo_

## Steps
1. _todo_
14 changes: 14 additions & 0 deletions files-on-ipfs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Files on IPFS

## Prerequisites
These tutorials are tested with go-ipfs version 0.4.3-rc1. _Please update this file on github to reflect any other versions that have been tested._

- You should have some familiarity with the commandline.
- You should have `ipfs` installed: this tutorial will use go-ipfs, version 0.4.3-rc1.

## Tutorials

1. [Adding Files to IPFS](/files-on-ipfs/1-add-files/README.md)
2. [Accessing files on IPFS](/files-on-ipfs/2-access-files/README.md)
3. [Updating Files on IPFS using IPNS](/files-on-ipfs/3-update-files/README.md)
4. [Making Sure Files Persist in IPFS](/files-on-ipfs/4-pin-files/README.md)
30 changes: 30 additions & 0 deletions ipfs-dag/blocks-from-scratch/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Creating a Merkle Tree from Scratch

*Work in Progress* _This is the content from [this existing tutorial](https://ipfs.io/ipfs/QmTkzDwWqPbnAh5YiV5VwcTLnGdwSNsNTn2aDxdXBFca7D/example#/ipfs/QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7/data/readme.md) vaguely re-framed to fit the tutorial framework._

## Goals
* Build a tree of data in IPFS using cryptographic hashes to link the pieces (a Merkle DAG)

## Explanation: Blocks vs Objects
In ipfs, a block refers to a single unit of data, identified by its key (hash).
a block can be any sort of data, and does not necessarily have any sort of
format associated with it. an object, on the other hand, refers to a block that
follows the merkledag protobuf data format. it can be parsed and manipulated
via the `ipfs object` command. any given hash may represent an object or a block.

# Steps
## Step 1
Creating your own blocks is easy! simply put your data in a file and run
`ipfs block put <yourfile>` on it, or you can pipe your filedata into
`ipfs block put`, like so:

## Step 2
```
$ echo "This is some data" | ipfs block put
QmfQ5QAjvg4GtA3wg3adpnDJug8ktA1BxurVqBD8rtgVjM
$ ipfs block get QmfQ5QAjvg4GtA3wg3adpnDJug8ktA1BxurVqBD8rtgVjM
This is some data
```
Note: When making your own block data, you wont be able to read the data with
`ipfs cat`, this is because you are inputting raw data without the unixfs data
format. To read raw blocks use `ipfs block get` as shown in the example.
53 changes: 53 additions & 0 deletions ipfs-dag/files-as-dags/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Turn a File into a Merkle Tree

*Work in Progress* _This is the content from [this existing tutorial](https://ipfs.io/ipfs/QmTkzDwWqPbnAh5YiV5VwcTLnGdwSNsNTn2aDxdXBFca7D/example#/ipfs/QmQwAP9vFjbCtKvD8RkJdCvPHqLQjZfW7Mqbbqx18zd8j7/data/readme.md) vaguely re-framed to fit the tutorial framework._

## Goals
* Explain how IPFS represents Files as Merkle trees
* Explore the Merkle Tree Blocks that make up a File in IPFS

## Steps
### Step 1

```
# ensure this file is larger than 256k
ipfs add alargefile
ipfs ls thathash
```

The above command should print out something like:
```
ipfs@earth ~> ipfs ls qms2hjwx8qejwm4nmwu7ze6ndam2sfums3x6idwz5myzbn
qmv8ndh7ageh9b24zngaextmuhj7aiuw3scc8hkczvjkww 7866189
qmuvjja4s4cgyqyppozttssquvgcv2n2v8mae3gnkrxmol 7866189
qmrgjmlhlddhvxuieveuuwkeci4ygx8z7ujunikzpfzjuk 7866189
qmrolalcquyo5vu5v8bvqmgjcpzow16wukq3s3vrll2tdk 7866189
qmwk51jygpchgwr3srdnmhyerheqd22qw3vvyamb3emhuw 5244129
```

What this is showing is all of the immediate sub-blocks of your file, and the
size of them and their children on disk.

### Step 2: inspect the blocks
If you feel adventurous you can get a lot of different information out of these
different blocks. you can use the sub-block hashes as input to `ipfs cat` to
see only the data in any given sub-tree (the data of that block and its
children). to see just the data of a given block and not its children, use
`ipfs block get`. but be careful, as `ipfs block get` on an intermediate block
will print out the raw binary data of its dag structure to your screen.

### Step 3: get the size of the blocks
`ipfs block stat` will tell you the exact size of a given block (without its
children), and `ipfs refs` will tell you all the children of that block.
similarly, `ipfs ls` or `ipfs object links` will show you all children and
their sizes. `ipfs refs` is a more suitable command for scripting something
to run on each child block of a given object.

## Explanation

The `ipfs add` command will create a merkle dag out of the data in the files you
specify, it follows the unixfs data format when doing this. what this means, is
that your files are broken down into blocks, and then arranged in a tree-like
structure using 'link nodes' to tie them together. a given files 'hash' is
actually the hash of the root (uppermost) node in the dag. for a given dag, you
can easily view the sub-blocks under it with `ipfs ls`.

0 comments on commit 855779a

Please sign in to comment.