Why does the PBNode have a links and data section?

[Stebalien]

That is, why not just have a data section or data and type sections and store the links inline?

[Stebalien]

The current system allows one to find related blocks without understanding the block data itself (unless the block is encrypted) but I can't see how this would be useful (and the paper doesn't explain it).

[jbenet]

because the user may put arbitrary data there that ipfs would not be able to understand. do you see the links below?

123c0a2212206677f25c4544d3c0a0ea51ba5950e8d8de95c66086baa00ea42e5142159054ed121
3696e7374616c6c2d736861726e6573732e736818e90a12360a221220b58fc18130c306dc0931be
5a999cb369010bc9022e8904bf968fb403e85c49a0120d72616e646f6d2d6465702e676f1894031
2320a2212206e45923d08e23900940717107a9b585225d4e163c11f61868859bb17ea94c87d1208
736861726e65737318ffed0512420a22122086313cd562a1cfcc40d79a1afaa6fa6f42118947921
cd9a3fad5be0c72122d1d1219746573742d6167677265676174652d726573756c74732e736818a4
03123b0a2212203fa0b89b52d278de219657a8f55d630fde813477d85ce4dee6629d059c02c07c1
212746573742d6c69622d6861736865732e7368188f0312340a2212208cd186673578a00b9f3d27
eba5ef48eea5f89f1eb0567497cdfde27c4d8b6f4c120b746573742d6c69622e7368188c460a020
801

[Stebalien]

The question was why is this baked into the node itself? That is, why not have IPBS (interplanetary block storage) and build things like ipfs on top of it. A type field that says "this is an fs object" would be enough for ipfs to look into the data section and interpret it as a directory/file. Basically, why encode the DAG at the protocol layer (loose terminology) instead of the application layer.

[jbenet]

that's the point of IPFS. to create one clean merkleized transport that layers on a universal path namespace, interoperable everywhere. and to do that, we need to be able to understand one thing: the things you want to be merkle links.

i understand your question + motivation, i'm saying that the underlying "IPBS" does exist (it's just bitswap over our libp2p, without IPFS proper). but the important proposition here is a "thin waist" of authenticated data structures

[Stebalien]

I think this is the piece I was missing:

universal path namespace

So if I ask for /ipfs/$hash/first/second/third, $hash, first, and second don't have to be directories, they just have to have links first, second, and third respectively? That is, all objects can name other objects?

I still don't understand why you didn't just split the roles like unix:

• Directories: Objects that define a namespace and only define a namespace.
• Files: Objects that store data and only store data (do not name). Files can refer to other objects either by name but don't "export" a namespace.

My primary concern is mapping ipfs onto a filesystem. In the proposed versioned file system, not all of the objects can be represented purely as files or directories (which is noted in the paper). This means that my version control application can't just rely on the filesystem abstraction but needs to understand ipfs.

So I guess a better question is: why both (data + links) instead of either (data OR links)? Efficiency?

[Stebalien]

Note: I understand how storing a namespace and some data under the same named file can be useful; it's how the web currently works (where index.html stores the data and the directory stores the namespace). I'm just wondering if this is sufficient motivation to break with the standard filesystem abstraction.

[jbenet]

Short answer (I can expand later): Because this is one thing unix got wrong. Directories should've been files too. Files should've been able to link to other files. HTML/HTTP got this right. Took me years to understand this.

Think about it this way: a traditional unix file system is an application of a generalized datastructure web.

It's a shame we have to have separate files and dirs in our unixfs to deal with legacy unix expectations, but it's not too bad in practice.

consider this: in git, commits, trees, and blobs are all "objects". Same in ipfs. It's just that our "blobs" can link to other blobs. (Or anything).

—
Sent from Mailbox

On Mon, Sep 14, 2015 at 5:21 PM, Steven Allen notifications@github.com
wrote:

Note: I understand how storing a namespace and some data under the same named file can be useful; it's how the web currently works (where index.html stores the data and the directory stores the namespace). I'm just wondering if this is sufficient motivation to break with the standard filesystem abstraction.

Reply to this email directly or view it on GitHub:
#29 (comment)

[jbenet]

Oh and the big win is path traversals over arbitrary other linked datastructures, like this:

$repo/branches/master/parent/parent/tree/foo/bar/baz/10

$torrent/pieces/5

$tar/foo/bar/baz/mode

$bitcoinblock/parent/parent/txns/12/inputs/0

—
Sent from Mailbox

On Mon, Sep 14, 2015 at 5:21 PM, Steven Allen notifications@github.com
wrote:

Note: I understand how storing a namespace and some data under the same named file can be useful; it's how the web currently works (where index.html stores the data and the directory stores the namespace). I'm just wondering if this is sufficient motivation to break with the standard filesystem abstraction.

Reply to this email directly or view it on GitHub:
#29 (comment)

[Stebalien]

You make a compelling case.

[whyrusleeping]

the tar thing you can try out today:

ipfs tar add something.tar
ipfs cat <tarhash>/-path/-inside/-tar/data

note: I prefixed each of the entry names in the tar with a - character so that we can have link names with special meaning, ala 'data'.

[madavieb]

This issue has been moved to https://discuss.ipfs.io/t/why-does-the-pbnode-have-a-links-and-data-section/459.