Forbid putting blocks over 2MiB in the blockstore

[Stebalien]

While potentially useful, our transports won't accept blocks of this size. If we allow users to add them to their datastore, they'll run into hard to debug issues where we won't actually transfer the files.

I believe I was thinking about banning this in the go-ipfs block API (and the add options) instead of at the blockstore level itself. This limitation is a side effect of the IPFS protocol as it exists today, not necessarily IPLD itself.

While I'd prefer not to add this blanket restriction to the default blockstore, it might make sense to implement it as a wrapper. That would make it easy for applications to specify a size policy without enforcing a default policy (alternatively, it can just be added as an optional/configurable limit to the default blockstore.

[Stebalien]

Note: We can lift this restriction later if/when we add transports that can transfer larger blocks. This is a restriction on what we allow now for current protocols and use-cases.

[aschmahmann]

@gammazero @Stebalien was the reason why ipfs/go-ipfs-blockstore#61 wasn't accepted because we wanted it to be configurable/not in the default blockstore?

While I'd prefer not to add this blanket restriction to the default blockstore, it might make sense to implement it as a wrapper. That would make it easy for applications to specify a size policy without enforcing a default policy (alternatively, it can just be added as an optional/configurable limit to the default blockstore.

Are we good with just having a blockstore wrapper in go-ipfs that errors on Put or PutMany if the block size exceeds 2MiB?

[ribasushi]

Are we good with just having a blockstore wrapper in go-ipfs that errors on Put or PutMany if the block size exceeds 2MiB?

I am totally good with that.

I objected to adding it to the core interface, as limiting the IPLD world by libp2p limitations is not the right way forward. I should be able to put together an http-speaking app, and use go-ipfs-blockstore to keep track of 1TiB "blocks".

Having a limitation in a libp2p-speaking app ( go-ipfs ) is totally fair game, an expected.