pkg/nixpath/chunker: add

[flokli]

This provides two different implementations to chunk data.

I'm not entirely sure if this should go into pkg/nixpath/chunker, or in another place.

[flokli]

I just used the values here I used in nix-casync initially. This probably should still be refined once we can ingest a bit of data.

[zimbatm]

will you be able to re-use that and be casync-compatible or is that a new system altogether?

[flokli]

It's a new chunking mechanism, with a much smaller interface. You pass in a reader to some data and can use some iterator interface to get chunks, while reading through the data.

nix-casync used desync, which provided a lot of functionality that we didn't use (.caidx). Also, the way it was designed required us to first write the data to be chunked to a (temporary) file.

The chunking method used to chunk up data shouldn't matter when it comes to substitution. However, using the same chunking method with similar parameters should yield more block reuse.

[flokli]

I marked this to a draft. I'm not entirely sure it belongs in pkg/nixpath, and if so, how to structure other things in the package.

Also, right now, go-nix mostly contains (re-)implementations of some common concepts found in Nix, and things like chunking mechanisms are not part of that. This might change once we have come up with a (already in-the-works) new remote store protocol.

[flokli]

Closing in favor of #86.