First draft for compressed data packages

[sabas]

This spec states only how to archive or compress and which extension to use.

See #132

[rufuspollock]

@sabas apologies for slowness on merging - i have been looking at this and just been mulling some points over.

@danfowler - i'd also invite you to do a quick review here :-)

[danfowler]

@sabas @rgrp if I'm understanding correctly, the datapackage.json file would be compressed along with the data files, right? This would mean some extraction operation would be required in order to read basic metadata about the package.

[rufuspollock]

@mfenner would you like to review this?

[sabas]

It's just a draft for discussion :-) The problem of discovering the metadata is interesting, something like the jar manifest? https://en.wikipedia.org/wiki/JAR_(file_format)#Manifest

[pwalsh]

@sabas wouldn't the datapackage.json in a compressed Data Package be the equivalent of a manifest file in a JAR?

[sabas]

Yes @pwalsh I linked it as a reference for a possible implementation. The datapackage manifest would be the datapackage.json

[mfenner]

In the spirit of keeping things simple I wouldn't provide two options (.dp and .dpz). And in the spirit of not reinventing the wheel I would look at https://researchobject.github.io/specifications/bundle/, which uses Universal Container Format (UCF). Or for a software packacking example Chrome extensions: https://developer.chrome.com/extensions/packaging

[mfenner]

Pebble watch files are another example: http://www.pebbledev.org/wiki/Applications/

[tfmorris]

I'll second @mfenner 's suggestion to exhaust all possible existing alternatives before defining a new format. If you are forced to define something new, I'd strongly consider using zip instead of tar, since every other container format in the world from JAR to EPUB to Research Object Bundle has settled on it. There's an old overview of a bunch of the zip-based formats here: http://broadcast.oreilly.com/2009/01/packaging-formats-of-famous-ap.html

[rufuspollock]

OK, it sounds like we do want to go with zip. I'm going to close this PR for the time being and suggest we move discussion back to the main issue at #132