Skip to content

Building a parcel

philipl edited this page Feb 14, 2014 · 22 revisions

Building a parcel is a relatively simple affair - put all your files and metadata in a directory and then tar it up.

Establish your parcel layout

As discussed [elsewhere](The parcel format), the internal layout of a parcel is up to you, beyond the meta directory. As an example, here is how the GPLExtras (LZO) plugin parcel looks:

meta/
meta/parcel.json
meta/gplextras_env.sh
lib/
lib/hadoop/
lib/hadoop/lib/
lib/hadoop/lib/hadoop-lzo.jar -> hadoop-lzo-0.4.15-gplextras5.0.0-beta-2-SNAPSHOT.jar
lib/hadoop/lib/COPYING.hadoop-lzo
lib/hadoop/lib/native/
lib/hadoop/lib/native/libgplcompression.a
lib/hadoop/lib/native/libgplcompression.la
lib/hadoop/lib/native/libgplcompression.lai
lib/hadoop/lib/native/libgplcompression.so.0.0.0
lib/hadoop/lib/native/libgplcompression.so.0
lib/hadoop/lib/native/libgplcompression.so
lib/hadoop/lib/hadoop-lzo-0.4.15-gplextras5.0.0-beta-2-SNAPSHOT.jar
lib/hadoop-0.20-mapreduce/
lib/hadoop-0.20-mapreduce/lib/
lib/hadoop-0.20-mapreduce/lib/hadoop-lzo.jar -> ../../hadoop/lib/hadoop-lzo.jar
lib/hadoop-0.20-mapreduce/lib/native/
lib/hadoop-0.20-mapreduce/lib/native/libgplcompression.a -> ../../../hadoop/lib/native/libgplcompression.a
lib/hadoop-0.20-mapreduce/lib/native/libgplcompression.la -> ../../../hadoop/lib/native/libgplcompression.la
lib/hadoop-0.20-mapreduce/lib/native/libgplcompression.lai -> ../../../hadoop/lib/native/libgplcompression.lai
lib/hadoop-0.20-mapreduce/lib/native/libgplcompression.so.0.0.0 -> ../../../hadoop/lib/native/libgplcompression.so.0.0.0
lib/hadoop-0.20-mapreduce/lib/native/libgplcompression.so.0 -> ../../../hadoop/lib/native/libgplcompression.so.0
lib/hadoop-0.20-mapreduce/lib/native/libgplcompression.so -> ../../../hadoop/lib/native/libgplcompression.so
lib/hadoop-0.20-mapreduce/lib/hadoop-lzo-0.4.15-gplextras5.0.0-beta-2-SNAPSHOT.jar -> ../../hadoop/lib/hadoop-lzo-0.4.15-gplextras5.0.0-beta-2-SNAPSHOT.jar
lib/impala/
lib/impala/lib/
lib/impala/lib/COPYING.impala-lzo
lib/impala/lib/libimpalalzo.so

Place your files in an appropriately named directory

For a given parcel with a name and a version, the top level directory must be named name-version. So, for our GPLExtras example, the name is GPLEXTRAS and the version is 5.0.0-gplextras5b2.p0.32, so the directory name must be GPLEXTRAS-5.0.0-gplextras5b2.p0.32. (Note that you don't have to use as complicated version strings as we do.)

Tar up your directory into an appropriately named parcel file

Creating a parcel file from the directory is a simple exercise of running the tar command. As with directory naming, parcel filenames must follow a specific format: [name]-[version]-[distro suffix].parcel. The list of valid distro suffixes is available [here](Parcel distro suffixes).

To continue our GPLExtras example:

$ tar zcvf GPLEXTRAS-5.0.0-gplextras5b2.p0.32-el6.parcel GPLEXTRAS-5.0.0-gplextras5b2.p0.32/ --owner=root --group=root

We strongly recommend that file ownership be normalised to root when building the parcels. This ensures that no file is accidentally put in a situation where it could be modified or replaced by an unprivileged user on an end-user machine. CM attempts to normalise ownership when unpacking parcels, but it's generally good practice to do it at build time too.

Validation

To help make sure that the parcels you build are correct, you should run our validation tool, available in this git repo, against your parcels. The tool offers different granularities of operation - it can validate individual json files, a parcel directory, or a parcel file - so you can use it throughout the development process.

Validate a parcel.json file

$ ./validator -p parcel.json

Validate an alternatives.json file

$ ./validator -a alternatives.json

Validate an permissions.json file

$ ./validator -r permissions.json

Validate a parcel directory

$ ./validator -d CDH-5.0.0-0.cdh5b2.p0.283/

Validate a parcel file

$ ./validator -f CDH-5.0.0-0.cdh5b2.p0.283-el6.parcel

Relevant environment variables for plugin parcels

As discussed in the section on the [environment script](The parcel defines script), connecting a plugin parcel to a service requires the plugin parcel's environment script to append to specific environment variables that allow the service to discover the plugin. The specific variables to use vary, depending on which server the plugin is for.

[This table](Plugin Parcel Environment Variables) lists the relevant variables for the various CDH services.

You can’t perform that action at this time.