| description | page_title |
|---|---|
It is possible to write custom builders using the Packer plugin interface, and
this page documents how to do that.
|
Custom Builders - Extending |
Packer builders are responsible for creating a virtual machine, setting the virtual machine up for provisioning, and then turning that provisioned virtual machine into a machine image. We officially maintain and distribute several builders, including builders to create images on Amazon EC2, VMware, Google Compute Engine, and many more. Refer to the Builders documentation for details.
This page explains how to use the Packer plugin interface to write custom builders. If you want your builder to support HashiCorp Cloud Platform (HCP) Packer, you should also review the HCP Packer Support documentation.
~> Warning: This is an advanced topic that requires strong knowledge of Packer and Packer plugins.
We recommend reviewing the following resources before you begin development:
- Developing Plugins - Overview
- The Go language. You must write custom plugins in Go, so this guide assumes you are familiar with the language.
To create your own builder, you must create a struct that implements the
packer.Builder interface. It is reproduced below for reference.
type Builder interface {
ConfigSpec() hcldec.ObjectSpec
Prepare(...interface{}) ([]string, []string, error)
Run(context.Context, ui Ui, hook Hook) (Artifact, error)
}This method returns a hcldec.ObjectSpec, which is a spec necessary for using HCL2 templates with Packer. For information on how to use and implement this function, check our object spec docs
The Prepare method for each builder will be called by the Packer core
at the beginning of the build. Its purpose is to parse and validate the
configuration template provided to Packer with packer build your_packer_template.json, but not to execute API calls or begin creating any
resources or artifacts.
The configuration from your Packer template is passed into the Prepare() method
as an array of interface{} types, but is generally map[string]interface{}.
The Prepare method is responsible for translating this configuration into an
internal structure, validating it, and returning any errors.
If multiple parameters are passed into Prepare(), they should be merged together into the final configuration, with later parameters overwriting any previous configuration. The exact semantics of the merge are left to the builder author.
We recommend that you use the
mapstructure library to
decode the interface{} into a meaningful structure. Mapstructure will take an
interface{} and decode it into an arbitrarily complex struct. If there are any
errors, it generates very human friendly errors that can be returned directly
from the prepare method. You can find many usage examples of this library within
the Prepare() methods of HashiCorp-maintained Packer plugins.
While Packer does not actively enforce this, no side effects should occur
from running the Prepare method. Specifically: don't create files, don't
launch virtual machines, etc. Prepare's purpose is solely to load the
configuration from the template into a format usable by your builder, to
validate that configuration, and to apply necessary defaults to that
configuration.
In addition to the configuration provided in the Packer template, Packer will
also supply a common.PackerConfig
containing meta-information such as the build name, builder type, core version,
etc, and coded into a map[string]interface{}. One important piece of
meta information in this map is the packer.DebugConfigKey set to boolean
true if debug mode is enabled for the build. If this is set to true, then the
builder should enable a debug mode which assists builder developers and
advanced users to introspect what is going on during a build. During debug
builds, parallelism is strictly disabled, so it is safe to request input from
stdin and so on.
Prepare() returns an array of strings and an error. The array of strings is a special list of keys for variables created at runtime which your builder will make accessible to provisioners using the generatedData mechanism (see below for more details) An example could be an instance ID for the cloud instance created by your builder. If you do not plan to make use of the generatedData feature, just return an empty list. The error should be used if there is something wrong with the user-provided configuration and the build should not proceed.
Run is executed, often in parallel for multiple builders, to actually build
the machine, provision it, and create the resulting machine image, which is
returned as an implementation of the packer.Artifact interface.
The Run method takes three parameters. The context.Context used to cancel the
build. The packer.Ui object is used to send output to the console.
packer.Hook is used to execute hooks, which are covered in more detail in the
Provisioning section below.
Because builder runs are typically a complex set of many steps, the packer-plugin-sdk contains a multistep module. Multistep allows you to separate your build logic into multiple distinct "steps" with separate run and cleanup phases, and run them in order. It supports cancellation mid-step, pausing between steps when debugging, the CLI's on-error flag, and more. All of the HashiCorp maintained builders make use of this module, and while it is not required for builder implementation, it will help you create your builder in a way that matches user and Packer Core assumptions. The SDK also provides a number of "helper" generic steps that may prevent you from having to re-implement work that has already been done by the HashiCorp maintainers. Examples include sending boot commands, connecting to SSH, and creating virtual CDs to mount on your VM. Take a look at the communicator and multistep/commonsteps modules in the SDK to see what tools are available to you.
Finally, Run should return an implementation of packer.Artifact. More
details on creating a packer.Artifact are covered in the artifact section
below. If something goes wrong during the build that prevents an artifact from
being correctly created, Run should return an error and a nil artifact. Note
that your builder is allowed to produce no artifact and no error, although this
is a rare use case.
The Cancel method can be called at any time and requests cancellation of any
builder run in progress. This method should block until the run actually stops.
Note that the Cancel method will not be called by Packer versions >= 1.4.0.
The <-ctx.Done() can unblock at any time and signifies request for
cancellation of any builder run in progress.
Cancels are most commonly triggered by external interrupts, such as the user
pressing Ctrl-C. Packer will only exit once all the builders clean up, so it
is important that you architect your builder in a way that it is quick to
respond to these cancellations and clean up after itself. If your builder makes
a long-running call, you should consider the possibility that a user may cancel
the build during that call, and make sure that such a cancellation is not
blocked.
The Run method is expected to return an implementation of the
packer.Artifact interface. Each builder must create its own implementation of
this interface.
Most of the pieces of an artifact should be fairly self-explanatory by reading the packer.Artifact interface documentation.
However one part of an artifact that may be confusing is the BuilderId method.
This method must return an absolutely unique ID for the builder. In general, a
reasonable ID would be the github username or organization that created the
builder, followed by the platform it is building for. For example, the builder
ID of the VMware builder is "hashicorp.vmware".
Post-processors use the builder ID value in order to make some assumptions about the artifact results and to determine whether they are even able to run against a given artifact, so it is important that this ID never changes once the builder is published.
The builder ID for each builder is included on the associated documentation page.
Packer has built-in support for provisioning using the Provisioner plugins. But builders themselves, rather than the Packer core, must determine when to invoke the provisioners since only the builder knows when the machine is running and ready for communication.
When the machine is ready to be provisioned, run the packer.HookProvision
hook, making sure the communicator is not nil, since this is required for
provisioners. An example of calling the hook is shown below:
hook.Run(context.Context, packer.HookProvision, ui, comm, nil)At this point, Packer will run the provisioners and no additional work is necessary.
If you are using the multistep tooling, the Packer plugin SDK contains a generic StepProvision which handles execution the provision hook for you and automatically supplies any custom builder generatedData you would like to provide to provisioners (see below for more details on generatedData.)
~> Note: In HCL2 the JSON template engine and generated data is slowly going to be deprecated in favor of using HCL2 objects, we will keep on supporting those but it is recommended to avoid using them if you can.
Packer JSON makes it possible to provide custom template engine variables to be
shared with provisioners and post-processors using the build function.
JSON template build docs are here
and HCL template build docs are here.
As of Packer v1.5.0, builder Prepare() methods return a list of custom variables
which we call generated data. We use that list of variables to generate a
custom placeholder map per builder that combines custom variables with the
placeholder map of default build variables created by Packer. Here's an example
snippet telling packer what will be made available by the builder:
func (b *Builder) Prepare(raws ...interface{}) ([]string, []string, error) {
// ...
generatedData := []string{"SourceImageName"}
return generatedData, warns, nil
}When a user provides a Packer template that uses a build function, Packer
validates that the key in the build function exists for a given builder using
this generatedData array. If it does not exist, then Packer validation will
fail.
Once the placeholder is set, it's necessary to pass the variables' real values when calling the provisioner. This can be done as the example below:
func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
// ...
// Create map of custom variable
generatedData := map[string]interface{}{"SourceImageName": "the source image name value"}
// Pass map to provisioner
hook.Run(context.Context, packer.HookProvision, ui, comm, generatedData)
// ...
}In order to make these same variables and the Packer default ones also available
to post-processors, your builder will need to add them to its Artifact.
This can be done by adding an attribute of type map[string]interface{} to the
Artifact and putting the generated data in it. The post-processor will access
this data later via the Artifact's State method.
The Artifact code should be implemented similar to the below:
type Artifact struct {
// ...
// StateData should store data such as GeneratedData
// to be shared with post-processors
StateData map[string]interface{}
}
// ...
func (a *Artifact) State(name string) interface{} {
return a.StateData[name]
}
// ...The builder should return the above Artifact containing the generated data and the code should be similar to the example snippet below:
func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) {
// ...
return &Artifact{
// ...
StateData: map[string]interface{}{"generated_data": state.Get("generated_data")},
}, nil
}The code above assigns the generated_data state to the StateData map with
the key generated_data.
Here some example of how this data will be used by post-processors:
func (p *PostProcessor) PostProcess(ctx context.Context, ui packer.Ui, source packer.Artifact) (packer.Artifact, bool, bool, error) {
generatedData := source.State("generated_data")
// generatedData will then be used for interpolation
// ...
}This page has focused up until now on the implementation details for the Builder interface. You will need to create a server and store the builder in a binary in order to make it available to the Packer core as a plugin. We have created a scaffolding repo to give you an idea of the relationship between the builder implementation and the server implementation within a repository, and then read basics of how Plugins work, which breaks down all the server details.