How are external Packer plugins supposed to configure the communicator?

[tintoy]

Hi.

I've been working on a "builder" plugin for Packer, but I've hit a bit of a snag when it comes to running provisioners. I'm trying to reuse StepConnect (hopefully I'm missing something obvious) to set up the communicator, but I can't see how to make this work.

The problem seems to be that since Packer vendors all its dependencies (including Go's SSH library) there is no way I can supply a satisfactory function signature:

import (
	gossh "golang.org/x/crypto/ssh"
)

// ...

&communicator.StepConnect{
	Config: &builder.settings.CommunicatorConfig,
	Host: func(state multistep.StateBag) (host string, err error) {
		settings := state.Get("settings").(*config.Settings)
		host = settings.CommunicatorConfig.Host()

		return
	},
	SSHConfig: func(state multistep.StateBag) (clientConfig *gossh.ClientConfig, err error) {
		settings := state.Get("settings").(*config.Settings)
		clientConfig = &gossh.ClientConfig{
			User: settings.CommunicatorConfig.SSHUsername,
			Auth: []gossh.AuthMethod{
				gossh.Password(settings.CommunicatorConfig.SSHPassword),
			},
		}

		return
	},
},

Since Packer has vendored gossh, Go 1.7 complains that SSHConfig has the wrong signature. It says that it wants the vendored gossh from Packer (github.com/mitchellh/packer/vendor/golang.org/x/crypto/ssh) rather than the only one Go will actually allow me to use (golang.org/x/crypto/ssh).

I tried searching old issues for information about this, but the only similar issue doesn't really seem to provide a useful solution.

Any suggestions would be appreciated :)

[rickard-von-essen]

Interesting, maybe no one tried this since we started to vendor deps.

[tintoy]

FWIW, it works fine if I build against v0.8 (pre-vendoring)...

[mwhooker]

what happens if you do

import (
	gossh "github.com/mitchellh/packer/vendor/golang.org/x/crypto/ssh"
)

[tintoy]

package github.com/DimensionDataResearch/packer-plugins-ddcloud/builders/customerimage
	imports github.com/mitchellh/packer/vendor/golang.org/x/crypto/ssh: use of vendored package not allowed

This is Go v1.7.3, BTW. I think they stopped permitting recursive vendoring in v1.6 or v1.7, but I'm not really sure when...

[tintoy]

Explained here; I believe it may be related to prometheus/prometheus#1720, if that helps - problem is that while Packer is an app, some of its components are more like libraries (and vendoring seems to cause problems when libraries do it).

[mwhooker]

unfortunately I think the only solution is to build the plugin from inside the packer source. If you can get it to work that way, I'll update the docs

[cbednarski]

@tintoy You should be able to work around this by building your plugin directly in the packer source tree, just like the official plugins are built. This is a bit inconvenient for checking your code into a separate git project but it should let you use packer's vendored libs as "normal" gopath libraries instead of having to reach into a vendor subpath to get them.

If you don't want to do this (I understand it's a bit complicated to start a separate repo) another option is to delete packer's vendor folder and specify the versions you need in your plugin. However, you'll have to make sure you use compatible versions.

The reason why this happens is that when building your plugin as a separate binary, that binary has its own vendor folder separate from packer's. These two sets of vendor folders have unique paths and Go treats the dependencies as unique types.

It would be convenient for the Go compiler to elide the external vendor (packer's vendor, in your case) but the Go tool does not properly handle this case right now. The way to force this is simply to delete packer's vendor folder when you do your build.

We unfortunately run into the same problem with serf, which is both a command-line and a library. We solve this by vendoring serf in consul and at the same time stripping serf's default vendor folder.

[tintoy]

Hmm, I'll have a think about this, you've given me good food for thought. Thanks :)

[SwampDragons]

@azr do you think this is something that could be solved now that we're moving to go mod?

[azr]

Hey there, my first guess would be yes, if two projects use the same go module with the same signature, then this error should be gone.

I'll try to reproduce and confirm.

[azr]

I confirm, this issue is already fixed on master.

~/go/src/github.com/hashicorp/potatobuilder
❯ GO111MODULE=off go run .
# github.com/hashicorp/potatobuilder
./main.go:11:3: cannot use func literal (type func(multistep.StateBag) (*"golang.org/x/crypto/ssh".ClientConfig, error)) as type func(multistep.StateBag) (*"github.com/hashicorp/packer/vendor/golang.org/x/crypto/ssh".ClientConfig, error) in field value

~/go/src/github.com/hashicorp/potatobuilder
❯ GO111MODULE=on go run . 

~/go/src/github.com/hashicorp/potatobuilder
❯

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.