amazon-ebssurrogate not able to lookup source volume id when using spot instances

[cread]

Overview of the Issue

When using spot instances with the amazon-ebssurrogate builder, the builder is not able to lookup the id of the source volume(s). This appears to be because it issues the describe instance call to AWS after it has stopped the EC2 instance, and not before.

Reproduction Steps

Try and use spot instances with amazon-ebssurrogate, you'll get the following error regardless of the value of delete_on_termination in the launch_block_device_mappings config:

==> amazon-ebssurrogate: 1 error occurred:
==> amazon-ebssurrogate: 	* Volume ID for device /dev/xvdf not found
==> amazon-ebssurrogate:
==> amazon-ebssurrogate:

Simply replacing the spot_* configuration options with instance_type works.

Packer version

Packer v1.5.4

Simplified Packer Buildfile

Fails:

{
  "builders": [
    {
      "ami_name": "packer debug {{ user `BuildTime` }}",
      "ami_virtualization_type": "hvm",

      "type": "amazon-ebssurrogate",
      "spot_instance_types": [
        "c5.large",
        "c5.xlarge"
      ],
      "spot_price": "auto",

      "launch_block_device_mappings": [
        {
          "volume_type": "gp2",
          "device_name": "{{ user `DeviceName` }}",
          "delete_on_termination": false,
          "volume_size": 20
        }
      ],

      "ami_root_device": {
        "delete_on_termination": true,
        "device_name": "/dev/xvda",
        "source_device_name": "{{ user `DeviceName` }}",
        "volume_size": 20,
        "volume_type": "gp2"
      }
    }
  ]
}

Works:

{
  "builders": [
    {
      "ami_name": "packer debug {{ user `BuildTime` }}",
      "ami_virtualization_type": "hvm",

      "type": "amazon-ebssurrogate",
      "instance_type": "c5.large",

      "launch_block_device_mappings": [
        {
          "volume_type": "gp2",
          "device_name": "{{ user `DeviceName` }}",
          "delete_on_termination": false,
          "volume_size": 20
        }
      ],

      "ami_root_device": {
        "delete_on_termination": true,
        "device_name": "/dev/xvda",
        "source_device_name": "{{ user `DeviceName` }}",
        "volume_size": 20,
        "volume_type": "gp2"
      }
    }
  ]
}

Operating system and Environment details

Tested on OSX and Linux.

Log Fragments and crash.log files

2020/02/20 14:07:52 ui: ==> amazon-ebssurrogate: Connected to SSH!
2020/02/20 14:07:52 packer-builder-amazon-ebssurrogate plugin: Running the provision hook
2020/02/20 14:07:52 ui error: ==> amazon-ebssurrogate: 1 error occurred:
==> amazon-ebssurrogate: 	* Volume ID for device /dev/xvdf not found
==> amazon-ebssurrogate:
==> amazon-ebssurrogate:
2020/02/20 14:07:52 ui: ==> amazon-ebssurrogate: Provisioning step had errors: Running the cleanup provisioner, if present...
2020/02/20 14:07:52 ui: ==> amazon-ebssurrogate: Terminating the source AWS instance...
2020/02/20 14:09:10 ui: ==> amazon-ebssurrogate: Cleaning up any extra volumes...
2020/02/20 14:09:11 ui: ==> amazon-ebssurrogate: No volumes to clean up, skipping
2020/02/20 14:09:11 [INFO] (telemetry) ending amazon-ebssurrogate
2020/02/20 14:09:11 ui error: Build 'amazon-ebssurrogate' errored: 1 error occurred:
	* Volume ID for device /dev/xvdf not found


2020/02/20 14:09:11 machine readable: error-count []string{"1"}
2020/02/20 14:09:11 ui error:
==> Some builds didn't complete successfully and had errors:
2020/02/20 14:09:11 machine readable: amazon-ebssurrogate,error []string{"1 error occurred:\n\t* Volume ID for device /dev/xvdf not found\n\n"}
2020/02/20 14:09:11 ui error: --> amazon-ebssurrogate: 1 error occurred:
	* Volume ID for device /dev/xvdf not found

2020/02/20 14:09:11 ui:
==> Builds finished but no artifacts were created.
2020/02/20 14:09:11 Cancelling builder after context cancellation context canceled

[SwampDragons]

Hi there, I was able to reproduce and trace this to a setting we're accidentally setting when we launch the spot devices. It's fixed in the PR #8945, and you can find patched builds on our CI server at https://circleci.com/gh/hashicorp/packer/41230#artifacts/containers/0

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.