Creating new ebs volume is taking forever

[erlingwl]

Hi,

I'm using v0.1.0, trying to add an EBS volume, this is my "data" data bag:

{
  "id": "data",
  "_default": {
    "devices" : {
      "/dev/sda2" : {
        "file_system"      : "ext3",
        "mount_options"    : "rw,user",
        "mount_path"       : "/usr/local/var/data/elasticsearch/disk1",
        "format_command"   : "mkfs.ext3",
        "fs_check_command" : "dumpe2fs",
        "ebs"            : {
          "size"                  : 10,
          "delete_on_termination" : false,
          "type"                  : "io1",
          "iops"                  : 100
        }
      }
    }
  }
}

I'm runnig a m1.small instance.

From the logs:

[2013-01-30T09:41:24+00:00] INFO: Processing chef_gem[fog] action install (elasticsearch::ebs line 13)
[2013-01-30T09:41:24+00:00] INFO: Processing ruby_block[Create EBS volume on /dev/sda2 (size: 10GB)] action create (elasticsearch::ebs line 14)
[2013-01-30T09:41:29+00:00] INFO: Attaching volume: vol-SOMEID

Then just . . . . . forever (1 hour+ and counting)

Is it the formatting that takes a long time? I've only specified a 10 GB volume. Is it because I'm using only a m1.small? Is something wrong in my settings, I can see that the format command is specified with -F in the vagrant file in the source code..

Thanks,
Erling

[karmi]

Hi, the data bag format seems legit to me, and it certainly shouldn't take neither too long to attach the volume, nor to format it. But this morning AWS is having some issues in eu-west, what region is this?

//cc @vhyza, can you look at this, please?

[erlingwl]

Hi,

It's in us-east. I can see the new volume in the AWS console. But chef just loops forever.

Thanks,
Erling

[erlingwl]

It even says in the AWS console that it is attached correctly:

Attachment: i-NODEID (node-name):/dev/sda2 (attached)

But I can't see it with 'ls /dev/' on the box

[erlingwl]

Probably related to this: http://stackoverflow.com/questions/6986737/aws-ebs-attached-but-cant-find-on-instance

I can see a /dev/xvda2 will try to change the values in the data bag. (I'm using Ubuntu 12.04 LTS)

[karmi]

@erlingwl Yes, the naming can differ, but usually /dev/sda* is just a symlink to /dev/xvda*... Can you see some device file like that in /dev/*?

[erlingwl]

@karmi no, I couldn't see any /dev/sda* symlinks. But changing to "/dev/xvdf" seemed to do the trick.

[karmi]

Thanks for the report and for the link, quoting for other people and Google here:

Some kernels use /dev/xvd* instead of /dev/sd*. In your case, if you have attached the volume with device name
/dev/sdh it would appear as /dev/xvdh in the machine.