bosh cf deploy assumes us-east-1

[marcus233]

i set my microbosh up on aws us west 2 (oregon). bosh cf prepare system installed bosh on us-west-2 as well.

upon running bosh cf deploy, i receive this error:

vcap@:~$ bosh cf deploy
Current CloudFoundry system is '/var/vcap/store/systems/production'
Deployment set to `/var/vcap/store/systems/production/deployments/production-core.yml'
bosh -n --color deploy
Getting deployment properties from director...
Unable to get properties list from director, trying without it...
Compiling deployment manifest...

Director task 3

Preparing deployment
  binding deployment (00:00:00)                                                                     
  binding releases (00:00:00)                                                                       
  binding existing deployment (00:00:00)                                                            
  binding resource pools (00:00:00)                                                                 
  binding stemcells (00:00:00)                                                                      
  binding templates (00:00:00)                                                                      
  binding properties (00:00:00)                                                                     
  binding unallocated VMs (00:00:00)                                                                
  binding instance networks (00:00:00)                                                              
Done                    9/9 00:00:00                                                                

Preparing package compilation
  finding packages to compile (00:00:00)                                                            
Done                    1/1 00:00:00                                                                

Compiling packages
  dea_node04/3: Invalid availability zone: [us-east-1a] (00:00:01)                                  
  dea_node06/3: Invalid availability zone: [us-east-1a] (00:00:01)                                  
  dea_jvm7/1: Invalid availability zone: [us-east-1a] (00:00:01)                                    
  redis22/1: Invalid availability zone: [us-east-1a] (00:00:01)                                     
  dea_node08/1: Invalid availability zone: [us-east-1a] (00:00:01)                                  
  dea_ruby19/7: Invalid availability zone: [us-east-1a] (00:00:01)                                  
  dea_jvm/3: Invalid availability zone: [us-east-1a] (00:00:01)                                     
  imagemagick/2: Invalid availability zone: [us-east-1a] (00:00:01)                                 
  dea_ruby18/8: Invalid availability zone: [us-east-1a] (00:00:01)                                  
  git/1: Invalid availability zone: [us-east-1a] (00:00:01)                                         
Error                   10/29 00:00:01                                                              

Error 100: Invalid availability zone: [us-east-1a]

Task 3 error
/usr/local/lib/ruby/1.9.1/rake/file_utils.rb:53:in `block in create_shell_runner': Command failed with status (1): [bosh -n --color deploy...] (RuntimeError)
    from /usr/local/lib/ruby/1.9.1/rake/file_utils.rb:45:in `call'
    from /usr/local/lib/ruby/1.9.1/rake/file_utils.rb:45:in `sh'
    from /usr/local/lib/ruby/gems/1.9.1/gems/bosh-cloudfoundry-0.5.0/lib/bosh/cli/commands/cf.rb:618:in `bosh_cmd'
    from /usr/local/lib/ruby/gems/1.9.1/gems/bosh-cloudfoundry-0.5.0/lib/bosh/cli/commands/cf.rb:153:in `block in deploy'
    from /usr/local/lib/ruby/gems/1.9.1/gems/bosh-cloudfoundry-0.5.0/lib/bosh/cli/commands/cf.rb:151:in `each'
    from /usr/local/lib/ruby/gems/1.9.1/gems/bosh-cloudfoundry-0.5.0/lib/bosh/cli/commands/cf.rb:151:in `deploy'
    from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/lib/cli/command_handler.rb:57:in `run'
    from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/lib/cli/runner.rb:61:in `run'
    from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/lib/cli/runner.rb:18:in `run'
    from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/bin/bosh:16:in `<top (required)>'
    from /usr/local/bin/bosh:23:in `load'
    from /usr/local/bin/bosh:23:in `<main>'

[mrdavidlaing]

Same issue for me; but using bosh-cloudfoundry-0.5.1

[mrdavidlaing]

Same issue for me; but using bosh-cloudfoundry-0.5.1

[drnic]

Firstly, congrats on creating issue #100 :)

[drnic]

Firstly, congrats on creating issue #100 :)

[drnic]

Can either of you paste in your /var/vcap/store/microboshes/deployments/*/micro_bosh.yml?

[drnic]

Can either of you paste in your /var/vcap/store/microboshes/deployments/*/micro_bosh.yml?

[marcus233]

---
name: microbosh-aws-us-west-2
env:
  bosh:
    password: xxxxx
logging:
  level: DEBUG
  network:  type: dynamic
  vip: xxxxxx
resources:  
  persistent_disk: 16384  
  cloud_properties:  
      instance_type: m1.medium
cloud:
  plugin: aws
  properties:    aws:
      access_key_id: xxxxxxxxxxx
      secret_access_key: xxxxxxxxxxxxx
      region: us-west-2
      ec2_endpoint: ec2.us-west-2.amazonaws.com
      default_security_groups:
      - microbosh-aws-us-west-2
      default_key_name: microbosh-aws-us-west-2
      ec2_private_key: /home/vcap/.ssh/microbosh-aws-us-west-2.pem
apply_spec:
  agent:
    blobstore:
      address: xxxxxx
    nats:
      address: xxxxxx
  properties:
    aws_registry:
      address: xxxxxxxxxxx

[marcus233]

---
name: microbosh-aws-us-west-2
env:
  bosh:
    password: xxxxx
logging:
  level: DEBUG
  network:  type: dynamic
  vip: xxxxxx
resources:  
  persistent_disk: 16384  
  cloud_properties:  
      instance_type: m1.medium
cloud:
  plugin: aws
  properties:    aws:
      access_key_id: xxxxxxxxxxx
      secret_access_key: xxxxxxxxxxxxx
      region: us-west-2
      ec2_endpoint: ec2.us-west-2.amazonaws.com
      default_security_groups:
      - microbosh-aws-us-west-2
      default_key_name: microbosh-aws-us-west-2
      ec2_private_key: /home/vcap/.ssh/microbosh-aws-us-west-2.pem
apply_spec:
  agent:
    blobstore:
      address: xxxxxx
    nats:
      address: xxxxxx
  properties:
    aws_registry:
      address: xxxxxxxxxxx

[drnic]

Ok thanks. This looks good to me; but perhaps it doesn't look good to
deployer gem.

Next, if you have time, look in bosh source (say deployer/ project) for
where it might override or default to us-east-1 region. It's ignoring the
region we're requesting!

Else I'll try to look later.

Dr Nic Williams
Stark & Wayne LLC - consultancy for Cloud Foundry users
http://drnicwilliams.com
http://starkandwayne.com
cell +1 (415) 860-2185
twitter @drnic

[drnic]

Ok thanks. This looks good to me; but perhaps it doesn't look good to
deployer gem.

Next, if you have time, look in bosh source (say deployer/ project) for
where it might override or default to us-east-1 region. It's ignoring the
region we're requesting!

Else I'll try to look later.

Dr Nic Williams
Stark & Wayne LLC - consultancy for Cloud Foundry users
http://drnicwilliams.com
http://starkandwayne.com
cell +1 (415) 860-2185
twitter @drnic

[mrdavidlaing]

I've been grepping around looking for us-east-1, but I'm not having any
luck. Grr.

On 12 February 2013 16:32, Dr Nic Williams notifications@github.com wrote:

Ok thanks. This looks good to me; but perhaps it doesn't look good to
deployer gem.

Next, if you have time, look in bosh source (say deployer/ project) for
where it might override or default to us-east-1 region. It's ignoring the
region we're requesting!

Else I'll try to look later.

Dr Nic Williams
Stark & Wayne LLC - consultancy for Cloud Foundry users
http://drnicwilliams.com
http://starkandwayne.com
cell +1 (415) 860-2185
twitter @drnic

—
Reply to this email directly or view it on GitHubhttps://github.com/StarkAndWayne/bosh-cloudfoundry/issues/100#issuecomment-13441281.

David Laing
Open source @ City Index - github.com/cityindex
http://davidlaing.com
Twitter: @davidlaing

[mrdavidlaing]

I've been grepping around looking for us-east-1, but I'm not having any
luck. Grr.

On 12 February 2013 16:32, Dr Nic Williams notifications@github.com wrote:

Ok thanks. This looks good to me; but perhaps it doesn't look good to
deployer gem.

Next, if you have time, look in bosh source (say deployer/ project) for
where it might override or default to us-east-1 region. It's ignoring the
region we're requesting!

Else I'll try to look later.

Dr Nic Williams
Stark & Wayne LLC - consultancy for Cloud Foundry users
http://drnicwilliams.com
http://starkandwayne.com
cell +1 (415) 860-2185
twitter @drnic

—
Reply to this email directly or view it on GitHubhttps://github.com/StarkAndWayne/bosh-cloudfoundry/issues/100#issuecomment-13441281.

David Laing
Open source @ City Index - github.com/cityindex
http://davidlaing.com
Twitter: @davidlaing

[drnic]

In my deployment, which was in us-east-1, so it didn't fail, also put the
core/0 VM into us-east-1a. So perhaps the AZ is being specifically
overridden/defaulted.

[drnic]

In my deployment, which was in us-east-1, so it didn't fail, also put the
core/0 VM into us-east-1a. So perhaps the AZ is being specifically
overridden/defaulted.

[marcus233]

i ended up changing DEFAULT_AVAILABILITY_ZONE in /usr/local/lib/ruby/gems/1.9.1/gems/bosh_aws_cpi-0.7.0/lib/cloud/aws/cloud.rb and got a bit further (then ran into another bug). perhaps this helps? i'll join the grep search in a moment.

[marcus233]

i ended up changing DEFAULT_AVAILABILITY_ZONE in /usr/local/lib/ruby/gems/1.9.1/gems/bosh_aws_cpi-0.7.0/lib/cloud/aws/cloud.rb and got a bit further (then ran into another bug). perhaps this helps? i'll join the grep search in a moment.

[mrdavidlaing]

Could this be related to a similar default_availability_zone bug in bosh-bootstrap ?

Perhaps we should be adding

availability_zone: eu-west-1b to /var/vcap/store/systems/production/deployments/production-core.yml ?

[mrdavidlaing]

Could this be related to a similar default_availability_zone bug in bosh-bootstrap ?

Perhaps we should be adding

availability_zone: eu-west-1b to /var/vcap/store/systems/production/deployments/production-core.yml ?

[drnic]

I'm interested in starting to do that because some AWS users have AZs that
are not available to them; and the BOSH team (martin) won't support a fix
for that.

Also, if we start managing the AZ allocation manually, then we can start
supporting multi-AZ deployments (say 3 dea resource pools, one per AZ, and
allocate the DEA job VMs to them equally).

[drnic]

I'm interested in starting to do that because some AWS users have AZs that
are not available to them; and the BOSH team (martin) won't support a fix
for that.

Also, if we start managing the AZ allocation manually, then we can start
supporting multi-AZ deployments (say 3 dea resource pools, one per AZ, and
allocate the DEA job VMs to them equally).

[mrdavidlaing]

Does it makes sense to split a single CF cluster over multiple AZs?

Wouldn't you run the risk of having a specific app allocated to a router in
one, DEA(s) in another and backend services services in yet another?

I'd assumed you would want them all as close together in network hop terms
as possible.

On Tuesday, February 12, 2013, Dr Nic Williams wrote:

I'm interested in starting to do that because some AWS users have AZs that
are not available to them; and the BOSH team (martin) won't support a fix
for that.

Also, if we start managing the AZ allocation manually, then we can start
supporting multi-AZ deployments (say 3 dea resource pools, one per AZ, and
allocate the DEA job VMs to them equally).

—
Reply to this email directly or view it on GitHubhttps://github.com/StarkAndWayne/bosh-cloudfoundry/issues/100#issuecomment-13453533.

David Laing
Open source @ City Index - github.com/cityindex
http://davidlaing.com
Twitter: @davidlaing

[mrdavidlaing]

Does it makes sense to split a single CF cluster over multiple AZs?

Wouldn't you run the risk of having a specific app allocated to a router in
one, DEA(s) in another and backend services services in yet another?

I'd assumed you would want them all as close together in network hop terms
as possible.

On Tuesday, February 12, 2013, Dr Nic Williams wrote:

I'm interested in starting to do that because some AWS users have AZs that
are not available to them; and the BOSH team (martin) won't support a fix
for that.

Also, if we start managing the AZ allocation manually, then we can start
supporting multi-AZ deployments (say 3 dea resource pools, one per AZ, and
allocate the DEA job VMs to them equally).

—
Reply to this email directly or view it on GitHubhttps://github.com/StarkAndWayne/bosh-cloudfoundry/issues/100#issuecomment-13453533.

David Laing
Open source @ City Index - github.com/cityindex
http://davidlaing.com
Twitter: @davidlaing

[mrdavidlaing]

Aha! It looks like

/var/vcap/store/systems/production/deployments/production-core.yml is missing the region from the compilation::cloud_properties

compilation:
  cloud_properties:  
      instance_type: m1.medium
      region: us-west-2

You might also want to specify the availability_zone: us-west-2b explicitly, to ensure it ends up in the same AZ as your micro-bosh

Manually correcting your production-core.yml with that extra property allows the compilation step to proceed.

[mrdavidlaing]

Aha! It looks like

/var/vcap/store/systems/production/deployments/production-core.yml is missing the region from the compilation::cloud_properties

compilation:
  cloud_properties:  
      instance_type: m1.medium
      region: us-west-2

You might also want to specify the availability_zone: us-west-2b explicitly, to ensure it ends up in the same AZ as your micro-bosh

Manually correcting your production-core.yml with that extra property allows the compilation step to proceed.

[marcus233]

i have no section compilation_workers. adding region: us-west-2 to the cloud properties section under compilation doesn't get me past the error.

[marcus233]

i have no section compilation_workers. adding region: us-west-2 to the cloud properties section under compilation doesn't get me past the error.

[drnic]

He's now talking about the deployment manifest at
/var/vcap/store/systems/NAME/deployments/...

Dr Nic Williams
Stark & Wayne LLC - consultancy for Cloud Foundry users
http://drnicwilliams.com
http://starkandwayne.com
cell +1 (415) 860-2185
twitter @drnic

[drnic]

He's now talking about the deployment manifest at
/var/vcap/store/systems/NAME/deployments/...

Dr Nic Williams
Stark & Wayne LLC - consultancy for Cloud Foundry users
http://drnicwilliams.com
http://starkandwayne.com
cell +1 (415) 860-2185
twitter @drnic

[marcus233]

This file?

vcap@:~$ cat /var/vcap/store/systems/production/deployments/production-core.yml 
--- 
name: production-core
director_uuid: xxxxxxxxxxxxx
release: 
  name: appcloud-staging
  version: latest
compilation: 
  workers: 10
  network: default
  reuse_compilation_vms: true
  cloud_properties: 
    instance_type: m1.medium
update: 
  canaries: 1
  canary_watch_time: 30000-150000
  update_watch_time: 30000-150000
  max_in_flight: 4
  max_errors: 1
networks: 
- name: default
  type: dynamic
  cloud_properties: 
    security_groups: 
    - cloudfoundry-production
...

[marcus233]

This file?

vcap@:~$ cat /var/vcap/store/systems/production/deployments/production-core.yml 
--- 
name: production-core
director_uuid: xxxxxxxxxxxxx
release: 
  name: appcloud-staging
  version: latest
compilation: 
  workers: 10
  network: default
  reuse_compilation_vms: true
  cloud_properties: 
    instance_type: m1.medium
update: 
  canaries: 1
  canary_watch_time: 30000-150000
  update_watch_time: 30000-150000
  max_in_flight: 4
  max_errors: 1
networks: 
- name: default
  type: dynamic
  cloud_properties: 
    security_groups: 
    - cloudfoundry-production
...

[drnic]

That's the one. compilation.cloud_properties is what Dave was referring to
above.

Dr Nic Williams
Stark & Wayne LLC - consultancy for Cloud Foundry users
http://drnicwilliams.com
http://starkandwayne.com
cell +1 (415) 860-2185
twitter @drnic

[drnic]

That's the one. compilation.cloud_properties is what Dave was referring to
above.

Dr Nic Williams
Stark & Wayne LLC - consultancy for Cloud Foundry users
http://drnicwilliams.com
http://starkandwayne.com
cell +1 (415) 860-2185
twitter @drnic

[drnic]

The region will need to be discovered via the micro_bosh.yml; like we currently discover the AWS API keys (to offer to provide automatic IP address provisioning).

[drnic]

The region will need to be discovered via the micro_bosh.yml; like we currently discover the AWS API keys (to offer to provide automatic IP address provisioning).

[drnic]

Specifically, via microbosh_config.rb #fog_connection_properties

[drnic]

Specifically, via microbosh_config.rb #fog_connection_properties

[drnic]

A fix for this (compilation VMs needing a region) is in branch https://github.com/StarkAndWayne/bosh-cloudfoundry/tree/compilation_region

The branch is also working on adding region to all cloud_properties (core, dea, & service jobs).

[drnic]

A fix for this (compilation VMs needing a region) is in branch https://github.com/StarkAndWayne/bosh-cloudfoundry/tree/compilation_region

The branch is also working on adding region to all cloud_properties (core, dea, & service jobs).

[drnic]

@pmenglund @frodenas do know if it recently became necessary for cloud_properties to include a "region" key for AWS? Is it just for the compilation VMs or all resource pools? When was this introduced?

[drnic]

@pmenglund @frodenas do know if it recently became necessary for cloud_properties to include a "region" key for AWS? Is it just for the compilation VMs or all resource pools? When was this introduced?

[pmenglund]

cloudfoundry/bosh@c8e3224

It is for all VMs, as you neet to pass it in the CPI options.

[pmenglund]

cloudfoundry/bosh@c8e3224

It is for all VMs, as you neet to pass it in the CPI options.

[drnic]

Does this mean that BOSH is going to become multi-region? Else why are we
now needing to provide the region information that the AWS CPI already had
from the micro_bosh.yml?

[drnic]

Does this mean that BOSH is going to become multi-region? Else why are we
now needing to provide the region information that the AWS CPI already had
from the micro_bosh.yml?

[pmenglund]

It might become multi region - I'm not making any promises on when...

You might want to have your micro bosh in one region and your bosh in another - my job is to make bosh configurable, yours is to pick good defaults ;)

[pmenglund]

It might become multi region - I'm not making any promises on when...

You might want to have your micro bosh in one region and your bosh in another - my job is to make bosh configurable, yours is to pick good defaults ;)

[drnic]

Sorry, I was under the impression from last year that BOSH can live in and
provision in one region. I am very happy for the lifting of that constraint!

[drnic]

Sorry, I was under the impression from last year that BOSH can live in and
provision in one region. I am very happy for the lifting of that constraint!

[pmenglund]

once I'm done with the fake stemcell I'm working on, you can - we are restricted to the current region when we upload create a stemcell, as it we need to attach an ebs volume to the current instance, but that will go away when I'm done (and we can just reference existing AMIs)

[pmenglund]

once I'm done with the fake stemcell I'm working on, you can - we are restricted to the current region when we upload create a stemcell, as it we need to attach an ebs volume to the current instance, but that will go away when I'm done (and we can just reference existing AMIs)

[drnic]

Ok, so if a user creates a custom stemcell or uses a normal stemcell (for which there is not a fleet of AWS AMI backing it), then we need to set the region to the region of the BOSH. If its AWS, and the chosen stemcell has a set of AMIs mapped to it, then we could prompt for which region to deploy to (based on whats available in the fake stemcell)

[drnic]

Ok, so if a user creates a custom stemcell or uses a normal stemcell (for which there is not a fleet of AWS AMI backing it), then we need to set the region to the region of the BOSH. If its AWS, and the chosen stemcell has a set of AMIs mapped to it, then we could prompt for which region to deploy to (based on whats available in the fake stemcell)

[anderssv]

I think you're aware of this but running bosh cf change deas 1 removed the additional information put in about zone, so you'll need to add it back.

[anderssv]

I think you're aware of this but running bosh cf change deas 1 removed the additional information put in about zone, so you'll need to add it back.

[mrdavidlaing]

There are actually 4 instances of cloud_properties in /var/vcap/store/systems/production/deployments/production-core.yml that need to be corrected.

The following grep shows how the "corrected" sections of your production-core.yml should look:

vcap@ip-10-245-25-35:~$ grep cloud_properties /var/vcap/store/systems/production/deployments/production-core.yml -3
  workers: 10
  network: default
  reuse_compilation_vms: true
  cloud_properties: 
    instance_type: m1.medium
    region: us-west-2
    availability_zone: us-west-2b
--
networks: 
- name: default
  type: dynamic
  cloud_properties: 
    region: us-west-2
    availability_zone: us-west-2b
    security_groups: 
    - cloudfoundry-production
- name: vip_network
  type: vip
  cloud_properties: 
    region: us-west-2
    availability_zone: us-west-2b
    security_groups: 
--
  stemcell: 
    name: bosh-stemcell
    version: 0.7.0
  cloud_properties: 
    instance_type: m1.large
    region: us-west-2
    availability_zone: us-west-2b

[mrdavidlaing]

There are actually 4 instances of cloud_properties in /var/vcap/store/systems/production/deployments/production-core.yml that need to be corrected.

The following grep shows how the "corrected" sections of your production-core.yml should look:

vcap@ip-10-245-25-35:~$ grep cloud_properties /var/vcap/store/systems/production/deployments/production-core.yml -3
  workers: 10
  network: default
  reuse_compilation_vms: true
  cloud_properties: 
    instance_type: m1.medium
    region: us-west-2
    availability_zone: us-west-2b
--
networks: 
- name: default
  type: dynamic
  cloud_properties: 
    region: us-west-2
    availability_zone: us-west-2b
    security_groups: 
    - cloudfoundry-production
- name: vip_network
  type: vip
  cloud_properties: 
    region: us-west-2
    availability_zone: us-west-2b
    security_groups: 
--
  stemcell: 
    name: bosh-stemcell
    version: 0.7.0
  cloud_properties: 
    instance_type: m1.large
    region: us-west-2
    availability_zone: us-west-2b

[nand2]

I confirm the bug; and I confirm the fix in the comment above (eu-west-1 zone).

[nand2]

I confirm the bug; and I confirm the fix in the comment above (eu-west-1 zone).

[qch1843]

Hi,

I had same "invalid availability zone" issue and followed above comments and modified 4 instances of cloud_properties in /var/vcap/store/systems/production/deployments/production-core.yml and re-run the bosh cf deploy, following errors appears, any suggestions?

$ bosh cf deploy
Current CloudFoundry system is '/var/vcap/store/systems/production'
Deployment set to `/var/vcap/store/systems/production/deployments/production-core.yml'
bosh -n --color deploy
Getting deployment properties from director...
Compiling deployment manifest...

Director task 4

Preparing deployment
  binding deployment (00:00:00)
  binding releases (00:00:00)
  binding existing deployment: Timed out sending `get_state' to b68ff6b0-6a2e-45f2-9108-422b9bd56a10 after 30 seconds (00:01:30)
Error               3/9 00:01:30

Error 450002: Timed out sending `get_state' to b68ff6b0-6a2e-45f2-9108-422b9bd56a10 after 30 seconds

Task 4 error
/usr/local/lib/ruby/1.9.1/rake/file_utils.rb:53:in `block in create_shell_runner': Command failed with status (1): [bosh -n --color deploy...] (RuntimeError)
        from /usr/local/lib/ruby/1.9.1/rake/file_utils.rb:45:in `call'
        from /usr/local/lib/ruby/1.9.1/rake/file_utils.rb:45:in `sh'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh-cloudfoundry-0.6.1/lib/bosh/cli/commands/cf.rb:632:in `bosh_cmd'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh-cloudfoundry-0.6.1/lib/bosh/cli/commands/cf.rb:156:in `block in deploy'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh-cloudfoundry-0.6.1/lib/bosh/cli/commands/cf.rb:154:in `each'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh-cloudfoundry-0.6.1/lib/bosh/cli/commands/cf.rb:154:in `deploy'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/lib/cli/command_handler.rb:57:in `run'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/lib/cli/runner.rb:61:in `run'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/lib/cli/runner.rb:18:in `run'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/bin/bosh:16:in `<top (required)>'
        from /usr/local/bin/bosh:23:in `load'
        from /usr/local/bin/bosh:23:in `<main>'

[qch1843]

Hi,

I had same "invalid availability zone" issue and followed above comments and modified 4 instances of cloud_properties in /var/vcap/store/systems/production/deployments/production-core.yml and re-run the bosh cf deploy, following errors appears, any suggestions?

$ bosh cf deploy
Current CloudFoundry system is '/var/vcap/store/systems/production'
Deployment set to `/var/vcap/store/systems/production/deployments/production-core.yml'
bosh -n --color deploy
Getting deployment properties from director...
Compiling deployment manifest...

Director task 4

Preparing deployment
  binding deployment (00:00:00)
  binding releases (00:00:00)
  binding existing deployment: Timed out sending `get_state' to b68ff6b0-6a2e-45f2-9108-422b9bd56a10 after 30 seconds (00:01:30)
Error               3/9 00:01:30

Error 450002: Timed out sending `get_state' to b68ff6b0-6a2e-45f2-9108-422b9bd56a10 after 30 seconds

Task 4 error
/usr/local/lib/ruby/1.9.1/rake/file_utils.rb:53:in `block in create_shell_runner': Command failed with status (1): [bosh -n --color deploy...] (RuntimeError)
        from /usr/local/lib/ruby/1.9.1/rake/file_utils.rb:45:in `call'
        from /usr/local/lib/ruby/1.9.1/rake/file_utils.rb:45:in `sh'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh-cloudfoundry-0.6.1/lib/bosh/cli/commands/cf.rb:632:in `bosh_cmd'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh-cloudfoundry-0.6.1/lib/bosh/cli/commands/cf.rb:156:in `block in deploy'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh-cloudfoundry-0.6.1/lib/bosh/cli/commands/cf.rb:154:in `each'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh-cloudfoundry-0.6.1/lib/bosh/cli/commands/cf.rb:154:in `deploy'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/lib/cli/command_handler.rb:57:in `run'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/lib/cli/runner.rb:61:in `run'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/lib/cli/runner.rb:18:in `run'
        from /usr/local/lib/ruby/gems/1.9.1/gems/bosh_cli-1.0.3/bin/bosh:16:in `<top (required)>'
        from /usr/local/bin/bosh:23:in `load'
        from /usr/local/bin/bosh:23:in `<main>'

[anderssv]

It first failed, then you corrected and ran again? I have never really understood how to recover from failures like this. You could try running bosh cloudcheck and choosing to delete the VM reference. If my understanding is correct this error stems from the fact that BOSH retains a reference to the VM it tries to create, but is unable to. So it will have to remove that reference before running again.

[anderssv]

It first failed, then you corrected and ran again? I have never really understood how to recover from failures like this. You could try running bosh cloudcheck and choosing to delete the VM reference. If my understanding is correct this error stems from the fact that BOSH retains a reference to the VM it tries to create, but is unable to. So it will have to remove that reference before running again.

[qch1843]

Yes, it failed first, and I modified the configuration file and run it again. And it failed with above error. With no other options, I have to start it over from very beginning (boot-bootstrap).

I will try the "bosh cloudcheck" next time...

Thanks!
Steven

[qch1843]

Yes, it failed first, and I modified the configuration file and run it again. And it failed with above error. With no other options, I have to start it over from very beginning (boot-bootstrap).

I will try the "bosh cloudcheck" next time...

Thanks!
Steven

[qch1843]

BTW, in case of a failure, such as bosh cf deploy failed, how can I cleanup the previous deployment and start it from beginning (not from where it left off from last run)?

[qch1843]

BTW, in case of a failure, such as bosh cf deploy failed, how can I cleanup the previous deployment and start it from beginning (not from where it left off from last run)?

[mrdavidlaing]

bosh cloudcheck - and say yes fix any problems

Then bosh delete deployment production-core --force

This should leave you with just your inception vm and your micro bosh. You
can then bosh cf deploy again

On Tuesday, March 19, 2013, Steven wrote:

BTW, in case of a failure, such as bosh cf deploy failed, how can I
cleanup the previous deployment and start it from beginning (not from where
it left off from last run)?

—
Reply to this email directly or view it on GitHubhttps://github.com/StarkAndWayne/bosh-cloudfoundry/issues/100#issuecomment-15100958
.

David Laing
Open source @ City Index - github.com/cityindex
http://davidlaing.com
Twitter: @davidlaing

[mrdavidlaing]

bosh cloudcheck - and say yes fix any problems

Then bosh delete deployment production-core --force

This should leave you with just your inception vm and your micro bosh. You
can then bosh cf deploy again

On Tuesday, March 19, 2013, Steven wrote:

BTW, in case of a failure, such as bosh cf deploy failed, how can I
cleanup the previous deployment and start it from beginning (not from where
it left off from last run)?

—
Reply to this email directly or view it on GitHubhttps://github.com/StarkAndWayne/bosh-cloudfoundry/issues/100#issuecomment-15100958
.

David Laing
Open source @ City Index - github.com/cityindex
http://davidlaing.com
Twitter: @davidlaing