Error 450001: Action Failed get_task:Task xxxxxxxxx result: 1 of 1 post-start scripts failed. Failed Jobs: uaa.

[joshnn]

'bosh deploy' fails at the very end with the error. Below is 'bosh vms' at the failure. I have no idea what to check next. Is there anything that I should check?

+---------------------------------------------------------------------------+---------+-----+-----------+-------------+
| VM | State | AZ | VM Type | IPs |
+---------------------------------------------------------------------------+---------+-----+-----------+-------------+
| api_worker_z1/0 (-1db0-4c34-8295-6090a0d70096) | running | n/a | small_z1 | 10.10.17.4 |
| api_worker_z2/0 (-41a6-4caa-823a-f9a950f637b8) | running | n/a | small_z2 | 10.10.81.2 |
| api_z1/0 (-bba2-487f-bb77-2e6e15bd090e) | running | n/a | large_z1 | 10.10.17.2 |
| api_z2/0 (-f32c-4ebe-b410-a40c05c69ad3) | running | n/a | large_z2 | 10.10.81.1 |
| clock_global/0 (-fdd3-43c0-9bc9-7d59ea594b62) | running | n/a | medium_z1 | 10.10.17.3 |
| consul_z1/0 (-7f44-4a8e-8487-f58a85a0d12c) | running | n/a | small_z1 | 10.10.16.37 |
| consul_z1/1 (-acad-4723-83de-dec77d6863d0) | running | n/a | small_z1 | 10.10.16.38 |
| consul_z2/0 (-b362-4b91-9f31-463bd0206de2) | running | n/a | small_z2 | 10.10.80.37 |
| doppler_z1/0 (-9c0b-4048-9ecd-3491922aeeee) | running | n/a | medium_z1 | 10.10.17.7 |
| doppler_z2/0 (-db6d-412a-9754-1c0057179ed7) | running | n/a | medium_z2 | 10.10.81.5 |
| etcd_z1/0 (-923e-400f-b860-31964e4b5146) | running | n/a | medium_z1 | 10.10.16.20 |
| etcd_z1/1 (-60c9-423b-a7de-c937cb8954f3) | running | n/a | medium_z1 | 10.10.16.35 |
| etcd_z2/0 (-d5d1-4b0e-b496-610a6e092619) | running | n/a | medium_z2 | 10.10.80.19 |
| hm9000_z1/0 (-12d4-46bd-bd00-1c2735ef560e) | running | n/a | medium_z1 | 10.10.17.5 |
| hm9000_z2/0 (-fe26-4f0f-986a-b7209bd46ada) | running | n/a | medium_z2 | 10.10.81.3 |
| loggregator_trafficcontroller_z1/0 (-1049-4864-9dd3-de03057f2833) | running | n/a | small_z1 | 10.10.17.8 |
| loggregator_trafficcontroller_z2/0 (-5d8f-4518-8428-5dc4893e966a) | running | n/a | small_z2 | 10.10.81.6 |
| nats_z1/0 (-f304-4a4d-a91c-cf4f19379511) | running | n/a | medium_z1 | 10.10.16.11 |
| nats_z2/0 (-a8fc-440e-84c0-7afa9a796a55) | running | n/a | medium_z2 | 10.10.80.11 |
| router_z1/0 (-cd6c-4100-be56-72b47a822b18) | running | n/a | router_z1 | 10.10.16.15 |
| router_z2/0 (-c96d-45f8-93a6-5b5c77044c52) | running | n/a | router_z2 | 10.10.80.15 |
| runner_z1/0 (-25d1-437c-9df1-038cc49f94ce) | running | n/a | runner_z1 | 10.10.17.6 |
| runner_z2/0 (-ebc6-425d-b604-5344e2cf4351) | running | n/a | runner_z2 | 10.10.81.4 |
| stats_z1/0 (-b716-46c1-a7a6-c540d8495ae5) | running | n/a | small_z1 | 10.10.17.0 |
| uaa_z1/0 (-e4ba-49c1-aed5-59f2a2101729) | failing | n/a | medium_z1 | 10.10.17.1 |
| uaa_z2/0 (-8d9f-4a2c-b799-26469ead7bc4) | running | n/a | medium_z2 | 10.10.81.0 |
+---------------------------------------------------------------------------+---------+-----+-----------+-------------+

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this. You can view the current status of your issue at: https://www.pivotaltracker.com/story/show/117855587.

[mbhave]

@joshnn this happens when the uaa job fails to startup correctly, possibly due to a misconfiguration. If you look at the uaa_z1 VM, it says it's failing. Can you send us the logs for that job? You can do this either by:
ssh-ing into the vm itself. The logs will be under /var/vcap/sys/log/uaa.
OR
bosh logs uaa_z1 will download the logs for you.
The log file we're interested in seeing is the /var/vcap/sys/log/uaa/uaa.log.

[joshnn]

Now I am stack at restarting all jobs (cloudfoundry/bosh#1227). I tend to get a new issue whenever I start over and I won't be able to reproduce the original problem anymore.. I would like to keep this open til the new one resolves, which will allow me to resume this.

[joshnn]

Hi mbhave, thanks for explaining how to get the log. I used that on my other problem.

[kikiya]

I'm having this same exact problem. I see this is closed but did anyone ever figure this out? Whether I try to start all vms or only the failed one, I get this error. The other jobs start without issue.

I used the bosh logs command and got the full logs download. I've attached the entire log, but the interesting, or so I think, snippet is below.

uaa.log.zip

[2016-12-03 03:59:29.562] uaa - 10112 [localhost-startStop-1] .... WARN --- XmlWebApplicationContext: Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanDefinitionStoreException: Invalid bean definition with name 'identityZoneConfigurationBootstrap' defined in ServletContext resource [/WEB-INF/spring-servlet.xml]: Could not resolve placeholder 'login.serviceProviderKey' in string value "${login.serviceProviderKey}"; nested exception is java.lang.IllegalArgumentException: Could not resolve placeholder 'login.serviceProviderKey' in string value "${login.serviceProviderKey}" [2016-12-03 03:59:29.567] uaa - 10112 [localhost-startStop-1] .... ERROR --- DispatcherServlet: Context initialization failed org.springframework.beans.factory.BeanDefinitionStoreException: Invalid bean definition with name 'identityZoneConfigurationBootstrap' defined in ServletContext resource [/WEB-INF/spring-servlet.xml]: Could not resolve placeholder 'login.serviceProviderKey' in string value "${login.serviceProviderKey}"; nested exception is java.lang.IllegalArgumentException: Could not resolve placeholder 'login.serviceProviderKey' in string value "${login.serviceProviderKey}" at org.springframework.beans.factory.config.PlaceholderConfigurerSupport.doProcessProperties(PlaceholderConfigurerSupport.java:223)

[fhanik]

you have to provide the properties

login:
  saml:
    serviceProviderKey:
    serviceProviderKeyPassword
    serviceProviderCertificate

[romswo]

Please note you will need to explicitly define a private key password property even if it is empty i.e.:
serviceProviderKeyPassword: ""
See known issues in https://github.com/cloudfoundry/cf-release/releases/tag/v249

[harshalk91]

@romswo
How to generate these keys?

login:
  saml:
    serviceProviderKey:
    serviceProviderKeyPassword
    serviceProviderCertificate

[romswo]

@harshalk91
The simplest way is to use the script included in Diego BOSH release:

$DIEGO_RELEASE_DIR/scripts/generate-uaa-saml-certs

Full description is here:
https://github.com/cloudfoundry/diego-release/tree/develop/examples/aws#configuring-security