Include line number and file name of template when rendering fails

[drich10]

ubuntu@pivotal-ops-manager:~$ bosh-init deploy vcloud-bosh-init.yml
Deployment manifest: '/home/ubuntu/vcloud-bosh-init.yml'
Deployment state: '/home/ubuntu/vcloud-bosh-init-state.json'

Started validating
  Validating release 'bosh'... Finished (00:00:02)
  Validating release 'bosh-vcloud-cpi'... Finished (00:00:01)
  Validating cpi release... Finished (00:00:00)
  Validating deployment manifest... Finished (00:00:00)
  Validating stemcell... Finished (00:01:15)
Finished validating (00:01:19)

Started installing CPI
  Compiling package 'ruby_vcloud_cpi/c7a2acd01b70350f60fb302c1642007a261922c8'... Finished (00:02:03)
  Compiling package 'bosh_vcloud_cpi_mkisofs/437eb06f189dc999e37c5c79aacd6cea3d709a17'... Finished (00:02:52)
  Compiling package 'bosh_vcloud_cpi/0ec91d7b30b38ccb9fe0fc444e147419759bd4c8'... Finished (00:01:11)
  Installing packages... Finished (00:00:01)
  Rendering job templates... Failed (00:00:03)
Failed installing CPI (00:06:11)

Command 'deploy' failed:
  Installing CPI:
    Rendering and uploading Jobs:
      Rendering job templates for installation:
        Rendering templates for job 'cpi/ec7455b0e690039ce09aaa93da7fd98bdba6a41d':
          Rendering template src: cpi.erb, dst: bin/cpi:
            Rendering template src: /home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/bosh-init-release890589487/extracted_jobs/cpi/templates/cpi.erb, dst: /home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/rendered-jobs941588650/bin/cpi:
              Running ruby to render templates:
                Running command: 'ruby /home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/erb-renderer705226241/erb-render.rb /home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/erb-renderer705226241/erb-context.json /home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/bosh-init-release890589487/extracted_jobs/cpi/templates/cpi.erb /home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/rendered-jobs941588650/bin/cpi', stdout: '', stderr: '/home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/erb-renderer705226241/erb-render.rb:77:in `[]': no implicit conversion of String into Integer (TypeError)
    from /home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/erb-renderer705226241/erb-render.rb:77:in `block in copy_property'
    from /home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/erb-renderer705226241/erb-render.rb:76:in `each'
    from /home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/erb-renderer705226241/erb-render.rb:76:in `copy_property'
    from /home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/erb-renderer705226241/erb-render.rb:34:in `block in initialize'
    from /home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/erb-renderer705226241/erb-render.rb:33:in `each'
    from /home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/erb-renderer705226241/erb-render.rb:33:in `initialize'
    from /home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/erb-renderer705226241/erb-render.rb:188:in `new'
    from /home/ubuntu/.bosh_init/installations/a31d3e84-2f7a-4da4-670d-1e9b9fc84bd0/tmp/erb-renderer705226241/erb-render.rb:188:in `<main>'

The issue was with the jobs:properties:vcd:entities section in our vcloud manifest (it was an array of hashes instead of just a hash of hashes. Ideally the stack trace would lead to somewhere in the template instead of the erb-renderer helper.

[allomov]

Did you try to run with BOSH_INIT_LOG_LEVEL=DEBUG env variable? It should output the sources of template and erb renderer. This may help you to reproduce the error.

I would suggest you to double check all properties that you have in your manifest, because this is the most likely reason.

[allomov]

Your stacktrace tells no implicit conversion of String into Integer (TypeError) and it's hard to tell what's wrong at the moment. Could you share your manifest of output of bosh-init with DEBUG log level?

[cppforlife]

@allomov yeah problem was with the format but we can do better at showing line numbers just like Director does.

[sbyiudxv]

bosh-init deploy bosh.yml
Deployment manifest: '/root/my-bosh/bosh.yml'
Deployment state: '/root/my-bosh/bosh-state.json'

Started validating
Downloading release 'bosh'... Skipped [Found in local cache] (00:00:00)
Validating release 'bosh'... Finished (00:00:04)
Downloading release 'bosh-openstack-cpi'... Skipped [Found in local cache] (00:00:00)
Validating release 'bosh-openstack-cpi'... Finished (00:00:00)
Validating cpi release... Finished (00:00:00)
Validating deployment manifest... Finished (00:00:00)
Downloading stemcell... Skipped [Found in local cache] (00:00:00)
Validating stemcell... Finished (00:00:04)
Finished validating (00:00:09)

Started installing CPI
Compiling package 'ruby_openstack_cpi/6f724febcae828dcbfea470faff5b104d48dd944'... Finished (00:00:00)
Compiling package 'bosh_openstack_cpi/119de5f71bee352cabba8575aa97fc993b974f56'... Finished (00:00:00)
Installing packages... Finished (00:00:01)
Rendering job templates... Failed (00:00:00)
Failed installing CPI (00:00:01)

Command 'deploy' failed:
Installing CPI:
Rendering and uploading Jobs:
Rendering job templates for installation:
Rendering templates for job 'openstack_cpi/ac48c4be17b995d915b4888a70d6efdff7b501c5':
Rendering template src: cpi.json.erb, dst: config/cpi.json:
Rendering template src: /root/.bosh_init/installations/5f9d8138-14f8-41de-58ac-8db9c07fd77e/tmp/bosh-init-release866400224/extracted_jobs/openstack_cpi/templates/cpi.json.erb, dst: /root/.bosh_init/installations/5f9d8138-14f8-41de-58ac-8db9c07fd77e/tmp/rendered-jobs875359508/config/cpi.json:
Running ruby to render templates:
Running command: 'ruby /root/.bosh_init/installations/5f9d8138-14f8-41de-58ac-8db9c07fd77e/tmp/erb-renderer257312614/erb-render.rb /root/.bosh_init/installations/5f9d8138-14f8-41de-58ac-8db9c07fd77e/tmp/erb-renderer257312614/erb-context.json /root/.bosh_init/installations/5f9d8138-14f8-41de-58ac-8db9c07fd77e/tmp/bosh-init-release866400224/extracted_jobs/openstack_cpi/templates/cpi.json.erb /root/.bosh_init/installations/5f9d8138-14f8-41de-58ac-8db9c07fd77e/tmp/rendered-jobs875359508/config/cpi.json', stdout: '', stderr: '/root/.bosh_init/installations/5f9d8138-14f8-41de-58ac-8db9c07fd77e/tmp/erb-renderer257312614/erb-render.rb:189:in rescue in render': Error filling in template '/root/.bosh_init/installations/5f9d8138-14f8-41de-58ac-8db9c07fd77e/tmp/bosh-init-release866400224/extracted_jobs/openstack_cpi/templates/cpi.json.erb' for openstack_cpi/0 (line 99: #<TemplateEvaluationContext::UnknownProperty: Can't find property 'nats.user'>) (RuntimeError) from /root/.bosh_init/installations/5f9d8138-14f8-41de-58ac-8db9c07fd77e/tmp/erb-renderer257312614/erb-render.rb:175:in render'
from /root/.bosh_init/installations/5f9d8138-14f8-41de-58ac-8db9c07fd77e/tmp/erb-renderer257312614/erb-render.rb:200:in `

'
':
exit status 1

[dpb587-pivotal]

It is there, but it is hiding. openstack_cpi/templates/cpi.json.erb' for openstack_cpi/0 (line 99: https://github.com/cloudfoundry-incubator/bosh-openstack-cpi-release/blob/v31/jobs/openstack_cpi/templates/cpi.json.erb#L99

…

-- Danny Berger