New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deep_merge does not work for consul data plugin #50

Closed
thorstenhuhn opened this Issue Mar 7, 2017 · 3 comments

Comments

Projects
None yet
3 participants
@thorstenhuhn
Copy link
Contributor

thorstenhuhn commented Mar 7, 2017

I have the following setup:

common.yaml (especially deep_merge set to true):

data_sources: [ "defaults", "consul" ]
template_sources: [ "file" ]
deep_merge: true

defaults:

  global:
    basic: basic_from_defaults
    nested:
      value: nested_from_defaults

  test.erb:
    target: test.txt

consul:
  url: 'http://localhost:8500'
  values:
    global: '/tiller/globals/all'
    per_env: '/tiller/globals/%e'
    template: '/tiller/values/%e/%t'
    target: '/tiller/target_values/%t/%e'

templates/test.erb:

basic_value: <%= basic %>
nested_value: <%= nested['value'] %>

fill consul key/value store:

#!/bin/bash
CONSUL_URL=http://localhost:8500
curl -X PUT -d @- $CONSUL_URL/v1/kv/tiller/globals/development/basic <<< "basic_from_consul"
curl -X PUT -d @- $CONSUL_URL/v1/kv/tiller/globals/development/nested/value <<< "nested_from_consul"

tiller run:

$ tiller -b . -d
tiller v1.0.0 (https://github.com/markround/tiller) <github@markround.com>
Executable: /Library/Ruby/Gems/2.0.0/gems/tiller-1.0.0/bin/tiller
Using common.yaml v2 format configuration file
Using configuration from .
Using plugins from /usr/local/lib/tiller
Using environment development
Template sources loaded [FileTemplateSource]
Data sources loaded [DefaultsDataSource, ConsulDataSource]
#<DefaultsDataSource:0x007fc75983c110> : Using values from v2 format common.yaml
#<DefaultsDataSource:0x007fc759845e68> : Using values from v2 format common.yaml
#<ConsulDataSource:0x007fc75984d2a8> : Connecting to Consul at http://localhost:8500
#<ConsulDataSource:0x007fc759866bb8> : Connecting to Consul at http://localhost:8500
#<ConsulDataSource:0x007fc759866bb8> : Fetching globals from /tiller/globals/all
#<ConsulDataSource:0x007fc759866bb8> : Fetching per-environment globals from /tiller/globals/development
#<ConsulDataSource:0x007fc759866bb8> : Fetching key tiller/globals/development/basic
#<ConsulDataSource:0x007fc759866bb8> : Fetching key tiller/globals/development/nested/value
Available templates : ["test.erb"]
#<DefaultsDataSource:0x007fc759030070> : Using values from v2 format common.yaml
#<DefaultsDataSource:0x007fc75b8d3ba8> : Using values from v2 format common.yaml
#<ConsulDataSource:0x007fc75b8d34f0> : Connecting to Consul at http://localhost:8500
#<ConsulDataSource:0x007fc75b8d2ac8> : Connecting to Consul at http://localhost:8500
#<ConsulDataSource:0x007fc75b8d2ac8> : Fetching globals from /tiller/globals/all
#<ConsulDataSource:0x007fc75b8d2ac8> : Fetching per-environment globals from /tiller/globals/development
#<ConsulDataSource:0x007fc75b8d2ac8> : Fetching key tiller/globals/development/basic
#<ConsulDataSource:0x007fc75b8d2ac8> : Fetching key tiller/globals/development/nested/value
#<ConsulDataSource:0x007fc75b8d34f0> : Fetching template values from /tiller/values/development/test.erb
#<ConsulDataSource:0x007fc75b8d34f0> : Fetching template target values from /tiller/target_values/test.erb/development
Building template test.erb
Not running as root, so not setting ownership/permissions on test.txt
Template generation completed

Resulting test.txt has basic_value from consul but nested_value from defaults

basic_value: basic_from_consul
nested_value: nested_from_defaults

Consul.log shows that the nested value is retrieved in the expected way:

2017/03/07 20:02:13 [DEBUG] http: Request GET /v1/kv/tiller/globals/all?dc=dc1&keys (69.048µs) from=172.17.0.1:39678
2017/03/07 20:02:13 [DEBUG] http: Request GET /v1/kv/tiller/globals/development?dc=dc1&keys (84.184µs) from=172.17.0.1:39680
2017/03/07 20:02:13 [DEBUG] http: Request GET /v1/kv/tiller/globals/development/basic?dc=dc1 (155.004µs) from=172.17.0.1:39682
2017/03/07 20:02:13 [DEBUG] http: Request GET /v1/kv/tiller/globals/development/nested/value?dc=dc1 (85.62µs) from=172.17.0.1:39684
2017/03/07 20:02:13 [DEBUG] http: Request GET /v1/kv/tiller/globals/all?dc=dc1&keys (67.771µs) from=172.17.0.1:39686
2017/03/07 20:02:13 [DEBUG] http: Request GET /v1/kv/tiller/globals/development?dc=dc1&keys (82.663µs) from=172.17.0.1:39688
2017/03/07 20:02:13 [DEBUG] http: Request GET /v1/kv/tiller/globals/development/basic?dc=dc1 (76.1µs) from=172.17.0.1:39690
2017/03/07 20:02:13 [DEBUG] http: Request GET /v1/kv/tiller/globals/development/nested/value?dc=dc1 (84.972µs) from=172.17.0.1:39692
2017/03/07 20:02:13 [DEBUG] http: Request GET /v1/kv/tiller/values/development/test.erb?dc=dc1&keys (66.934µs) from=172.17.0.1:39694
2017/03/07 20:02:13 [DEBUG] http: Request GET /v1/kv/tiller/values/development/test.erb?dc=dc1&keys (85.853µs) from=172.17.0.1:39696
2017/03/07 20:02:13 [DEBUG] http: Request GET /v1/kv/tiller/target_values/test.erb/development?dc=dc1&keys (100.774µs) from=172.17.0.1:39698

Debugging fetch request for nested element

$ curl -s http://localhost:8500/v1/kv/tiller/globals/development/nested/value?raw
nested_from_consul

Expected result for test.txt:

basic_value: basic_from_consul
nested_value: nested_from_consul
@markround

This comment has been minimized.

Copy link
Owner

markround commented Mar 8, 2017

Thanks for the detailed issue report! I'll take a look at this and get back to you.

@markround markround self-assigned this Mar 8, 2017

@EugenMayer

This comment has been minimized.

Copy link

EugenMayer commented Mar 8, 2017

@markround AFAICS this has to do with the way diplomat works in the old ways, you should use the new method so it extracts to a hashmap, instead of a flat key/value store

thorstenhuhn added a commit to thorstenhuhn/tiller that referenced this issue Apr 19, 2017

markround added a commit that referenced this issue Apr 20, 2017

@markround

This comment has been minimized.

Copy link
Owner

markround commented Apr 20, 2017

Thanks for the PR, @thorstenhuhn . This will make it into 1.1.0.

@markround markround closed this Apr 24, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment