[Discussion] TOFS/pillar.stack/stack-formula for map.jinja #116
Comments
|
Maybe it's a good idea to implement #103 on the way. |
|
This is an incredibly important discussion, so I'm just going to invite the cavalry to start bringing in their opinions (only inviting to the discussion, not forcing to participate). By the way, I reckon we're going have to break this down into multiple stages. There's already a lot going on at a basic level, with the introduction of |
myii
assigned aboe76, javierbertoli, daks, baby-gnu, noelmcloughlin, vutny and n-rodriguez
|
Please don't be offended if I've invited you (when you rather wouldn't have been) or if I've forgotten to invite you (because TL;DR: |
|
Thanks @n-rodriguez, that's prompted me to start the checklist at the top of this issue. |
|
@myii You are allowed to hijack the first comment. 👍 😄 |
Can we really say that?
As I already said before: we (Saltstack itself, community...) don't already have solutions to propose to get out of pillars, so it always seems premature to me to push people out of pillars when we don't have anything to propose. This all [1] Those should not be in the formulas but in salt itself, in my opinion |
|
Hello. I'm seting up of a new infrastructure and I have a little time to think about how to properly do things. I thought quite a lot abouth differents discussions on the maintainability of the formula: In the maintainance of libvirt-formula, one of the problem I have is “how to maintain only the distributions in which I'm interested whithout putting barrier for people to manage others?”. I think we could start by generalizing how
The actual NB: we could groups This will permits:
We could think about adding a The only thing I don't know is how to configure the merging. Do we have access to some salt functions to do that like Regards. |
|
We could do the merge with salt.slsutil.merge which support:
For example, YAML files could look like: strategy: 'overwrite'
merge_lists: 'true'
values:
foo: 1
bar:
- 'a'
- 'b'
- 'c'we then load the YAML files with something like: {%- import_yaml tlproot ~ "/defaults.yaml" as default_settings %}
{%- import_yaml tlproot ~ "/somefile.yaml" as somefile %}
{%- set config = salt.slsutil.merge(default_settings,
somefile.get('values', {}),
strategy=somefile.get('strategy', 'smart'),
merge_lists=somefile.get('merge_lists', False) | to_bool
) %}I made some tests and this is working with |
|
Now, I need a way to make it works even when the file does not exists. |
|
I found a way to handle missing files: {%- import_yaml tlproot ~ "/defaults.yaml" as default_settings %}
{%- set yamlfiles = ["file1.yaml", "file2.yaml", "file3.yaml"] %}
{%- set _config = {'values': default_settings} %}
{%- for yamlfile in yamlfiles %}
{%- load_yaml as loaded_values %}
{%- include yamlfile ignore missing %}
{%- endload %}
{%- if loaded_values %}
{%- do _config.update({'values': salt.slsutil.merge(_config['values'],
loaded_values.get('values', {}),
strategy=loaded_values.get('strategy', 'smart'),
merge_lists=loaded_values.get('merge_lists', False) | to_bool)})
%}
{%- endif %}
{%- endfor %}
{%- set config = _config['values'] %}So, now I'll be able to provide a pull request ;-) |
Currently
map.jinjarelies heavily on Pillar data. This shifts the load to the master which does not scale well. With the recent introduction ofconfig.getby @myii other data sources are accessible¹, but there are some caveats:salt-ssh)@claudekenni implemented a prototype mimicking
pillar.stack: https://github.com/claudekenni/stack-formulaHe also wrote an execution module https://github.com/claudekenni/stack_module which provides
stack.items,stack.getand others,and a SDB module: https://github.com/claudekenni/sdbstack.
How can we take the experience of TOFS,
pillar.stackand @claudekenni's prototypes and move ~80% of our Pillar data onto the minions?salt(ZMQ),salt-call(local) andsalt-ssh(SSH)¹ SDB <; Minion configuration <; Minion's grains <; Minion's pillar data < ... (credit: @myii)
Common checklist
@alxwr Hope you don't mind me hijacking this part of the comment as a common checklist for anyone to edit! It shows up properly in the GitHub UI if it's in the first comment:
config.geton existing formulas withsemantic-releaseand a valid test suite -- PR.The text was updated successfully, but these errors were encountered: