Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup calculation of template sls/tpl context #58238

Merged
merged 38 commits into from Oct 8, 2020
Merged
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
3c3e953
Cleanup calculation of template sls/tpl context
Aug 18, 2020
2e99347
fix string formatting
Aug 18, 2020
3aec8c0
Add unit test that works against old version (with bugs)
Aug 19, 2020
1c6d51b
Updated unit tests with non-buggy values and fix found bugs
Aug 19, 2020
8d371c8
cleanup unit tests to test underlying function
Aug 19, 2020
31a62b0
remove old unit test components no longer used
Aug 19, 2020
938d517
Cleanup
Aug 19, 2020
fd85b92
Merge branch 'master' into feature/56410-tpldir-calculation
mlasevich Aug 19, 2020
08253f4
More Cleanup
Aug 19, 2020
52aebff
Merge branch 'feature/56410-tpldir-calculation' of github.com:mlasevi…
Aug 19, 2020
237c62b
More cleanup. Add Mock to test.support.unit
Aug 19, 2020
0b626a0
Add changelog entries
Aug 19, 2020
05d0a5c
Fix Mockery
Aug 19, 2020
6617911
Merge branch 'master' into feature/56410-tpldir-calculation
mlasevich Aug 19, 2020
e8e0b3c
Import order fix
Aug 19, 2020
baef2d7
Handle backslashes in sls names under *nix
Aug 20, 2020
1c5b224
Cleanup
Aug 20, 2020
40cb816
Make sure we return a dictionary from jinja.load_map
Aug 20, 2020
ccf8e54
Fix scenario when sls is empty but present
Aug 20, 2020
ec298bb
Touched another file - Cleanup to make pre-commit happy
Aug 20, 2020
1d68a9f
Adding variables to docs
Aug 20, 2020
ff67a24
Merge branch 'master' into feature/56410-tpldir-calculation
mlasevich Aug 20, 2020
74186a3
Fix expected tplpath value to be OS specific and note so in docs
Aug 20, 2020
d9cd4ae
Merge branch 'master' into feature/56410-tpldir-calculation
sagetherage Sep 18, 2020
ff6f4d0
Merge branch 'master' into feature/56410-tpldir-calculation
sagetherage Sep 23, 2020
0a3ed38
Merge branch 'master' into feature/56410-tpldir-calculation
sagetherage Sep 29, 2020
4de3f4e
Merge branch 'master' into feature/56410-tpldir-calculation
mlasevich Oct 1, 2020
f4df5b5
Merge branch 'master' into feature/56410-tpldir-calculation
mlasevich Oct 1, 2020
1df0116
removing comments from imports as per pre-commit
Oct 1, 2020
88483e9
Merge branch 'master' into feature/56410-tpldir-calculation
mlasevich Oct 1, 2020
2fef670
removing comments from imports as per pre-commit
Oct 1, 2020
ce7fd94
Merge branch 'feature/56410-tpldir-calculation' of github.com:mlasevi…
Oct 1, 2020
d841bc5
Merge branch 'master' into feature/56410-tpldir-calculation
mlasevich Oct 2, 2020
9380b56
Put slsvars changes behind a feature flag
dwoz Oct 6, 2020
e3dc751
Better documentation for enable_slsvars_fixes feature flag
dwoz Oct 6, 2020
6cd8f56
Merge branch 'master' into feature/56410-tpldir-calculation
twangboy Oct 7, 2020
be8ece2
Merge branch 'master' into feature/56410-tpldir-calculation
dwoz Oct 7, 2020
2ff0dc8
Fix test that should be skipped on windows
dwoz Oct 7, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog/56410.fixed
@@ -0,0 +1 @@
Proper calculation of tpldir and related context parameters
1 change: 1 addition & 0 deletions changelog/58249.fixed
@@ -0,0 +1 @@
Fix blank tplfile context parameter when loading top level sls files
1 change: 1 addition & 0 deletions changelog/58250.fixed
@@ -0,0 +1 @@
Do not include init directory in sls context parameters if explicitly specified
4 changes: 4 additions & 0 deletions changelog/58652.added
@@ -0,0 +1,4 @@
Added features config option for feature flags. Added a feature flag
`enable_slsvars_fixes` to enable fixes to tpldir, tplfile and sls_path.
This flag will be depricated in the Phosphorus release when this functionality
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo:

Suggested change
This flag will be depricated in the Phosphorus release when this functionality
This flag will be deprecated in the Phosphorus release when this functionality

becomes the default.
108 changes: 105 additions & 3 deletions doc/ref/states/vars.rst
Expand Up @@ -2,6 +2,21 @@
SLS Template Variable Reference
===============================


.. warning::
In the 3002 release sls_path, tplfile, and tpldir have had some significate
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo mainly, formatting for consistency:

Suggested change
In the 3002 release sls_path, tplfile, and tpldir have had some significate
In the 3002 release ``sls_path``, ``tplfile``, and ``tpldir`` have had some significant

improvements which have the potential to break states that rely on old and
broken functionality. These fixes can be enabled by setting the
``use_slsvars_fixes`` feature flag to ``True`` in your minion's config file.
This functionality will become the default in the 3005 release.

.. code-block:: yaml

features:
use_slsvars_fixes: True



The template engines available to sls files and file templates come loaded
with a number of context variables. These variables contain information and
functions to assist in the generation of templates. See each variable below
Expand Down Expand Up @@ -81,8 +96,13 @@ from an environment.

{{ saltenv }}

SLS Only Variables
==================
The following are only available when processing sls files. If you need these
in other templates, you can usually pass them in as template context.

sls
====
---

The `sls` variable contains the sls reference value, and is only available in
the actual SLS file (not in any files referenced in that SLS). The sls
Expand All @@ -94,14 +114,96 @@ include option.
{{ sls }}

slspath
=======
-------

The `slspath` variable contains the path to the directory of the current sls
file. The value of `slspath` in files referenced in the current sls depends on
the reference method. For jinja includes `slspath` is the path to the current
directory of the file. For salt includes `slspath` is the path to the directory
of the included file.
of the included file. If current sls file is in root of the file roots, this
will return ""

.. code-block:: jinja

{{ slspath }}


sls_path
--------

A version of `slspath` with underscores as path separators instead of slashes.
So, if `slspath` is `path/to/state` then `sls_path` is `path_to_state`

.. code-block:: jinja

{{ sls_path }}

slsdotpath
----------

A version of `slspath` with dots as path separators instead of slashes. So, if
`slspath` is `path/to/state` then `slsdotpath` is `path.to.state`. This is same
as `sls` if `sls` points to a directory instead if a file.

.. code-block:: jinja

{{ slsdotpath }}


slscolonpath
------------

A version of `slspath` with colons (`:`) as path separators instead of slashes.
So, if `slspath` is `path/to/state` then `slscolonpath` is `path:to:state`.

.. code-block:: jinja

{{ slscolonpath }}

tplpath
-------

Full path to sls template file being process on local disk. This is usually
pointing to a copy of the sls file in a cache directory. This will be in OS
specific format (windows vs posix). (It is probably best not to use this.)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
specific format (windows vs posix). (It is probably best not to use this.)
specific format (Windows vs POSIX). (It is probably best not to use this.)


.. code-block:: jinja

{{ tplpath }}


tplfile
-------

Relative path to exact sls template file being processed relative to file
roots.

.. code-block:: jinja

{{ tplfile }}

tpldir
------

Directory, relative to file roots, of the current sls file. If current sls file
is in root of the file roots, this will return ".". This is usually identical
to `slspath` except in case of root-level sls, where this will return a "`.`".

A Common use case for this variable is to generate relative salt urls like:
.. code-block:: jinja

my-file:
file.managed:
source: salt://{{ tpldir }}/files/my-template


tpldot
------

A version of `tpldir` with dots as path separators instead of slashes. So, if
`tpldir` is `path/to/state` then `tpldot` is `path.to.state`. NOTE: if `tpldir`
is `.`, this will be set to ""

.. code-block:: jinja

{{ tpldot }}
3 changes: 2 additions & 1 deletion salt/config/__init__.py
@@ -1,7 +1,6 @@
"""
All salt configuration loading and defaults should be in this module
"""

import codecs
import glob
import logging
Expand Down Expand Up @@ -944,6 +943,8 @@ def _gather_buffer_space():
# client via the Salt API
"netapi_allow_raw_shell": bool,
"disabled_requisites": (str, list),
# Feature flag config
"features": dict,
}
)

Expand Down
28 changes: 28 additions & 0 deletions salt/features.py
@@ -0,0 +1,28 @@
"""
Feature flags
"""
import logging

log = logging.getLogger(__name__)


class Features:
def __init__(self, _features=None):
if _features is None:
self.features = {}
else:
self.features = _features
self.setup = False

def setup_features(self, opts):
if not self.setup:
self.features.update(opts.get("features", {}))
else:
log.warn("Features already setup")

def get(self, key, default=None):
return self.features.get(key, default)


features = Features()
setup_features = features.setup_features