Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Jinja imports with relative paths. Issue #13889 #47490

Merged
merged 16 commits into from Jun 8, 2018

Conversation

Projects
None yet
4 participants
@plastikos
Copy link
Contributor

commented May 4, 2018

Jinja imports will be interpreted as originating from the top of each
of the directories in the searchpath when the template name does not
begin with './' or '../'. When a template name begins with './' or
'../' then the import will be relative to the importing file.

Prior practices required the following construct:

{% from tpldir ~ '/foo' import bar %}

This patch supports a more "natural" construct:

{% from './foo' import bar %}

Comparatively when importing from a parent directory - prior practice:

{% from tpldir ~ '/../foo' import bar %}

Construct supported by this patch:

{% from '../foo' import bar %}

What does this PR do?

Supports relative Jinja imports.

What issues does this PR fix or reference?

#13889

Previous Behavior

Unfortunately {% from './foo' import bar %} does not work. This super-secret technique was required: {% from tpldir ~ '/foo' import bar %}

New Behavior

This works: {% from './foo' import bar %}

Tests written?

Yes

Commits signed with GPG?

No

Please review Salt's Contributing Guide for best practices.

See GitHub's page on GPG signing for more information about signing commits with GPG.

Thayne Harbaugh
Support Jinja imports with relative paths. Issue #13889
Jinja imports will be interpreted as originating from the top of each
of the directories in the searchpath when the template name does not
begin with './' or '../'.  When a template name begins with './' or
'../' then the import will be relative to the importing file.

Prior practices required the following construct:

.. code-block:: yaml

  {% from tpldir ~ '/foo' import bar %}

This patch supports a more "natural" construct:

.. code-block:: yaml

  {% from './foo' import bar %}

Comparatively when importing from a parent directory - prior practice:

.. code-block:: yaml

  {% from tpldir ~ '/../foo' import bar %}

Construct supported by this patch:

.. code-block:: yaml

  {% from '../foo' import bar %}
@plastikos
Copy link
Contributor Author

left a comment

There are a few details that should be cleaned-up before this is merged.

the importing file.
'''
# FIXME: somewhere do seprataor replacement: '\\' => '/'

This comment has been minimized.

Copy link
@plastikos

plastikos May 4, 2018

Author Contributor

Translation of \ => / might be broken. Help me, Obi Juan, you're my only hope!

This comment has been minimized.

Copy link
@terminalmage

terminalmage May 16, 2018

Contributor

@dwoz @twangboy When using jinja imports on Windows, are Windows-specific path-separators supported? If so, are they the only ones supported, or are forward slashes also supported? I would assume both are supported, since they are in Python proper, but I don't want to assume without knowing for sure.

This comment has been minimized.

Copy link
@twangboy

twangboy May 17, 2018

Contributor

I use Windows separators in my Jinja. I don't even have to escape them. Jinja apparently does some escaping for you.

This comment has been minimized.

Copy link
@plastikos

plastikos May 18, 2018

Author Contributor

The tpldir = os.path.dirname(template).replace('\\', '/') construct was in the original code and appears to do some path translation on Windows. I'm not sure how to test it to ensure that I haven't broken the functionality. Is there a unit test?

'''
Salt-specific loader to find imported jinja files.
Jinja imports will be interpreted as originating from the top

This comment has been minimized.

Copy link
@plastikos

plastikos May 4, 2018

Author Contributor

This import behavior needs to be in the documentation. Where is the best location for it?

@plastikos

This comment has been minimized.

Copy link
Contributor Author

commented May 8, 2018

RETEST

@plastikos

This comment has been minimized.

Copy link
Contributor Author

commented May 9, 2018

Help! Suggestions welcome!

@rallytime rallytime requested a review from terminalmage May 9, 2018

@@ -332,7 +332,7 @@ def parse_args(self, args=None, values=None):
os.path.basename(fpath).startswith('test_'):
self.options.name.append(fpath)
continue
self.exit(status=1, msg='\'{}\' is not a valid test module'.format(fpath))
self.exit(status=1, msg='\'{}\' is not a valid test module\n'.format(fpath))

This comment has been minimized.

Copy link
@terminalmage

terminalmage May 16, 2018

Contributor

Can you help me understand this change? I don't know what adding a newline here fixes.

This comment has been minimized.

Copy link
@plastikos

plastikos May 18, 2018

Author Contributor

Without the newline then the error message is not on a line by itself and the command prompt is left on the same line as the error message.

'Relative path "%s" cannot be resolved without an environment',
template
)
print('Relative path "{0}" cannot be resolved without an environment'.format(template))

This comment has been minimized.

Copy link
@terminalmage

terminalmage May 16, 2018

Contributor

We should not be printing to the CLI here.

This comment has been minimized.

Copy link
@plastikos

plastikos May 18, 2018

Author Contributor

Well that's embarrassing - that should have been removed.

if _template.split('/', 1)[0] == '..':
log.warning(
'Discarded template path "%s": attempts to'
' ascend outside of file roots', template

This comment has been minimized.

Copy link
@terminalmage

terminalmage May 16, 2018

Contributor

Since we have multiple backends, we can't assume people will be using file_roots, so this log message should reference salt:// instead.

This comment has been minimized.

Copy link
@plastikos

plastikos May 18, 2018

Author Contributor

Okay.

the importing file.
'''
# FIXME: somewhere do seprataor replacement: '\\' => '/'

This comment has been minimized.

Copy link
@terminalmage

terminalmage May 16, 2018

Contributor

@dwoz @twangboy When using jinja imports on Windows, are Windows-specific path-separators supported? If so, are they the only ones supported, or are forward slashes also supported? I would assume both are supported, since they are in Python proper, but I don't want to assume without knowing for sure.

plastikos added some commits May 18, 2018

Code review fixes.
Change message referencing "file roots" to "salt://"
@plastikos

This comment has been minimized.

Copy link
Contributor Author

commented May 19, 2018

@terminalmage - review changes, please.

@rallytime rallytime requested review from terminalmage and twangboy May 21, 2018

plastikos added some commits May 23, 2018

@rallytime

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2018

@plastikos Is this ready to go from your side?

If so, can you also add a mention of this addition to the Fluorine release notes?

@plastikos

This comment has been minimized.

Copy link
Contributor Author

commented Jun 8, 2018

@rallytime - I'll add the mention in the release notes and it will be ready to go.

plastikos and others added some commits Jun 8, 2018

@plastikos

This comment has been minimized.

Copy link
Contributor Author

commented Jun 8, 2018

@rallytime , thanks for the fix-ups.

@rallytime rallytime merged commit 160cdb2 into saltstack:develop Jun 8, 2018

5 of 10 checks passed

codeclimate 2 issues to fix
Details
default Build started sha1 is merged.
Details
jenkins/PR/salt-pr-linode-cent7-py3 Pull Requests » Salt PR - Linode CentOS 7 - PY3 #5595 — RUNNING
Details
jenkins/PR/salt-pr-linode-ubuntu16-py3 Pull Requests » Salt PR - Linode Ubuntu16.04 - PY3 #10566 — RUNNING
Details
jenkins/PR/salt-pr-rs-cent7-n Pull Requests » Salt PR - RS CentOS 7 #19649 — RUNNING
Details
WIP ready for review
Details
jenkins/PR/salt-pr-clone Pull Requests » Salt PR - Clone #25793 — SUCCESS
Details
jenkins/PR/salt-pr-docs-n Pull Requests » Salt PR - Docs #17858 — SUCCESS
Details
jenkins/PR/salt-pr-linode-ubuntu14-n Pull Requests » Salt PR - Linode Ubuntu14.04 #23525 — SUCCESS
Details
jenkins/PR/salt-pr-lint-n Pull Requests » Salt PR - Code Lint #22489 — SUCCESS
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.