Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix masterless jinja imports #54780

Merged
merged 3 commits into from Sep 29, 2019

Conversation

@dwoz
Copy link
Contributor

dwoz commented Sep 26, 2019

What does this PR do?

Fixes and adds tests for #54765 #54768

Tests written?

Yes

Commits signed with GPG?

Yes

@dwoz

This comment has been minimized.

Copy link
Contributor Author

dwoz commented Sep 26, 2019

re-run full all

@dwoz dwoz force-pushed the dwoz:fix-masterless-jinja-imports branch 3 times, most recently from 0939d4e to 43e67a2 Sep 26, 2019
@dwoz dwoz force-pushed the dwoz:fix-masterless-jinja-imports branch from 43e67a2 to 00eba43 Sep 26, 2019
@dwoz dwoz force-pushed the dwoz:fix-masterless-jinja-imports branch from 00eba43 to 6aea83a Sep 26, 2019
kiemlicz added a commit to kiemlicz/ambassador that referenced this pull request Sep 26, 2019
@kiemlicz

This comment has been minimized.

Copy link
Contributor

kiemlicz commented Sep 26, 2019

I've pulled the jinja.py and added to _utils dir in state tree root

Still getting:

{% from "_common/util.jinja" import retry with context %}    <======================
@s0undt3ch

This comment has been minimized.

Copy link
Member

s0undt3ch commented Sep 27, 2019

@kiemlicz this PR adds a test which covers the proposed fix.

It seems that what you're trying to do is to "push" the fix to the minions through your state tree. That should be filed as a separate issue so we can better track it. It's not related to the change this PR is addressing.

@dwoz

This comment has been minimized.

Copy link
Contributor Author

dwoz commented Sep 27, 2019

re-run full all

Erik Johnson and others added 2 commits Sep 26, 2019
The addition of fileclient caching in the JinjaCacheLoader has broken
all jinja imports in state files, in masterless salt-call runs. This is
because masterless Pillar compilation uses the `PillarClient` while
regular file caching (states, etc.) use the `LocalClient`. Since Pillar
is compiled first in the masterless run, the class-level attribute where
the cached fileclient is stored only ever usese the PillarClient, and
thus any Jinja imports will cache the templates to the wrong location
(under `pillar_roots`).

This expands the cached fileclient support to distinguish between pillar
and non-pillar, and keep a separate cached fileclient for each.
@dwoz dwoz force-pushed the dwoz:fix-masterless-jinja-imports branch 2 times, most recently from 29b31fc to e901a83 Sep 28, 2019
@dwoz

This comment has been minimized.

Copy link
Contributor Author

dwoz commented Sep 28, 2019

This PR has the same patch as #54774 and adds tests

@kiemlicz

This comment has been minimized.

Copy link
Contributor

kiemlicz commented Sep 29, 2019

@s0undt3ch thanks for reply, now I'm truly disappointed

I know I can override the execution modules, state modules etc. using e.g. _utils dir from local state tree.
It was always the fastest way to 'port' the fixes to the running salt version, however it turns that it doesn't work for utils/jinja.py: why?

Overriding the local installed file: cp /srv/salt/base/_utils/jinja.py /usr/lib/python3/dist-packages/salt/utils/jinja.py proves that the fix is working, however the sudden inability to use _utils/ dir is frightening! Could you explain this please?

@dwoz dwoz merged commit b9459e6 into saltstack:2019.2.1 Sep 29, 2019
21 of 24 checks passed
21 of 24 checks passed
ci/py2/centos6 This commit is being built
Details
ci/py3/centos7 This commit is being built
Details
ci/py3/centos7/tcp This commit is being built
Details
WIP Ready for review
Details
ci/docs This commit looks good
Details
ci/lint This commit looks good
Details
ci/py2/amazon2 This commit looks good
Details
ci/py2/centos7 This commit looks good
Details
ci/py2/centos7/tcp This commit looks good
Details
ci/py2/debian8 This commit looks good
Details
ci/py2/debian9 This commit looks good
Details
ci/py2/fedora29 This commit looks good
Details
ci/py2/ubuntu1604 This commit looks good
Details
ci/py2/ubuntu1604/tcp This commit looks good
Details
ci/py2/ubuntu1804 This commit looks good
Details
ci/py2/windows2016 This commit looks good
Details
ci/py3/amazon2 This commit looks good
Details
ci/py3/debian8 This commit looks good
Details
ci/py3/debian9 This commit looks good
Details
ci/py3/fedora29 This commit looks good
Details
ci/py3/ubuntu1604 This commit looks good
Details
ci/py3/ubuntu1604/tcp This commit looks good
Details
ci/py3/ubuntu1804 This commit looks good
Details
ci/py3/windows2016 This commit looks good
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can’t perform that action at this time.