Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow slashes in file_roots envs. #51212

Merged
merged 2 commits into from Jan 17, 2019

Conversation

Projects
None yet
3 participants
@sathieu
Copy link
Contributor

commented Jan 17, 2019

What does this PR do?

Allow slashes in file_roots envs.

What issues does this PR fix or reference?

Fixes #51211.

Previous Behavior

Using this master config (to workaround #48132):

file_roots:
  base:
    - /srv/salt
  feature/2:
    - /srv/salt

Salt works incorrectly and logs:

2019-01-08 09:57:23,653 [salt.master      :1795][ERROR   ][28091] Error in function _file_list:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/master.py", line 1788, in run_func
    ret = getattr(self, func)(load)
  File "/usr/lib/python2.7/dist-packages/salt/utils/decorators/__init__.py", line 594, in wrapped
    **salt.utils.data.decode_dict(kwargs)
  File "/usr/lib/python2.7/dist-packages/salt/fileserver/__init__.py", line 752, in file_list
    ret.update(self.servers[fstr](load))
  File "/usr/lib/python2.7/dist-packages/salt/fileserver/roots.py", line 422, in file_list
    return _file_lists(load, 'files')
  File "/usr/lib/python2.7/dist-packages/salt/fileserver/roots.py", line 310, in _file_lists
    __opts__, form, list_cache, w_lock
  File "/usr/lib/python2.7/dist-packages/salt/fileserver/__init__.py", line 116, in check_file_list_cache
    if not os.path.isfile(list_cache) and _lock_cache(w_lock):
  File "/usr/lib/python2.7/dist-packages/salt/fileserver/__init__.py", line 50, in _lock_cache
    os.mkdir(w_lock)
OSError: [Errno 2] No such file or directory: '/var/cache/salt/master/file_lists/roots/.feature/2.w'

New Behavior

Slashes in file_roots envs are allowed.

Tests written?

Yes

Commits signed with GPG?

Yes

@sathieu

This comment has been minimized.

Copy link
Contributor Author

commented Jan 17, 2019

I'm trying TDD. Another commit will come with the proposed fix.

@sathieu sathieu changed the title WIP File roots slash WIP Allow slashes in file_roots envs. Jan 17, 2019

@sathieu

This comment has been minimized.

Copy link
Contributor Author

commented Jan 17, 2019

Confirmed failing:

11:54:32        ERROR [0.045s]: test_file_list_with_slash (unit.fileserver.test_roots.RootsTest)
11:54:32        ----------------------------------------------------------------------
11:54:32        Traceback (most recent call last):
11:54:32          File "/tmp/kitchen/testing/tests/unit/fileserver/test_roots.py", line 163, in test_file_list_with_slash
11:54:32            ret = roots.file_list(load)
11:54:32          File "/tmp/kitchen/testing/salt/fileserver/roots.py", line 422, in file_list
11:54:32            return _file_lists(load, 'files')
11:54:32          File "/tmp/kitchen/testing/salt/fileserver/roots.py", line 310, in _file_lists
11:54:32            __opts__, form, list_cache, w_lock
11:54:32          File "/tmp/kitchen/testing/salt/fileserver/__init__.py", line 122, in check_file_list_cache
11:54:32            if not os.path.isfile(list_cache) and _lock_cache(w_lock):
11:54:32          File "/tmp/kitchen/testing/salt/fileserver/__init__.py", line 56, in _lock_cache
11:54:32            os.mkdir(w_lock)
11:54:32        OSError: [Errno 2] No such file or directory: '/tmp/salt-tests-tmpdir/tmpaGcNUd/file_lists/roots/.foo/bar.w'

@sathieu sathieu force-pushed the sathieu:file_roots_slash branch from 2e751f6 to cfec8c5 Jan 17, 2019

@sathieu

This comment has been minimized.

Copy link
Contributor Author

commented Jan 17, 2019

@terminalmage @rallytime Please review.

I can squash the commits if you want.

@sathieu sathieu changed the title WIP Allow slashes in file_roots envs. Allow slashes in file_roots envs. Jan 17, 2019

sathieu added some commits Jan 17, 2019

Test for file_roots envs with slash leads to "No such file or directory"
See #51211.

2019-01-08 09:57:23,653 [salt.master      :1795][ERROR   ][28091] Error in function _file_list:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/master.py", line 1788, in run_func
    ret = getattr(self, func)(load)
  File "/usr/lib/python2.7/dist-packages/salt/utils/decorators/__init__.py", line 594, in wrapped
    **salt.utils.data.decode_dict(kwargs)
  File "/usr/lib/python2.7/dist-packages/salt/fileserver/__init__.py", line 752, in file_list
    ret.update(self.servers[fstr](load))
  File "/usr/lib/python2.7/dist-packages/salt/fileserver/roots.py", line 422, in file_list
    return _file_lists(load, 'files')
  File "/usr/lib/python2.7/dist-packages/salt/fileserver/roots.py", line 310, in _file_lists
    __opts__, form, list_cache, w_lock
  File "/usr/lib/python2.7/dist-packages/salt/fileserver/__init__.py", line 122, in check_file_list_cache
    if not os.path.isfile(list_cache) and _lock_cache(w_lock):
  File "/usr/lib/python2.7/dist-packages/salt/fileserver/__init__.py", line 56, in _lock_cache
    os.mkdir(w_lock)
OSError: [Errno 2] No such file or directory: '/var/cache/salt/master/file_lists/roots/.foo/bar.w'

@sathieu sathieu force-pushed the sathieu:file_roots_slash branch from cfec8c5 to b769f18 Jan 17, 2019

@sathieu

This comment has been minimized.

Copy link
Contributor Author

commented Jan 17, 2019

Re-pushed with signature.

@terminalmage

This comment has been minimized.

Copy link
Contributor

commented Jan 17, 2019

Nicely done, thanks!

@dwoz

dwoz approved these changes Jan 17, 2019

@dwoz dwoz merged commit 26008ef into saltstack:2018.3 Jan 17, 2019

10 checks passed

WIP Ready for review
Details
continuous-integration/jenkins/pr-merge This commit looks good
Details
jenkins/pr/docs The docs job has passed
Details
jenkins/pr/lint Python lint test has passed
Details
jenkins/pr/py2-centos-7 The py2-centos-7 job has passed
Details
jenkins/pr/py2-ubuntu-1604 The py2-ubuntu-1604 job has passed
Details
jenkins/pr/py2-windows-2016 The py2-windows-2016 job has passed
Details
jenkins/pr/py3-centos-7 The py3-centos-7 job has passed
Details
jenkins/pr/py3-ubuntu-1604 The py3-ubuntu-1604 job has passed
Details
jenkins/pr/py3-windows-2016 The py3-windows-2016 job has passed
Details
@sathieu

This comment has been minimized.

Copy link
Contributor Author

commented Feb 27, 2019

@dwoz @terminalmage Sadly, this PR is not in 2019.2.0 (like #51317).

@sathieu sathieu deleted the sathieu:file_roots_slash branch Mar 7, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.