KeyError: 'file.replace' ('file.blockreplace') on Salt: 2014.1.0 with Linux Master and Windows Minion

[uvsmtid]

I'm consistently reproducing the following issue on multiple hosts (different masters and minions, but the same software versions and Salt states).

Problem

Create /srv/salt/replace_a_file.sls state file (on Linux Salt master - see versions below) with the following content:

# replace_a_file.sls

{% if grains['os'] in [ 'Windows' ] %}

{% set file_dir = 'C:\deleteme' %}

'{{ file_dir }}\qwer.txt':
    file.replace:
        - pattern: 'whatever'
        - repl: 'whatever_replaced'
        - show_changes: True

{% endif %}

Create C:\deleteme\qwer.txt sample file (on Windows Salt minion - see versions below) with any random text content.

Push the state from the master:

 salt 'winclient' state.sls replace_a_file
winclient:
----------
          ID: C:\deleteme\qwer.txt
    Function: file.replace
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "salt/state.py", line 1371, in call
                File "salt/states/file.py", line 1963, in replace
              KeyError: 'file.replace'
     Changes:

Notes

• Actually, I don't remember cases when any state using file.replace function successfully executed from Linux Master on Windows Minion with Salt version 2014.1.0. Please feedback anyone if there are known examples.
• I also tried to modify state code here and there and the only thing I was able to get is another error: On Windows file state throws exception in check_perms: global name '__salt__' is not defined #10773 (maybe "__salt__ is not defined" problem is completely unrelated here because it appears intermittently and restarting Windows minion usually helps to recover).
• Linux Master to Linux Minion works fine (I have multiple examples in the same state tree which concurrently execute for Linuxes without problems).

Versions

Master: Linux F20 64 bit

> salt --versions-report
           Salt: 2014.1.0
         Python: 2.7.5 (default, Feb 19 2014, 13:47:28)
         Jinja2: 2.7.1
       M2Crypto: 0.21.1
 msgpack-python: 0.1.13
   msgpack-pure: Not Installed
       pycrypto: 2.6.1
         PyYAML: 3.10
          PyZMQ: 13.0.2
            ZMQ: 3.2.4

> rpm -qf $(which salt-call)
salt-minion-2014.1.0-1.fc20.noarch

Minion: Windows 7 64 bit

c:\salt> salt-minion.exe --versions-report
           Salt: 2014.1.0-5-g32d3463
         Python: 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit (AMD64)]
         Jinja2: 2.7.1
       M2Crypto: 0.21.1
 msgpack-python: 0.4.0
   msgpack-pure: Not Installed
       pycrypto: 2.6
         PyYAML: 3.10
          PyZMQ: 14.0.1
            ZMQ: 4.0.3

[basepi]

Looks like the replace and search functions are not being imported into the win_file module, which is why they were not found in __salt__ by the state. Assuming those functions are Windows-safe, the import should be all we need.

[UtahDave]

I just tested this on the latest develop and file.replace is working based on your example above. @basepi will cherry pick this so it should be in 2014.1.2

[uvsmtid]

@UtahDave, thanks!

[uvsmtid]

@UtahDave, I have exactly the same issue with file.blockreplace (not just replace) on latest 2014.1.5 release for Windows:

          ID: managed_hosts_file
    Function: file.blockreplace
        Name: C:\Windows\system32\drivers\etc\hosts
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "salt/state.py", line 1379, in call
                File "salt/states/file.py", line 2129, in blockreplace
              KeyError: 'file.blockreplace'

Please re-open the issue.
Based on the description it should be relatively easy to fix for the next 2014.1.* release.

[UtahDave]

ok, @uvsmtid thanks for the heads up! I'll reopen this and see if I can get a fix in for the next release.

[UtahDave]

@uvsmtid hey, can you provide an example of an sls file using file.blockreplace that causes this error? It's working for me on develop. I'll test on 2014.1.5 now

[uvsmtid]

This SLS with file.blockreplace fails (confirmed) on Windows 7 x86_64:

#
managed_hosts_file:                                                             
    file.blockreplace:                                                          
{% if grains['kernel'] in [ 'Linux' ] %}                       
        - name: '/etc/hosts'                                                    
{% endif %}                                                                     
{% if grains['kernel'] in [ 'Windows' ] %}                                          
        - name: 'C:\Windows\system32\drivers\etc\hosts'                         
{% endif %}                                                                     
{% if grains['kernel'] in [ 'Linux' ] %}                       
        - user: root                                                            
        - group: root                                                           
        - mode: 644                                                             
{% endif %}                                                                     
        - backup: '.salt.backup'                                                
        - append_if_not_found: True                                             
        - marker_start: "# <<< AUTOMATICALLY MANAGED by Salt"                   
        - content: |                                                            
            whatever
        - marker_end:   "# >>> AUTOMATICALLY MANAGED by Salt"             

If saved as deleteme.sls, the output is:

 sudo salt 'windows_minion_id' state.sls deleteme
windows_minion_id:
----------
          ID: managed_hosts_file
    Function: file.blockreplace
        Name: C:\Windows\system32\drivers\etc\hosts
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "salt/state.py", line 1379, in call
                File "salt/states/file.py", line 2129, in blockreplace
              KeyError: 'file.blockreplace'
     Changes:   

Summary
------------
Succeeded: 0
Failed:    1
------------
Total:     1

Double-confirm version:

salt-minion.exe --versions
           Salt: 2014.1.5
         Python: 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit (AMD64)]
         Jinja2: 2.7.1
       M2Crypto: 0.21.1
 msgpack-python: 0.4.2
   msgpack-pure: Not Installed
       pycrypto: 2.6
         PyYAML: 3.11
          PyZMQ: 14.1.1
            ZMQ: 4.0.4

[UtahDave]

Thanks for the quick reply!

I just confirmed that blockreplace has not been added to win_file.py in the 2014.1 branch, but it does exist in the develop branch.

This may mean we'll have to wait for the next full release for this feature. I'll see if this is something we can back port.

[rallytime]

I suspect this has been addressed on the 2014.7 branch. Has anyone been able to try this on 2014.7.0?

[soodr]

Doesn't work with Salt master running on Salt: 2014.7.0rc2. Identical issue as above for @uvsmtid.

My state definition is

update_datbase_bat:
    file.blockreplace:
        - name: 'D:\scripts\mssql_install\create_databases.bat'
        - marker_start: "rem Set these variable to reflect the local environment:"
        - marker_end: "rem Demo data parameters"
       - content: |
           set CONNECTIDENTIFIER=abcd\{{ pillar["DB_SCHEMA"] }}
           set ADMINNAME="abcd"
           set ADMINPASSWORD={{ pillar["ADMIN_PASSWD"] }}
           set SERVERDB_NAME=test_server
           set SERVERDB_USER="test"
           set SERVERDB_PASSWORD="test"
       - append_if_not_found: True
       - backup: '.bak'
       - show_changes: True

[uvsmtid]

I've just tested all examples with salt-master-2014.7.0-3.fc20.noarch on Linux (Fedora 20) and salt-2014.7.0.win-amd64 on Windows 7 both 64.

The only issues I found are related to file endings on windows #12284, #14002, #14135 (which I'm not even link-mentioning here because these are orthogonally separate issues).

Functions themselves are invoked and this issue can be closed.

@soodr, to make your example work, I had to fix indents in YAML (i.e. look at you content item - it is one space off). Obviously, I fixed everything to what is available on my system: D: to C:, pillar data to constants, etc.

[rallytime]

@uvsmtid Just to confirm, but in your testing of the 2014.7.0 windows packages, is this fixed in 2014.7.0 or is this still an issue?

@soodr Definitely change your - content spacing, as that is indeed invalid YAML. Give that a try and see if it fixes the issue. Also, a lot of code changed between 2014.7.0rc2 and 2014.7.0 (windows pacakages are available now: http://docs.saltstack.com/en/latest/topics/installation/windows.html), so depending on @uvsmtid to the question above, you might want to give that a try as well with the spacing change in your sls file. Let us know if that helps!

[uvsmtid]

@rallytime, it's all fixed. Windows package salt-2014.7.0.win-amd64 works as expected.

Only line endings as specified in my previous comment shows odd behaviours (i.e. second run of the same *replace state updates some white spaces) - but those are rather unrelated and more general issues.

I would close this (11471) issue.

[soodr]

I was pointing to the missing function in salt dict

ID: update_datbase_bat
Function: file.blockreplace
    Name: D:\scripts\mssql_install\create_databases.bat
  Result: False
 Comment: An exception occurred in this state: Traceback (most recent call last):
            File "salt/state.py", line 1382, in call
            File "salt/states/file.py", line 2129, in blockreplace
          KeyError: 'file.blockreplace'

Minion setup :

c:\salt>salt-minion --versions-report
           Salt: 2014.1.13
         Python: 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)]
         Jinja2: 2.7.1
       M2Crypto: 0.21.1
 msgpack-python: 0.4.2
   msgpack-pure: Not Installed
       pycrypto: 2.6
         PyYAML: 3.11
          PyZMQ: 14.1.1
            ZMQ: 4.0.4

[soodr]

Upgraded salt-minion to make it work. However the versions-report stack-traces. The minion is running on a Windows Server 2008 R2 DataCenter VM.

c:\salt>salt-minion --versions-report
           Salt: 2014.7.0
         Python: 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit (AMD64)]
         Jinja2: 2.7.1
       M2Crypto: 0.21.1
 msgpack-python: 0.4.2
   msgpack-pure: Not Installed
       pycrypto: 2.6
        libnacl: Not Installed
         PyYAML: 3.11
          ioflo: Not Installed
          PyZMQ: 14.1.1
           RAET: Not Installed
            ZMQ: 4.0.4
           Mako: Not Installed
Traceback (most recent call last):
  File "<string>", line 6, in <module>
  File "__main__.py", line 726, in <module>
  File "__main__.py", line 332, in bootstrap
  File "__main__.py", line 359, in chainload
  File "__main__.py", line 715, in _chainload
  File "__main__.py", line 128, in <module>
  File "__main__salt-minion__.py", line 17, in <module>
  File "salt/scripts.py", line 56, in salt_minion
  File "salt/__init__.py", line 262, in start
  File "salt/__init__.py", line 168, in prepare
  File "salt/utils/parsers.py", line 126, in parse_args
  File "salt/utils/parsers.py", line 194, in print_versions_report
AttributeError: 'module' object has no attribute 'EX_OK'

[rallytime]

@soodr Hrm. That looks like a different issue entirely. Can you file a new bug report please?

[soodr]

#18244

[rallytime]

Thanks for opening a new issue! We'll take a look at that one.

I'm going to close this one for now since it's reported as being fixed. If this pops up again, leave a comment and we will definitely reopen and take another look.