hosts.rm_host changes /etc/hosts file with odd characters

[sunyq]

Dear all:
I am using Saltstack to modify /etc/hosts file.
Sometimes I just found salt-minion modified my /etc/hosts with odd characters.
Here is some minion log(I obscured some IP&username info):

And sometimes the same command can run correctly on the same machine:

So does anybody give me some tips about this?
Thanks a lot!

Salt Version:
           Salt: 2018.3.2

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8.1
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.13 (default, Mar 30 2018, 15:31:59)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.5.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5

System Versions:
           dist: centos 6.4 Final
         locale: UTF-8
        machine: x86_64
        release: 2.6.32-504.12.2.02.qiyi.el6.x86_64
         system: Linux
        version: CentOS 6.4 Final

[Ch3LL]

can you paste the command you are running?

[sunyq]

@Ch3LL Hi, I used curl as the http client and call salt-api to modify /etc/hosts.
Here is my command:

curl  http://salt-api:8000 -H 'Accept: application/jsonl' -H 'X-Auth-Token: x-auth-token' -d client=local -d tgt='target_ip' -d fun=hosts.rm_host -d arg='ip=10.10.10.10' -d arg='alias=test_host_1'

Here are screenshots which may be helpful:

when I open it in vim:

I just found another issue which may be relative with mine:

Thank you.

[Ch3LL]

i'm having a hard time replicating this:

[root@d0f76961c2f7 /]# cat /etc/hosts
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2      d0f76961c2f7
10.10.10.10    t_host_2 t_host_3 t_host_4

[root@d0f76961c2f7 /]# curl -sS localhost:8000/run -H 'Accept: application/x-yaml' -d client='local' -d tgt='*' -d username='saltdev' -d password='saltdev' -d eauth='pam' -d fun=hosts.rm_host -d arg='ip=10.10.10.10' -d arg='alias=t_host_2'
return:
- c6c2e89d6b0b: true

[root@d0f76961c2f7 /]# cat /etc/hosts
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2      d0f76961c2f7
10.10.10.10              t_host_3 t_host_4

how often does this occur?

[sunyq]

[root@devsrv-sunyq:~]# curl -Ss http://salt-api-server:8000/run -H 'Accept: application/x-yaml' -d client='local' -d tgt='10.10.11.11' -d username='devuser' -d password='devpass' -d eauth='pam' -d fun=hosts.list_hosts
return:
- 10.10.11.11:
    10.10.10.10:
    - test_host_1
    - test_host_2
    127.0.0.1:
    - localhost
    - localhost.localdomain
    - localhost4
    - localhost4.localdomain4
    ::1:
    - localhost
    - localhost.localdomain
    - localhost6
    - localhost6.localdomain6
[root@devsrv-sunyq:~]# curl -Ss http://salt-api-server:8000/run -H 'Accept: application/x-yaml' -d client='local' -d tgt='10.10.11.11' -d username='devuser' -d password='devpass' -d eauth='pam' -d fun=hosts.rm_host -d arg='10.10.10.10' -d arg='test_host_1'
return:
- 10.10.11.11: true
[root@devsrv-sunyq:~]# curl -Ss http://salt-api-server:8000/run -H 'Accept: application/x-yaml' -d client='local' -d tgt='10.10.11.11' -d username='devuser' -d password='devpass' -d eauth='pam' -d fun=hosts.list_hosts
return:
- 10.10.11.11:
    "\0\0\0": []
    "1\0\0\00\0\0\0.\0\0\01\0\0\00\0\0\0.\0\0\01\0\0\00\0\0\0.\0\0\01\0\0\00\0\0\0":
    - "\0\0\0"
    - "\0\0\0"
    - "\0\0\0t\0\0\0e\0\0\0s\0\0\0t\0\0\0_\0\0\0h\0\0\0o\0\0\0s\0\0\0t\0\0\0_\0\0\02\0\0\0"
    127.0.0.1:
    - localhost
    - localhost.localdomain
    - localhost4
    - localhost4.localdomain4
    ::1:
    - localhost
    - localhost.localdomain
    - localhost6
    - localhost6.localdomain6

Very strange, nearly almost in my case .
I am not using Docker, and I am using virtual machines(CentOS 6) for this test.

[sunyq]

Hi @Ch3LL , just get some more info:

• It seems that function rm_host results in this problem. I tried add_host&set_host and didn't get those odd characters
• Open /etc/hosts with vim(not cat) and you will find these odd characters(or show the content of /etc/hosts with function list_hosts).

Hope that will do some help when you replicate this problem.

[Ch3LL]

okay looks like i'm able to replicate this. It only occurs in the following situations:

1. using cent6
2. using salt-api. (if i use salt-call it works)

To note this does not affect the /etc/hosts file only the output from list_hosts

[sunyq]

Hi, I am able to replicate this problem with:

1. salt CLI
2. minion running on CentOS-7 with prompt:

Passed invalid arguments to hosts.rm_host: a bytes-like object is required, not 'str'

[Ch3LL]

can you paste the command you are running? I was never even able to replicate this problem on centos7

[sunyq]

salt-master

• centos 6
• python 3.6
• salt 2018.3.2

salt-minion

• centos 7
• python 3.6
• salt 2018.3.2

(py3_virtualenv) [root@salt-master]# salt '10.12.10.11' hosts.add_host '10.10.10.10' 'test_1'
10.12.10.11:
    True
(py3_virtualenv) [root@salt-master]#  salt '10.12.10.11' hosts.add_host '10.10.10.10' 'test_2'
10.12.10.11:
    True
(py3_virtualenv) [root@salt-master]# salt '10.12.10.11' hosts.add_host '10.10.10.10' 'test_3'
10.12.10.11:
    True
(py3_virtualenv) [root@salt-master]# salt '10.12.10.11' hosts.list_hosts
10.12.10.11:
    ----------
    10.10.10.10:
        - test_1
        - test_2
        - test_3
    127.0.0.1:
        - localhost
        - localhost.localdomain
        - localhost4
        - localhost4.localdomain4
    ::1:
        - localhost
        - localhost.localdomain
        - localhost6
        - localhost6.localdomain6
(py3_virtualenv) [root@salt-master]# salt '10.12.10.11' hosts.rm_host '10.10.10.10' 'test_2'
10.12.10.11:
    Passed invalid arguments to hosts.rm_host: a bytes-like object is required, not 'str'

        Remove a host entry from the hosts file

        CLI Example:

        .. code-block:: bash

            salt '*' hosts.rm_host <ip> <alias>

I install salt by running pip install salt

[Ch3LL]

ahh python3. in your initial versions report it shows python2. With python3 i can replicate this. As noted above though it does remove the entry and the file looks good.

[sunyq]

Oh, sorry for obscure description.

• With python2.7, rm_host changes /etc/hosts with odd characters
• With python3(3.6 in my case), rm_host will remove the whole entry and the file does look good.

It seems that the version of CentOS is not the key.

I modified(just changed some codes, perhaps my code looks "ugly" but it works) the implementation of function rm_host in salt/modules/hosts.py inspired by function set_host and find it works correctly now.

Glad to paste my code here or submit a pull request if you are interested in that.
Great thanks to you for your attention to my issue.

[Ch3LL]

yeah we would love a PR :) That would be great