Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hosts.rm_host changes /etc/hosts file with odd characters #49150

Closed
sunyq opened this issue Aug 16, 2018 · 12 comments

Comments

Projects
None yet
3 participants
@sunyq
Copy link

commented Aug 16, 2018

Dear all:
I am using Saltstack to modify /etc/hosts file.
Sometimes I just found salt-minion modified my /etc/hosts with odd characters.
Here is some minion log(I obscured some IP&username info):
image
And sometimes the same command can run correctly on the same machine:
image

So does anybody give me some tips about this?
Thanks a lot!

Salt Version:
           Salt: 2018.3.2

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8.1
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.13 (default, Mar 30 2018, 15:31:59)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.5.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5

System Versions:
           dist: centos 6.4 Final
         locale: UTF-8
        machine: x86_64
        release: 2.6.32-504.12.2.02.qiyi.el6.x86_64
         system: Linux
        version: CentOS 6.4 Final
@Ch3LL

This comment has been minimized.

Copy link
Contributor

commented Aug 16, 2018

can you paste the command you are running?

@Ch3LL Ch3LL added the Info Needed label Aug 16, 2018

@Ch3LL Ch3LL added this to the Blocked milestone Aug 16, 2018

@sunyq

This comment has been minimized.

Copy link
Author

commented Aug 16, 2018

@Ch3LL Hi, I used curl as the http client and call salt-api to modify /etc/hosts.
Here is my command:

curl  http://salt-api:8000 -H 'Accept: application/jsonl' -H 'X-Auth-Token: x-auth-token' -d client=local -d tgt='target_ip' -d fun=hosts.rm_host -d arg='ip=10.10.10.10' -d arg='alias=test_host_1'

Here are screenshots which may be helpful:
image
when I open it in vim:
image

I just found another issue which may be relative with mine:

Thank you.

@Ch3LL

This comment has been minimized.

Copy link
Contributor

commented Aug 17, 2018

i'm having a hard time replicating this:

[root@d0f76961c2f7 /]# cat /etc/hosts
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2      d0f76961c2f7
10.10.10.10    t_host_2 t_host_3 t_host_4
[root@d0f76961c2f7 /]# curl -sS localhost:8000/run -H 'Accept: application/x-yaml' -d client='local' -d tgt='*' -d username='saltdev' -d password='saltdev' -d eauth='pam' -d fun=hosts.rm_host -d arg='ip=10.10.10.10' -d arg='alias=t_host_2'
return:
- c6c2e89d6b0b: true
[root@d0f76961c2f7 /]# cat /etc/hosts
127.0.0.1       localhost
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2      d0f76961c2f7
10.10.10.10              t_host_3 t_host_4

how often does this occur?

@sunyq

This comment has been minimized.

Copy link
Author

commented Aug 17, 2018

[root@devsrv-sunyq:~]# curl -Ss http://salt-api-server:8000/run -H 'Accept: application/x-yaml' -d client='local' -d tgt='10.10.11.11' -d username='devuser' -d password='devpass' -d eauth='pam' -d fun=hosts.list_hosts
return:
- 10.10.11.11:
    10.10.10.10:
    - test_host_1
    - test_host_2
    127.0.0.1:
    - localhost
    - localhost.localdomain
    - localhost4
    - localhost4.localdomain4
    ::1:
    - localhost
    - localhost.localdomain
    - localhost6
    - localhost6.localdomain6
[root@devsrv-sunyq:~]# curl -Ss http://salt-api-server:8000/run -H 'Accept: application/x-yaml' -d client='local' -d tgt='10.10.11.11' -d username='devuser' -d password='devpass' -d eauth='pam' -d fun=hosts.rm_host -d arg='10.10.10.10' -d arg='test_host_1'
return:
- 10.10.11.11: true
[root@devsrv-sunyq:~]# curl -Ss http://salt-api-server:8000/run -H 'Accept: application/x-yaml' -d client='local' -d tgt='10.10.11.11' -d username='devuser' -d password='devpass' -d eauth='pam' -d fun=hosts.list_hosts
return:
- 10.10.11.11:
    "\0\0\0": []
    "1\0\0\00\0\0\0.\0\0\01\0\0\00\0\0\0.\0\0\01\0\0\00\0\0\0.\0\0\01\0\0\00\0\0\0":
    - "\0\0\0"
    - "\0\0\0"
    - "\0\0\0t\0\0\0e\0\0\0s\0\0\0t\0\0\0_\0\0\0h\0\0\0o\0\0\0s\0\0\0t\0\0\0_\0\0\02\0\0\0"
    127.0.0.1:
    - localhost
    - localhost.localdomain
    - localhost4
    - localhost4.localdomain4
    ::1:
    - localhost
    - localhost.localdomain
    - localhost6
    - localhost6.localdomain6

Very strange, nearly almost in my case .
I am not using Docker, and I am using virtual machines(CentOS 6) for this test.

@sunyq

This comment has been minimized.

Copy link
Author

commented Aug 20, 2018

Hi @Ch3LL , just get some more info:

  • It seems that function rm_host results in this problem. I tried add_host&set_host and didn't get those odd characters
  • Open /etc/hosts with vim(not cat) and you will find these odd characters(or show the content of /etc/hosts with function list_hosts).

Hope that will do some help when you replicate this problem.

@Ch3LL

This comment has been minimized.

Copy link
Contributor

commented Aug 20, 2018

okay looks like i'm able to replicate this. It only occurs in the following situations:

  1. using cent6
  2. using salt-api. (if i use salt-call it works)

To note this does not affect the /etc/hosts file only the output from list_hosts

@Ch3LL Ch3LL modified the milestones: Blocked, Approved Aug 20, 2018

@sunyq

This comment has been minimized.

Copy link
Author

commented Aug 22, 2018

Hi, I am able to replicate this problem with:

  1. salt CLI
  2. minion running on CentOS-7 with prompt:
Passed invalid arguments to hosts.rm_host: a bytes-like object is required, not 'str'
@Ch3LL

This comment has been minimized.

Copy link
Contributor

commented Aug 22, 2018

can you paste the command you are running? I was never even able to replicate this problem on centos7

@sunyq

This comment has been minimized.

Copy link
Author

commented Aug 23, 2018

salt-master

  • centos 6
  • python 3.6
  • salt 2018.3.2

salt-minion

  • centos 7
  • python 3.6
  • salt 2018.3.2
(py3_virtualenv) [root@salt-master]# salt '10.12.10.11' hosts.add_host '10.10.10.10' 'test_1'
10.12.10.11:
    True
(py3_virtualenv) [root@salt-master]#  salt '10.12.10.11' hosts.add_host '10.10.10.10' 'test_2'
10.12.10.11:
    True
(py3_virtualenv) [root@salt-master]# salt '10.12.10.11' hosts.add_host '10.10.10.10' 'test_3'
10.12.10.11:
    True
(py3_virtualenv) [root@salt-master]# salt '10.12.10.11' hosts.list_hosts
10.12.10.11:
    ----------
    10.10.10.10:
        - test_1
        - test_2
        - test_3
    127.0.0.1:
        - localhost
        - localhost.localdomain
        - localhost4
        - localhost4.localdomain4
    ::1:
        - localhost
        - localhost.localdomain
        - localhost6
        - localhost6.localdomain6
(py3_virtualenv) [root@salt-master]# salt '10.12.10.11' hosts.rm_host '10.10.10.10' 'test_2'
10.12.10.11:
    Passed invalid arguments to hosts.rm_host: a bytes-like object is required, not 'str'

        Remove a host entry from the hosts file

        CLI Example:

        .. code-block:: bash

            salt '*' hosts.rm_host <ip> <alias>

I install salt by running pip install salt

@sunyq sunyq changed the title hosts.rm_host and hosts.add_host changed /etc/hosts file with odd characters hosts.rm_host changed /etc/hosts file with odd characters Aug 23, 2018

@sunyq sunyq changed the title hosts.rm_host changed /etc/hosts file with odd characters hosts.rm_host changes /etc/hosts file with odd characters Aug 23, 2018

@Ch3LL

This comment has been minimized.

Copy link
Contributor

commented Aug 23, 2018

ahh python3. in your initial versions report it shows python2. With python3 i can replicate this. As noted above though it does remove the entry and the file looks good.

@sunyq

This comment has been minimized.

Copy link
Author

commented Aug 24, 2018

Oh, sorry for obscure description.

  • With python2.7, rm_host changes /etc/hosts with odd characters
  • With python3(3.6 in my case), rm_host will remove the whole entry and the file does look good.

It seems that the version of CentOS is not the key.

I modified(just changed some codes, perhaps my code looks "ugly" but it works) the implementation of function rm_host in salt/modules/hosts.py inspired by function set_host and find it works correctly now.

Glad to paste my code here or submit a pull request if you are interested in that.
Great thanks to you for your attention to my issue.

@Ch3LL

This comment has been minimized.

Copy link
Contributor

commented Aug 27, 2018

yeah we would love a PR :) That would be great

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.