[BUG] salt-ssh error when targetting IPs or hostnames directly

[ghost]

Description
Running salt-ssh directly on an IP or hostname errors.

Setup
Ubuntu 16.04 with only salt-ssh installed.

Steps to Reproduce the behavior

salt-ssh '198.51.100.0' test.ping
# or
salt-ssh 'ex1.example.com' test.ping

Returns the following output:

$ salt-ssh '198.51.100.0' test.ping
[ERROR   ] An un-handled exception was caught by salt's global exception handler:
AttributeError: 'str' object has no attribute 'get'
Traceback (most recent call last):
  File "/usr/bin/salt-ssh", line 9, in <module>
    load_entry_point('salt==3002.2', 'console_scripts', 'salt-ssh')()
  File "/usr/lib/python3/dist-packages/salt/scripts.py", line 479, in salt_ssh
    client.run()
  File "/usr/lib/python3/dist-packages/salt/cli/ssh.py", line 25, in run
    ssh = salt.client.ssh.SSH(self.config)
  File "/usr/lib/python3/dist-packages/salt/client/ssh/__init__.py", line 234, in __init__
    self._expand_target()
  File "/usr/lib/python3/dist-packages/salt/client/ssh/__init__.py", line 373, in _expand_target
    if hostname in [host_id, roster_data[host_id].get("host")]:
AttributeError: 'str' object has no attribute 'get'
Traceback (most recent call last):
  File "/usr/bin/salt-ssh", line 9, in <module>
    load_entry_point('salt==3002.2', 'console_scripts', 'salt-ssh')()
  File "/usr/lib/python3/dist-packages/salt/scripts.py", line 479, in salt_ssh
    client.run()
  File "/usr/lib/python3/dist-packages/salt/cli/ssh.py", line 25, in run
    ssh = salt.client.ssh.SSH(self.config)
  File "/usr/lib/python3/dist-packages/salt/client/ssh/__init__.py", line 234, in __init__
    self._expand_target()
  File "/usr/lib/python3/dist-packages/salt/client/ssh/__init__.py", line 373, in _expand_target
    if hostname in [host_id, roster_data[host_id].get("host")]:
AttributeError: 'str' object has no attribute 'get'

Expected behavior
This is how it works in version 3001.
It also works if I query the roster file instead of directly pointing at .

$ salt-ssh 'ex1.example.com' test.ping
ex1.example.com:
    True

Versions Report

salt --versions-report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

$ salt-ssh --versions-report
Salt Version:
          Salt: 3002.2

Dependency Versions:
          cffi: Not Installed
      cherrypy: Not Installed
      dateutil: 2.8.1
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 2.8
       libgit2: Not Installed
      M2Crypto: Not Installed
          Mako: 1.0.3
       msgpack: 0.6.2
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     pycparser: Not Installed
      pycrypto: 2.6.1
  pycryptodome: 3.4.7
        pygit2: Not Installed
        Python: 3.5.2 (default, Oct  7 2020, 17:19:02)
  python-gnupg: 0.3.8
        PyYAML: 3.11
         PyZMQ: 17.1.2
         smmap: Not Installed
       timelib: Not Installed
       Tornado: 4.5.3
           ZMQ: 4.1.4

System Versions:
          dist: ubuntu 16.04 Xenial Xerus
        locale: UTF-8
       machine: x86_64
       release: 4.4.0-194-generic
        system: Linux
       version: Ubuntu 16.04 Xenial Xerus

Additional context
As I mentioned above, I can fix this by downgrading to salt-ssh version 3001.

[max-arnold]

Could be caused by #58163 that was shipped in 3002.

Cc: @Ch3LL

[Ch3LL]

I am having a hard time replicating this. I've targeted a host via IP address and hostname thats in a roster and one thats not in a roster, without issue. Any other information to help replicate this issue?

[ghost]

I'm still able to replicate this issue.

As I mentioned, I'm using the Ubuntu 16.04 desktop operating system.
I installed salt with the following commands:

UBUNTU_RELEASE=$(lsb_release -rs)
UBUNTU_CODENAME=$(lsb_release -cs)
wget -O - https://repo.saltstack.com/py3/ubuntu/${UBUNTU_RELEASE}/amd64/latest/SALTSTACK-GPG-KEY.pub | sudo apt-key add -
echo "deb http://repo.saltstack.com/py3/ubuntu/${UBUNTU_RELEASE}/amd64/latest ${UBUNTU_CODENAME} main" | sudo tee /etc/apt/sources.list.d/saltstack.list
sudo apt-get update
sudo apt-get install salt-ssh

This gives me salt-ssh version 3002.2.

The repo I'm working on is open source. The only notable thing I can think of between this and other installs is that Saltfile and salt-config/master configures all of the salt paths to local directories.
https://github.com/open-contracting/deploy/

I'm running salt-ssh against a server that isn't mentioned in the roster at all. You can see the command output here.
I haven't redacted any information from this one.

command output

``` $ salt-ssh 'ex1.example.com' test.ping [ERROR ] No matching targets found in roster. $ salt-ssh 'rhtest.dogshost.com' test.ping [ERROR ] An un-handled exception was caught by salt's global exception handler: AttributeError: 'str' object has no attribute 'get' Traceback (most recent call last): File "/usr/bin/salt-ssh", line 9, in load_entry_point('salt==3002.2', 'console_scripts', 'salt-ssh')() File "/usr/lib/python3/dist-packages/salt/scripts.py", line 479, in salt_ssh client.run() File "/usr/lib/python3/dist-packages/salt/cli/ssh.py", line 25, in run ssh = salt.client.ssh.SSH(self.config) File "/usr/lib/python3/dist-packages/salt/client/ssh/__init__.py", line 234, in __init__ self._expand_target() File "/usr/lib/python3/dist-packages/salt/client/ssh/__init__.py", line 373, in _expand_target if hostname in [host_id, roster_data[host_id].get("host")]: AttributeError: 'str' object has no attribute 'get' Traceback (most recent call last): File "/usr/bin/salt-ssh", line 9, in load_entry_point('salt==3002.2', 'console_scripts', 'salt-ssh')() File "/usr/lib/python3/dist-packages/salt/scripts.py", line 479, in salt_ssh client.run() File "/usr/lib/python3/dist-packages/salt/cli/ssh.py", line 25, in run ssh = salt.client.ssh.SSH(self.config) File "/usr/lib/python3/dist-packages/salt/client/ssh/__init__.py", line 234, in __init__ self._expand_target() File "/usr/lib/python3/dist-packages/salt/client/ssh/__init__.py", line 373, in _expand_target if hostname in [host_id, roster_data[host_id].get("host")]: AttributeError: 'str' object has no attribute 'get' $ salt-ssh '46.43.3.55' test.ping [ERROR ] An un-handled exception was caught by salt's global exception handler: AttributeError: 'str' object has no attribute 'get' Traceback (most recent call last): File "/usr/bin/salt-ssh", line 9, in load_entry_point('salt==3002.2', 'console_scripts', 'salt-ssh')() File "/usr/lib/python3/dist-packages/salt/scripts.py", line 479, in salt_ssh client.run() File "/usr/lib/python3/dist-packages/salt/cli/ssh.py", line 25, in run ssh = salt.client.ssh.SSH(self.config) File "/usr/lib/python3/dist-packages/salt/client/ssh/__init__.py", line 234, in __init__ self._expand_target() File "/usr/lib/python3/dist-packages/salt/client/ssh/__init__.py", line 373, in _expand_target if hostname in [host_id, roster_data[host_id].get("host")]: AttributeError: 'str' object has no attribute 'get' Traceback (most recent call last): File "/usr/bin/salt-ssh", line 9, in load_entry_point('salt==3002.2', 'console_scripts', 'salt-ssh')() File "/usr/lib/python3/dist-packages/salt/scripts.py", line 479, in salt_ssh client.run() File "/usr/lib/python3/dist-packages/salt/cli/ssh.py", line 25, in run ssh = salt.client.ssh.SSH(self.config) File "/usr/lib/python3/dist-packages/salt/client/ssh/__init__.py", line 234, in __init__ self._expand_target() File "/usr/lib/python3/dist-packages/salt/client/ssh/__init__.py", line 373, in _expand_target if hostname in [host_id, roster_data[host_id].get("host")]: AttributeError: 'str' object has no attribute 'get' ```

The output of the first command salt-ssh 'ex1.example.com' test.ping is interesting, testing more around this, I get a different output from salt version 3002.2 if salt has connected to the server successfully before using version 3001.
Perhaps something is getting cached that shouldn't be? this might also be a distraction from the error.
I thought salt should try connecting to the server regardless of if it's in roster or not.

[javicacheiro]

I am having the same issue with 3003.

After some debugging I have found that there are several possible outcomes.

Using the minion's IP address

salt-ssh using the IP address always fails with the above error.

Using the minion's hostname

In this case the behaviour is different depending if the minion is defined in the roster file and if it is or not resolved via the DNS (only direct resolution, not inverse)

Defined in roster	Resolved by DNS	Resolved by /etc/hosts	Result
No	Yes	No	Fail
Yes	Yes	No	Fail
Yes	No	Yes	Fail
Yes	No	No	Success

So in the case the minion is defined in the roster it works but only if the host is not resolved by DNS or /etc/hosts.

[javicacheiro]

After some debugging I found the origin of the issue.

The problem is when the roster file has one-line definitions of the type:
myhost: 10.1.2.3

This syntax worked in previous versions until #58163.

Now the code in client/ssh/__init__.py instead of:
if hostname in [host_id, roster_data.get("host")]:
uses:
if hostname in [host_id, roster_data[host_id].get("host")]:

That is fine of course, and @Ch3LL did a good work and this is the correct call, but the problem is that now the resulta of roster_data[host_id] must be a ordereddict for the get to work. In cases like the one-line minion definitions it will fail because it is trying to call the get method over a str type.

I would propose to check the type and in case it is not and ordereddict just to use the value given, something like:

if isinstance(roster_data[host_id], salt.utils.odict.OrderedDict):
    roster_host = roster_data[host_id].get("host")
else:
    roster_host = roster_data[host_id]

If you think it would be useful I can create a pull request.

[Ch3LL]

@javicacheiro thanks for taking the time to dive into this. That looks like an appropriate fix. Would you mind pushing that up as a PR with tests. I can help write tests too if needed. Just ping me on the PR when you push it up.

[javicacheiro]

Great, let's proceed that way.

[Ch3LL]

Did you want to push the PR or I can?

[javicacheiro]

@Ch3LL just created the PR, I would need some help with the tests. Looking at existing pytests/unit I see client/ssh and roster/test_dir.py, but I don't know if I miss something because I do not find the original tests for the expand_target function.

[Ch3LL]

There are a couple of expand_target tests here: tests/unit/client/test_ssh.py. Did you want me to write the tests? More than willing to just let me know.

[javicacheiro]

If you can it would be great!

[Ch3LL]

Will do. I'll try to get to it this week or the beginning of next week :)