New module: routeros — manage MikroTik RouterOS

[heuels]

SUMMARY

For quite some time I have been seeing questions popping up on StackOverflow, official wiki and even here (#29333) to add native support for managing MikroTik RouterOS devices to Ansible. There have been several projects, but all of them either don't support the latest Ansible release or are long abandoned. This pull request implements a simple network_cli module to support RouterOS devices. I also plan to further support this module and add more commands and functionality to it in the future.

ISSUE TYPE

• New Module Pull Request

COMPONENT NAME

routeros_command

ANSIBLE VERSION

ansible 2.7.0.dev0 (routeros 4cd32a8cc6) last updated 2018/06/05 21:37:11 (GMT +300)
  config file = None
  configured module search path = ['/Users/heuels/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/heuels/.pyenv/versions/3.6.5/lib/python3.6/site-packages/ansible/lib/ansible
  executable location = /Users/heuels/.pyenv/versions/3.6.5/lib/python3.6/site-packages/ansible/bin/ansible
  python version = 3.6.5 (default, May  8 2018, 10:48:52) [GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)]

ADDITIONAL INFORMATION

[ansibot]

cc @Qalthos @ganeshrn @gdpak @gundalow @privateip @rcarrillocruz @trishnaguha
click here for bot help

[ansibot]

The test ansible-test sanity --test boilerplate [explain] failed with 2 errors:

lib/ansible/modules/network/routeros/routeros_command.py:0:0: missing: __metaclass__ = type
lib/ansible/modules/network/routeros/routeros_command.py:0:0: missing: from __future__ import (absolute_import, division, print_function)

The test ansible-test sanity --test validate-modules [explain] failed with 3 errors:

lib/ansible/modules/network/routeros/routeros_command.py:0:0: E307 version_added should be 2.7. Currently 2.6
lib/ansible/modules/network/routeros/routeros_command.py:0:0: E316 ANSIBLE_METADATA.metadata_version: not a valid value for dictionary value @ data['metadata_version']. Got '1.0'
lib/ansible/modules/network/routeros/routeros_command.py:0:0: E322 "provider" is listed in the argument_spec, but not documented in the module

click here for bot help

[ansibot]

The test ansible-test sanity --test validate-modules [explain] failed with 2 errors:

lib/ansible/modules/network/routeros/routeros_command.py:0:0: E322 "authorize" is listed in the argument_spec, but not documented in the module
lib/ansible/modules/network/routeros/routeros_command.py:0:0: E325 argument_spec for "authorize" defines type="bool" but documentation does not

click here for bot help

[felixfontein]

I don't have any experience so far with network modules, so maybe I'm doing something really wrong ;) I've tried to use this module, but so far I always get timeouts. I'm using the following playbook:

---
- name: RouterOS test
  hosts: router
  gather_facts: no
  tasks:
  - routeros_command:
      commands: /system routerboard print
      host: 192.168.88.1
      port: 22
      transport: routeros
      username: admin
      password: xxx
    register: output
  - debug:
      var: output

My inventory:

[router]
router ansible_host=192.168.88.1

[router:vars]
ansible_connection=network_cli
ansible_network_os=routeros
ansible_user=admin
ansible_remote_user=admin
ansible_ssh_pass=xxx

When running ansible-playbook with -vvv, I always get (after waiting for some time):

The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/paramiko/channel.py", line 683, in recv
    out = self.in_buffer.read(nbytes, self.timeout)
  File "/usr/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 160, in read
    raise PipeTimeout()
paramiko.buffered_pipe.PipeTimeout

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../ansible/bin/ansible-connection", line 87, in start
    self.connection._connect()
  File ".../ansible/lib/ansible/plugins/connection/network_cli.py", line 325, in _connect
    newline=self._terminal.terminal_inital_prompt_newline)
  File ".../ansible/lib/ansible/plugins/connection/network_cli.py", line 400, in receive
    data = self._ssh_shell.recv(256)
  File "/usr/lib/python3.6/site-packages/paramiko/channel.py", line 685, in recv
    raise socket.timeout()
socket.timeout

fatal: [router]: FAILED! => {
    "msg": ""
}

[heuels]

@felixfontein Thanks for testing it out!
The reason you're getting the timeout is because you're trying to login to a shell with console colours enabled. You need to append +cet to your username in order to disable them, like this:

routeros:
  hosts:
    example:
      ansible_host: 192.168.88.1
      ansible_user: admin+cet
      ansible_ssh_pass: ''
      ansible_network_os: routeros
      ansible_connection: network_cli

And here is a sample playbook:

---
  - hosts: example
    gather_facts: no
    connection: network_cli
    tasks:
      - routeros_command:
          commands:
            - /system resource print
        register: output
      - debug: 
          var: output

Unfortunately this is the only way to disable console colours, you can't do it after you're logged in. I guess I could put it into the docs for this module, however it would be much better if Ansible could add the suffix all by itself. But I'm pretty new to Ansible myself and have not been able to figure out a way yet.

I have tried doing it using an action plugin, but it does not seem to be called anymore when I set the network_cli connection type.

[heuels]

I have figured it out. The action plugin now automatically appends +cet to your username.

[felixfontein]

Yes, adding +cet indeed does the trick; with that it works.

I've tried your updated version (with the action plugin), but I still need to append +cet for it to work. I've tried debugging this; first, it looks like the action plugin must be called routeros_command, otherwise it won't be executed. Next, the SSH connection is opened long before the action plugin is called. The SSH connection is opened by plugins/connection/network_cli.py (line 350). Without the +cet, the action plugin object isn't even constructed before the connection timeouts.

I've tried tinkering with various things, and I'm not convinced there's anything we can do about it (without deeper changes in network_cli). @resmo: do you have any idea? Or know whom to ask about this?

[felixfontein]

I made a little bit progress: if I copy the following lines from plugins/terminal/aruba.py:

    ansi_re = [
        # check ECMA-48 Section 5.4 (Control Sequences)
        re.compile(br'(\x1b\[\?1h\x1b=)'),
        re.compile(br'((?:\x9b|\x1b\x5b)[\x30-\x3f]*[\x20-\x2f]*[\x40-\x7e])'),
        re.compile(br'\x08.')
    ]

to plugins/terminal/routeros.py, I only need to append +t (disable terminal detection) to the username.
I guess if we could somehow handle the terminal detection, we could survive without +cet. In any case, without +t or +cet, the last thing sent is '\r\n\r\r\x1bZ ' (after some banner text).
[... some time later ...]
I can get a bit further, by adding:

    terminal_initial_prompt = [
        br'\x1bZ',
    ]
    terminal_initial_answer = b'\x1b/Z'

The router then sends '\x1b<' and waits again. This apparently means "Enter ANSI mode". I don't know how to react on that. (Also, I don't think network_cli allows a proper reaction. Whatever a proper reaction would be...)
So I guess we will have to at least enforce +t.

[ansibot]

The test ansible-test sanity --test docs-build [explain] failed with 31 errors:

docs/docsite/rst/index.rst:0:0: unknown: 
docs/docsite/rst/index.rst:0:0: unknown: +----------------+-------------------------+----------------------+----------------------+------------------+------------------+
docs/docsite/rst/index.rst:0:0: unknown: +------------------+-------------------------+----------------------+----------------------+------------------+------------------+
docs/docsite/rst/index.rst:0:0: unknown: +------------------+-------------------------+----------------------+----------------------+------------------+------------------+
docs/docsite/rst/index.rst:0:0: unknown: +------------------+-------------------------+----------------------+----------------------+------------------+------------------+
docs/docsite/rst/index.rst:0:0: unknown: +------------------+-------------------------+----------------------+----------------------+------------------+------------------+
docs/docsite/rst/index.rst:0:0: unknown: +------------------+-------------------------+----------------------+----------------------+------------------+------------------+
docs/docsite/rst/index.rst:0:0: unknown: +------------------+-------------------------+----------------------+----------------------+------------------+------------------+
docs/docsite/rst/index.rst:0:0: unknown: +------------------+-------------------------+----------------------+----------------------+------------------+------------------+
docs/docsite/rst/index.rst:0:0: unknown: +------------------+-------------------------+----------------------+----------------------+------------------+------------------+
docs/docsite/rst/index.rst:0:0: unknown: +------------------+-------------------------+----------------------+----------------------+------------------+------------------+
docs/docsite/rst/index.rst:0:0: unknown: +------------------+-------------------------+----------------------+----------------------+------------------+------------------+
docs/docsite/rst/index.rst:0:0: unknown: +------------------+-------------------------+----------------------+----------------------+------------------+------------------+
docs/docsite/rst/index.rst:0:0: unknown: +------------------+-------------------------+----------------------+----------------------+------------------+------------------+
docs/docsite/rst/index.rst:0:0: unknown: +------------------+-------------------------+----------------------+----------------------+------------------+------------------+
docs/docsite/rst/index.rst:0:0: unknown: +==================+=========================+======================+======================+==================+==================+
docs/docsite/rst/index.rst:0:0: unknown: | Arista EOS*      | ``eos``                 | in v. >=2.5          | N/A                  | in v. >=2.6      |  in v. >=2.4     |
docs/docsite/rst/index.rst:0:0: unknown: | Cisco ASA        | ``asa``                 | in v. >=2.5          | N/A                  | N/A              |  in v. >=2.4     |
docs/docsite/rst/index.rst:0:0: unknown: | Cisco IOS XR*    | ``iosxr``               | in v. >=2.5          | N/A                  | N/A              |  in v. >=2.4     |
docs/docsite/rst/index.rst:0:0: unknown: | Cisco IOS*       | ``ios``                 | in v. >=2.5          | N/A                  | N/A              |  in v. >=2.4     |
docs/docsite/rst/index.rst:0:0: unknown: | Cisco NX-OS*     | ``nxos``                | in v. >=2.5          | N/A                  | in v. >=2.6      |  in v. >=2.4     |
docs/docsite/rst/index.rst:0:0: unknown: | Extreme IronWare | ``ironware``            | in v. >=2.5          | N/A                  | N/A              |  in v. >=2.5     |
docs/docsite/rst/index.rst:0:0: unknown: | F5 BIG-IP        | N/A                     | N/A                  | N/A                  | N/A              |  in v. >=2.0     |
docs/docsite/rst/index.rst:0:0: unknown: | F5 BIG-IQ        | N/A                     | N/A                  | N/A                  | N/A              |  in v. >=2.0     |
docs/docsite/rst/index.rst:0:0: unknown: | Junos OS*        | ``junos``               | in v. >=2.5          | in v. >=2.5          | N/A              |  in v. >=2.4     |
docs/docsite/rst/index.rst:0:0: unknown: | Network OS       | ``ansible_network_os:`` | network_cli          | netconf              | httpapi          | local            |
docs/docsite/rst/index.rst:0:0: unknown: | Nokia SR OS      | ``sros``                | in v. >=2.5          | N/A                  | N/A              |  in v. >=2.4     |
docs/docsite/rst/index.rst:0:0: unknown: | RouterOS       | ``routeros``            | in v. >=2.7          | N/A                  | N/A              |  N/A             |
docs/docsite/rst/index.rst:0:0: unknown: | VyOS*            | ``vyos``                | in v. >=2.5          | N/A                  | N/A              |  in v. >=2.4     |
docs/docsite/rst/index.rst:0:0: unknown: |..                |                         | ``ansible_connection:`` settings available                                        |
docs/docsite/rst/network/user_guide/platform_index.rst:25:0: warning: Malformed table.

The test ansible-test sanity --test import --python 2.6 [explain] failed with 1 error:

lib/ansible/modules/network/routeros/routeros_command.py:133:0: ImportError: cannot import name check_args

The test ansible-test sanity --test import --python 2.7 [explain] failed with 1 error:

lib/ansible/modules/network/routeros/routeros_command.py:133:0: ImportError: cannot import name check_args

The test ansible-test sanity --test import --python 3.5 [explain] failed with 1 error:

lib/ansible/modules/network/routeros/routeros_command.py:133:0: ImportError: cannot import name 'check_args'

The test ansible-test sanity --test import --python 3.6 [explain] failed with 1 error:

lib/ansible/modules/network/routeros/routeros_command.py:133:0: ImportError: cannot import name 'check_args'

The test ansible-test sanity --test import --python 3.7 [explain] failed with 1 error:

lib/ansible/modules/network/routeros/routeros_command.py:133:0: ImportError: cannot import name 'check_args' from 'ansible.module_utils.network.routeros.routeros' (/root/ansible/test/sanity/import/lib/ansible/module_utils/network/routeros/routeros.py)

The test ansible-test sanity --test rstcheck [explain] failed with 1 error:

docs/docsite/rst/network/user_guide/platform_index.rst:25:0: Malformed table.

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

lib/ansible/modules/network/routeros/routeros_command.py:0:0: E321 Exception attempting to import module for argument_spec introspection, 'cannot import name 'check_args''

click here for bot help

[ansibot]

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

lib/ansible/modules/network/routeros/routeros_command.py:0:0: E322 "waitfor" is listed in the argument_spec, but not documented in the module

click here for bot help

[felixfontein]

I guess you should change ios.yml to routeros.yml ;-)

[heuels]

My bad, thanks! 🙂

[heuels]

@ganeshrn I've addressed all your comments, thank you for reviewing!

[ganeshrn]

@heuels @felixfontein Thank you!

[felixfontein]

@ganeshrn Thank you very much for reviewing and merging this!
@heuels And thank you very much for all your work for this!
I'm really looking forward to use this :)