Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New HTTPAPI plugin for FortiOS #56870

Merged
merged 10 commits into from Jun 22, 2019
@@ -4,7 +4,8 @@
# still belong to the author of the module, and may assign their own license
# to the complete work.
#
# Copyright (c), Benjamin Jolivot <bjolivot@gmail.com>, 2014
# Copyright (c), Benjamin Jolivot <bjolivot@gmail.com>, 2014,
# Miguel Angel Munoz <magonzalez@fortinet.com>, 2019
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without modification,
@@ -33,6 +34,9 @@
from ansible.module_utils._text import to_native
from ansible.module_utils.basic import env_fallback

import json

# BEGIN DEPRECATED

# check for pyFG lib
try:
@@ -72,6 +76,105 @@
'-61': "Command error"
}

# END DEPRECATED


class FortiOSHandler(object):

def __init__(self, conn):
self._conn = conn

def cmdb_url(self, path, name, vdom=None, mkey=None):

url = '/api/v2/cmdb/' + path + '/' + name
if mkey:
url = url + '/' + str(mkey)
if vdom:
if vdom == "global":
url += '?global=1'
else:
url += '?vdom=' + vdom
return url

def schema(self, path, name, vdom=None):
if vdom is None:
url = self.cmdb_url(path, name) + "?action=schema"
else:
url = self.cmdb_url(path, name, vdom=vdom) + "&action=schema"

status, result_data = self._conn.send_request(url=url)

if status == 200:
if vdom == "global":
return json.loads(result_data.decode('utf-8'))[0]['results']
else:
return json.loads(result_data.decode('utf-8'))['results']
else:
return json.loads(result_data.decode('utf-8'))

def get_mkeyname(self, path, name, vdom=None):
schema = self.schema(path, name, vdom=vdom)
try:
keyname = schema['mkey']
except KeyError:
return False
return keyname

def get_mkey(self, path, name, data, vdom=None):

keyname = self.get_mkeyname(path, name, vdom)
if not keyname:
return None
else:
try:
mkey = data[keyname]
except KeyError:
return None
return mkey

def set(self, path, name, data, mkey=None, vdom=None, parameters=None):

if not mkey:
mkey = self.get_mkey(path, name, data, vdom=vdom)
url = self.cmdb_url(path, name, vdom, mkey)

status, result_data = self._conn.send_request(url=url, params=parameters, data=json.dumps(data), method='PUT')

if status == 404 or status == 405 or status == 500:
return self.post(path, name, data, vdom, mkey)
else:
return self.formatresponse(result_data, vdom=vdom)

def post(self, path, name, data, vdom=None,
mkey=None, parameters=None):

if mkey:
mkeyname = self.get_mkeyname(path, name, vdom)
data[mkeyname] = mkey

url = self.cmdb_url(path, name, vdom, mkey=None)

status, result_data = self._conn.send_request(url=url, params=parameters, data=json.dumps(data), method='POST')

return self.formatresponse(result_data, vdom=vdom)

def delete(self, path, name, vdom=None, mkey=None, parameters=None, data=None):
if not mkey:
mkey = self.get_mkey(path, name, data, vdom=vdom)
url = self.cmdb_url(path, name, vdom, mkey)
status, result_data = self._conn.send_request(url=url, params=parameters, data=json.dumps(data), method='DELETE')
return self.formatresponse(result_data, vdom=vdom)

def formatresponse(self, res, vdom=None):
if vdom == "global":
resp = json.loads(res.decode('utf-8'))[0]
resp['vdom'] = "global"
else:
resp = json.loads(res.decode('utf-8'))
return resp

# BEGIN DEPRECATED


def backup(module, running_config):
backup_path = module.params['backup_path']
@@ -198,3 +301,5 @@ def get_error_infos(self, cli_errors):

def get_empty_configuration_block(self, block_name, block_type):
return FortiConfig(block_name, block_type)

# END DEPRECATED
@@ -0,0 +1,134 @@
# This code is part of Ansible, but is an independent component.
# This particular file snippet, and this file snippet only, is BSD licensed.
# Modules you write using this snippet, which is embedded dynamically by Ansible
# still belong to the author of the module, and may assign their own license
# to the complete work.
#
# (c) 2019 Fortinet, Inc
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without modification,
# are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
# USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


from __future__ import (absolute_import, division, print_function)
__metaclass__ = type

DOCUMENTATION = """
---
author:
- Miguel Angel Muñoz (@magonzalez)
httpapi : fortios
short_description: HttpApi Plugin for Fortinet FortiOS Appliance or VM
description:
- This HttpApi plugin provides methods to connect to Fortinet FortiOS Appliance or VM via REST API
version_added: "2.9"
"""

from ansible.plugins.httpapi import HttpApiBase
from ansible.module_utils.basic import to_text
import urllib
import json
import re


class HttpApi(HttpApiBase):
def __init__(self, connection):
super(HttpApi, self).__init__(connection)

self._ccsrftoken = ''

def set_become(self, become_context):
"""
Elevation is not required on Fortinet devices - Skipped
:param become_context: Unused input.
:return: None
"""
return None

def login(self, username, password):
"""Call a defined login endpoint to receive an authentication token."""

data = "username=" + urllib.parse.quote(username) + "&secretkey=" + urllib.parse.quote(password) + "&ajax=1"
dummy, result_data = self.send_request(url='/logincheck', data=data, method='POST')
if result_data[0] != '1':
raise Exception('Wrong credentials. Please check')

def logout(self):
""" Call to implement session logout."""

self.send_request(url='/logout', method="POST")

def update_auth(self, response, response_text):
"""
Get cookies and obtain value for csrftoken that will be used on next requests
:param response: Response given by the server.
:param response_text Unused_input.
:return: Dictionary containing headers
"""

headers = {}

for attr, val in response.getheaders():
if attr == 'Set-Cookie' and 'APSCOOKIE_' in val:
headers['Cookie'] = val

elif attr == 'Set-Cookie' and 'ccsrftoken=' in val:
csrftoken_search = re.search('\"(.*)\"', val)
if csrftoken_search:
self._ccsrftoken = csrftoken_search.group(1)
This conversation was marked as resolved by mamunozgonzalez

This comment has been minimized.

Copy link
@Qalthos

Qalthos Jun 6, 2019

Contributor

And as the return is a dictionary of headers, instead of tracking self._ccsrftoken here, you should be able to set your return dictionary to be {'x-csrftoken': csrftoken_search.group(1)}, and everything should work as expected.

This comment has been minimized.

Copy link
@mamunozgonzalez

mamunozgonzalez Jun 11, 2019

Author Contributor

Thanks Qalthos, I have just tried your proposal but I am afraid I have to store the csrctoken anyway. The reason is that our fortios device sends the token on the reply to the first login request only. That means that the next request after the login works fine, as it gets the headers correctly with the csrftoken, but next requests do not get the csrftoken anymore and fortios requires the csrftoken on every request.

This comment has been minimized.

Copy link
@Qalthos

Qalthos Jun 11, 2019

Contributor

If you're only going to get APSCOOKIE_ or ccsrftoken, then you can just return None if you don't find anything and the existing token will be reused.

If you are expecting to have both, then I would just dedent the next line to be outside the for loop, so that the token is always added to the dictionary on every run. Then you should be able to at least remove the manual headers building in send_request.

This comment has been minimized.

Copy link
@mamunozgonzalez

mamunozgonzalez Jun 11, 2019

Author Contributor

Ok, I'd go for the latter option as it seems safer for our fortios. However, how can I access the headers in 'send_request' method? I need to know if there is a csrftoken set bcs if not I must raise an exception to instruct the user to log in first.

This comment has been minimized.

Copy link
@mamunozgonzalez

mamunozgonzalez Jun 12, 2019

Author Contributor

Hi again @Qalthos, I have been doing some more additional tests and I think I have no means to access the headers in 'send_request' method (which I need to know if there is a csrf_token from a previous login before). Therefore I will have to leave the code it as it is now. I would mark this as resolved, please let me know if it is not ok or if I missed something and you want me to perform more changes.

headers['x-csrftoken'] = self._ccsrftoken

return headers

def handle_httperror(self, exc):
"""
Not required on Fortinet devices - Skipped
:param exc: Unused input.
:return: exc
"""
return exc

def send_request(self, **message_kwargs):
"""
Responsible for actual sending of data to the connection httpapi base plugin.
:param message_kwargs: A formatted dictionary containing request info: url, data, method
:return: Status code and response data.
"""
url = message_kwargs.get('url', '/')
data = message_kwargs.get('data', '')
method = message_kwargs.get('method', 'GET')

if self._ccsrftoken == '' and not (method == 'POST' and 'logincheck' in url):
raise Exception('Not logged in. Please login first')

This comment has been minimized.

Copy link
@Qalthos

Qalthos Jun 20, 2019

Contributor

Perhaps I'm missing something, but I just don't see how this check should ever trigger, nor why the user should get this message.

  • Your login() method gets called automatically as the first request to the device. The response from that will pass through update_auth() and set 'x-csrftoken'
  • Requests are made to the API. Because you returned the csrf token in update_auth() previously, that is automatically added to the request headers without having to add them below.
  • At some point I guess you clear the token? This is the part I don't understand. Furthermore, as you indicate that handle_httperror() is unneeded, I'm presuming that the API returns its errors in 200 responses for some reason.
  • So you can parse the access denied error from the response to send() below, and if you need to login again, you call login() again before retrying the call and failing if it doesn't work a second time. Or you could let it pass all the way back to the module_utils, reset the connection and retry, depending on where this sort of thing is likely to happen.

Do I have something wrong here? The whole point of httpapi is to abstract the authentication dance away from the user, otherwise we might as well go back to calling open_url() in module_utils. I'm just struggling to see why telling the user they have to login is ever a thing that should happen.

This comment has been minimized.

Copy link
@mamunozgonzalez

mamunozgonzalez Jun 20, 2019

Author Contributor

Hi Qalthos,
thanks for your answer. If the system guarantees that the first call is a login then I think you are right and the exception will never raise. I can remove that part, thanks for the clarification. I just added it as an extra 'safety' measure.

Regarding the csrf token lost, our device sends the csrf token and the cookie in the first login, but in the later request it may send only the cookie, or nothing. There is no rule about that. However whenever we send a PUT or POST we must add both the token and the cookie always. And here is where I found that if I don't store the csrf token it will be lost in later requests. I have run some tests, and the ansible system do not keep the csrftoken after the first request (even if I return "None" in update_auth). Therefore, I see no other way to make it work than storing the csrf token in the class. Please let me know if I am doing something wrong. This is the code I am testing according to your comments but it does not work bcs later requests do not keep the csrf token:

    def update_auth(self, response, response_text):

        headers = {}

        for attr, val in response.getheaders():
            if attr == 'Set-Cookie' and 'APSCOOKIE_' in val:
                headers['Cookie'] = val

            elif attr == 'Set-Cookie' and 'ccsrftoken=' in val:
                csrftoken_search = re.search('\"(.*)\"', val)
                if csrftoken_search:
                    headers['x-csrftoken'] = self._ccsrftoken

        if len (headers) == 2:
            return headers
        else:
            return None

    def send_request(self, **message_kwargs):

        url = message_kwargs.get('url', '/')
        data = message_kwargs.get('data', '')
        method = message_kwargs.get('method', 'GET')

        try:
            response, response_data = self.connection.send(url, data, #headers=headers,
                                                           method=method)

            return response.status, to_text(response_data.getvalue())
        except Exception as err:
            raise Exception(err)

Please let me know if I am doing something wrong or if I misunderstood something.
Thanks!

This comment has been minimized.

Copy link
@Qalthos

Qalthos Jun 20, 2019

Contributor

I have run some tests, and the ansible system do not keep the csrftoken after the first request (even if I return "None" in update_auth).

This is what I was alluding to in my previous comments- if you need to keep the token for that purpose, then I have no problem with that. Set ccsrftoken when you see it, and just always return it in the dictianry regardless. My main goal was to impress that you are doing a bunch of things you don't have to inside of send_request().

Multiple required tokens is a thing I admit I hadn't considered before. It's possible that instead of overwriting _auth in the connection, we should instead update the dictionary. I'm not immediately sure if there are any problems that would cause, but that would also fix your issue, I think.

This comment has been minimized.

Copy link
@mamunozgonzalez

mamunozgonzalez Jun 21, 2019

Author Contributor

Ok, thanks for the clarification @Qalthos.
I have added a commit with the changes, hoping to reflect your comments as much as possible. I have reduced to a minimum the things done in 'send_request' and now I return always the csrftoken as headers in update_auth. Do you think it could be ok this way?


headers = {}
if self._ccsrftoken != '':
headers['x-csrftoken'] = self._ccsrftoken

if method == 'POST' or 'PUT':
headers['Content-Type'] = 'application/json'

try:
response, response_data = self.connection.send(url, data, headers=headers, method=method)

return response.status, to_text(response_data.getvalue())
except Exception as err:
raise Exception(err)
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.