add how to create HttpApi plugins for network modules (ansible#54340)

* initial port over from gdoc

* crosslink httpapi pages

* Add some examples?

* Add quick example of using Connection

* Fix indentation of python code blocks

* moved to a full developer guide for networks

* Try to clarify some examples

* Fix typos

How did I even do that?

docs/docsite/rst/dev_guide/index.rst

@@ -22,6 +22,7 @@ Find the task that best describes what you want to do:
    * What do I need to know before I start coding?
    * I want to :ref:`set up my Python development environment <environment_setup>`.
    * I want to :ref:`get started writing a module <developing_modules_general>`.
+   * I want to :ref:`write a network module <developing_modules_network>`.
    * I want to :ref:`write a Windows module <developing_modules_general_windows>`.
    * I want to `write an Amazon module <https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/amazon/GUIDELINES.md>`_.
    * I want to :ref:`write a series of related modules <developing_modules_in_groups>` that integrate Ansible with a new product (for example, a database, cloud provider, network platform, etc.).

docs/docsite/rst/network/dev_guide/developing_plugins_network.rst

@@ -0,0 +1,118 @@
+
+.. _developing_modules_network:
+.. _developing_plugins_network:
+
+**************************
+Network connection plugins
+**************************
+
+Each network connection plugin has a set of its own plugins which provide a specification of the
+connection for a particular set of devices. The specific plugin used is selected at runtime based
+on the value of the ``ansible_network_os`` variable assigned to the host. This variable should be
+set to the same value as the name of the plugin to be loaed. Thus, ``ansible_network_os=nxos``
+will try to load a plugin in a file named ``nxos.py``, so it is important to name the plugin in a
+way that will be sensible to users.
+
+Public methods of these plugins may be called from a module or module_utils with the connection
+proxy object just as other connection methods can. The following is a very simple example of using
+such a call in a module_utils file so it may be shared with other modules.
+
+.. code-block:: python
+
+  from ansible.module_utils.connection import Connection
+
+  def get_config(module):
+      # module is your AnsibleModule instance.
+      connection = Connection(module._socket_path)
+
+      # You can now call any method (that doesn't start with '_') of the connection
+      # plugin or its platform-specific plugin
+      return connection.get_config()
+
+.. contents::
+   :local:
+
+.. _developing_plugins_httpapi:
+
+Developing httpapi plugins
+==========================
+
+:ref:`httpapi plugins <httpapi_plugins>` serve as adapters for various HTTP(S) APIs for use with the ``httpapi`` connection plugin. They should implement a minimal set of convenience methods tailored to the API you are attempting to use.
+
+Specifically, there are a few methods that the ``httpapi`` connection plugin expects to exist.
+
+Making requests
+---------------
+
+The ``httpapi`` connection plugin has a ``send()`` method, but an httpapi plugin needs a ``send_request(self, data, **message_kwargs)`` method as a higher-level wrapper to ``send()``. This method should prepare requests by adding fixed values like common headers or URL root paths. This method may do more complex work such as turning data into formatted payloads, or determining which path or method to request. It may then also unpack responses to be more easily consumed by the caller.
+
+.. code-block:: python
+
+   from ansible.module_utils.six.moves.urllib.error import HTTPError
+
+   def send_request(self, data, path, method='POST'):
+       # Fixed headers for requests
+       headers = {'Content-Type': 'application/json'}
+       try:
+           response, response_content = self.connection.send(path, data, method=method, headers=headers)
+       except HTTPError as exc:
+           return exc.code, exc.read()
+
+       # handle_response (defined separately) will take the format returned by the device
+       # and transform it into something more suitable for use by modules.
+       # This may be JSON text to Python dictionaries, for example.
+       return handle_response(response_content)
+
+Authenticating
+--------------
+
+By default, all requests will authenticate with HTTP Basic authentication. If a request can return some kind of token to stand in place of HTTP Basic, the ``update_auth(self, response, response_text)`` method should be implemented to inspect responses for such tokens. If the token is meant to be included with the headers of each request, it is sufficient to return a dictionary which will be merged with the computed headers for each request. The default implementation of this method does exactly this for cookies. If the token is used in another way, say in a query string, you should instead save that token to an instance variable, where the ``send_request()`` method (above) can add it to each request
+
+.. code-block:: python
+
+   def update_auth(self, response, response_text):
+       cookie = response.info().get('Set-Cookie')
+       if cookie:
+           return {'Cookie': cookie}
+
+       return None
+
+If instead an explicit login endpoint needs to be requested to receive an authentication token, the ``login(self, username, password)`` method can be implemented to call that endpoint. If implemented, this method will be called once before requesting any other resources of the server. By default, it will also be attempted once when a HTTP 401 is returned from a request.
+
+.. code-block:: python
+
+   def login(self, username, password):
+       login_path = '/my/login/path'
+       data = {'user': username, 'password': password}
+
+       response = self.send_request(data, path=login_path)
+       try:
+           # This is still sent as an HTTP header, so we can set our connection's _auth
+           # variable manually. If the token is returned to the device in another way,
+           # you will have to keep track of it another way and make sure that it is sent
+           # with the rest of the request from send_request()
+           self.connection._auth = {'X-api-token': response['token']}
+       except KeyError:
+           raise AnsibleAuthenticationFailure(message="Failed to acquire login token.")
+
+Similarly, ``logout(self)`` can be implemented to call an endpoint to invalidate and/or release the current token, if such an endpoint exists. This will be automatically called when the connection is closed (and, by extension, when reset).
+
+.. code-block:: python
+
+   def logout(self):
+       logout_path = '/my/logout/path'
+       self.send_request(None, path=logout_path)
+
+       # Clean up tokens
+       self.connection._auth = None
+
+Error handling
+--------------
+
+The ``handle_httperror(self, exception)`` method can deal with status codes returned by the server. The return value indicates how the plugin will continue with the request:
+
+* A value of ``true`` means that the request can be retried. This my be used to indicate a transient error, or one that has been resolved. For example, the default implementation will try to call ``login()`` when presented with a 401, and return ``true`` if successful.
+
+* A value of ``false`` means that the plugin is unable to recover from this response. The status code will be returned to the calling module as an exception. Any other value will be taken as a nonfatal response from the request. This may be useful if the server returns error messages in the body of the response. Returning the original exception is usually sufficient in this case, as HTTPError objects have the same interface as a successful response.
+
+For example httpapi plugins, see the `source code for the httpapi plugins <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/httpapi>`_ included with Ansible Core.

docs/docsite/rst/network/dev_guide/index.rst

@@ -0,0 +1,26 @@
+.. _network_developer_guide:
+
+**************************************
+Developer Guide for Network Automation
+**************************************
+
+Welcome to the Developer Guide for Ansible Network Automation!
+
+**Who should use this guide?**
+
+If you want to extend Ansible for Network Automation by creating a module or plugin, this guide is for you. This guide is specific to networking. You should already be familiar with how to create, test, and document modules and plugins, as well as the prerequisites for getting your module or plugin accepted into the main Ansible repository.  See the  :ref:`developer_guide` for details. Before you proceed, please read:
+
+* How  to :ref:`add a custom plugin or module locally <developing_locally>`.
+* How to figure out if :ref:`developing a module is the right approach <module_dev_should_you>` for my use case.
+* How to :ref:`set up my Python development environment <environment_setup>`.
+* How to :ref:`get started writing a module <developing_modules_general>`.
+
+
+Find the network developer task that best describes what you want to do:
+
+   * I want to :ref:`develop a network connection plugin <developing_plugins_network>`.
+
+.. toctree::
+  :maxdepth: 1
+
+  developing_plugins_network

docs/docsite/rst/network/index.rst

@@ -15,3 +15,4 @@ For documentation on using a particular network module, consult the :ref:`list o
 
    getting_started/index
    user_guide/index
+   dev_guide/index

docs/docsite/rst/plugins/httpapi.rst

@@ -18,7 +18,7 @@ others might be usable on a variety of platforms (RESTCONF).
 Adding httpapi plugins
 -------------------------
 
-You can extend Ansible to support other APIs by dropping a custom plugin into the ``httpapi_plugins`` directory.
+You can extend Ansible to support other APIs by dropping a custom plugin into the ``httpapi_plugins`` directory. See :ref:`developing_plugins_httpapi` for details.
 
 .. _using_httpapi:
 
@@ -51,6 +51,8 @@ Use ``ansible-doc -t httpapi <plugin name>`` to see detailed documentation and e
 
    :ref:`Ansible for Network Automation<network_guide>`
        An overview of using Ansible to automate networking devices.
+   :ref:`Developing network modules<developing_modules_network>`
+       How to develop network modules.
    `User Mailing List <https://groups.google.com/group/ansible-devel>`_
        Have a question?  Stop by the google group!
    `irc.freenode.net <http://irc.freenode.net>`_