updating keystone developer documentation

updating docstrings to remove errors in automodule generation
updating setup.py to generate source documentation
blueprint keystone-documentation
bug 843056

Change-Id: Ie8dfedc89c1a6d9ffa5106d29dd19837b02746ce

doc/README.rst

@@ -0,0 +1,30 @@
+==========================
+Building the Documentation
+==========================
+
+Using setup.py
+==============
+
+From the project root, just type::
+
+  % setup.py build_sphinx
+
+
+
+Manually
+========
+
+  1. Generate the code.rst file so that Sphinx will pull in our docstrings::
+
+      % ./generate_autodoc_index.py
+
+  2. Run `sphinx_build`::
+
+      % sphinx-build -b html source build/html
+
+
+The docs have been built
+========================
+
+Check out the `build` directory to find them. Yay!
+

doc/generate_autodoc_index.py

@@ -0,0 +1,57 @@
+#!/usr/bin/env python
+"""Generates files for sphinx documentation using a simple Autodoc based
+template.
+To use:
+cd keystone/doc
+./generate_autodoc_index.py
+"""
+
+import os
+
+RSTDIR="source/sourcecode"
+SOURCE="../keystone"
+
+def find_autodoc_modules():
+    """returns a list of modules in the SOURCE directory"""
+    modlist = []
+    for root, dirs, files in os.walk(SOURCE):
+        for filename in files:
+            if filename.endswith(".py"):
+                # root = ../keystone/test/unit
+                # filename = base.py
+                # remove the first two pieces of the root
+                elements = root.split(os.path.sep)[1:]
+                # and get the base module name
+                base, extension = os.path.splitext(filename)
+                if not (base == "__init__"):
+                    elements.append(base)
+                modlist.append(".".join(elements))
+    return modlist
+
+if not(os.path.exists(RSTDIR)):
+    os.mkdir(RSTDIR)
+
+INDEXOUT = open("%s/autoindex.rst" % RSTDIR, "w")
+INDEXOUT.write("Source Code Index\n")
+INDEXOUT.write("=================\n")
+INDEXOUT.write(".. toctree::\n")
+INDEXOUT.write("   :maxdepth: 1\n")
+INDEXOUT.write("\n")
+
+for module in find_autodoc_modules():
+    generated_file = "%s/%s.rst" % (RSTDIR, module)
+    print "Generating %s" % generated_file
+
+    INDEXOUT.write("   %s\n" % module)
+    FILEOUT = open(generated_file, "w")
+    FILEOUT.write("The :mod:`%s` Module\n" % module)
+    FILEOUT.write("=============================="
+                  "=============================="
+                  "==============================\n")
+    FILEOUT.write(".. automodule:: %s\n" % module)
+    FILEOUT.write("  :members:\n")
+    FILEOUT.write("  :undoc-members:\n")
+    FILEOUT.write("  :show-inheritance:\n")
+    FILEOUT.close()
+
+INDEXOUT.close()

doc/source/architecture.rst

@@ -17,3 +17,81 @@
 Keystone Architecture
 =====================
 
+Keystone has two major components: Authentication and a Service Catalog.
+
+Authentication
+--------------
+
+In providing a token-based authentication service for OpenStack, keystone
+has several major concepts:
+
+Tenant
+    A grouping used in OpenStack to contain relevant OpenStack services. A
+    tenant maps to a Nova "project-id", and in object storage, a tenant can
+    have multiple containers. Depending on the installation, a tenant can
+    represent a customer, account, organization, or project.
+
+User
+    Represents an individual within OpenStack for the purposes of
+    authenticating them to OpenStack services. Users have credentials, and may
+    be assigned to one or more tenants. When authenticated, a token is
+    provided that is specific to a single tenant.
+
+Credentials
+    Password or other information that uniquely identifies a User to Keystone
+    for the purposes of providing a token.
+
+Token
+    A token is an arbitrary bit of text that is used to share authentication
+    with other OpenStack services so that Keystone can provide a central
+    location for authenticating users for access to OpenStack services. A
+    token may be "scoped" or "unscoped". A scoped token represents a user
+    authenticated to a Tenant, where an unscoped token represents just the
+    user.
+
+    Tokens are valid for a limited amount of time and may be revoked at any
+    time.
+
+Role
+    A role is a set of permissions to access and use specific operations for
+    a given user when applied to a tenant. Roles are logical groupings of
+    those permissions to enable common permissions to be easily grouped and
+    bound to users associated with a given tenant.
+
+Service Catalog
+---------------
+
+Keystone also provides a list of REST API endpoints as a definitive list for
+an OpenStack installation. Key concepts include:
+
+Service
+    An OpenStack service such as nova, swift, glance, or keystone. A service
+    may have one of more endpoints through which users can interact with
+    OpenStack services and resources.
+
+Endpoint
+    A network accessible address (typically a URL) that represents the API
+    interface to an OpenStack service. Endpoints may also be grouped into
+    templates which represent a group of consumable OpenStack services
+    available across regions.
+
+Template
+    A collection of endpoints representing a set of consumable OpenStack
+    service endpoints.
+
+Components of Keystone
+----------------------
+
+Keystone includes a command-line interface which interacts with the Keystone
+API for administrating keystone and related services.
+
+* keystone - runs both keystone-admin and keystone-service
+* keystone-admin - the administrative API for manipulating keystone
+* keystone-service - the user oriented API for authentication
+* keystone-manage - the command line interface to manipulate keystone
+
+Keystone also includes WSGI middelware to provide authentication support
+for Nova and Swift.
+
+Keystone uses a built-in SQLite datastore - and may use an external LDAP
+service to authenticate users instead of using stored credentials.

doc/source/conf.py

@@ -44,6 +44,7 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
 extensions = ['sphinx.ext.autodoc',
               'sphinx.ext.coverage',
+              'sphinx.ext.viewcode',
               'sphinx.ext.ifconfig',
               'sphinx.ext.intersphinx',
               'sphinx.ext.pngmath',

doc/source/gettingstarted.rst

@@ -63,4 +63,4 @@ Curl examples
    :maxdepth: 1
 
    adminAPI_curl_examples
-   serviceAPI_curl_examples 
+   serviceAPI_curl_examples

doc/source/index.rst

@@ -27,19 +27,35 @@ Using Keystone
 ==============
 
 .. toctree::
-   :maxdepth: 1
+    :maxdepth: 1
 
-   installing
-   gettingstarted
+    installing
+    gettingstarted
 
 Developer Docs
 ==============
 
 .. toctree::
-   :maxdepth: 1
+    :maxdepth: 1
 
-   architecture
-   community
+    architecture
+    community
+
+Man Pages
+=========
+
+.. toctree::
+    :maxdepth: 1
+
+    man/keystonemanage.rst
+
+Source Code
+===========
+
+.. toctree::
+    :maxdepth: 1
+
+    sourcecode/autoindex
 
 Outstanding Documentation Tasks
 ===============================