Merge pull request #108 from pydap/server-docs

Update the server docs
Fixes #46
Rewrote the Apache section, rewrote the uwsgi section, removed the paste references, added
instructions for using the standalone server script, added a Flask section, added
docker links.

docs/conf.py

@@ -40,7 +40,7 @@
 
 # General information about the project.
 project = u'Pydap'
-copyright = u'2003-2014, Roberto De Almeida'
+copyright = u'2003-2017, Pydap contributors'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the

docs/index.rst

@@ -40,23 +40,18 @@ This will install Pydap together with all the required dependencies. You can now
             [  6.38000011e-01,   8.95384610e-01,   7.21666634e-01,
                8.10000002e-01]]], dtype=float32), array([ 366.]), array([-69., -67., -65., -63.]), array([ 41.,  43.,  45.,  47.])]
 
-For more information, please check the documentation on `using Pydap as a client <client.html>`_. Pydap also comes with a simple server, implemented as a `WSGI <http://wsgi.org/>`_ application. To use it, you first need to install a data handler:
+For more information, please check the documentation on `using Pydap as a client <client.html>`_. Pydap also comes with a simple server, implemented as a `WSGI <http://wsgi.org/>`_ application. To use it, you first need to install Pydap with the server extras dependencies and a data handler:
 
 .. code-block:: bash
 
+    $ pip install Pydap[server]
     $ pip install pydap.handlers.netcdf
 
-This will install support for `netCDF <http://www.unidata.ucar.edu/software/netcdf/>`_ files; more `handlers <handlers.html>`_ for different formats are available, if necessary. Now create a directory for your server data:
+This will install support for `netCDF <http://www.unidata.ucar.edu/software/netcdf/>`_ files; more `handlers <handlers.html>`_ for different formats are available, if necessary. To run a simple standalone server just issue the command:
 
 .. code-block:: bash
 
-    $ paster create -t pydap myserver
-
-To run the server just issue the command:
-
-.. code-block:: bash
-
-    $ paster serve ./myserver/server.ini
+    $ pydap --data ./myserver/data/ --port 8001
 
 This will start a standalone server running on http://localhost:8001/, serving netCDF files from ``./myserver/data/``, similar to the test server at http://test.pydap.org/. Since the server uses the `WSGI <http://wsgi.org/>`_ standard, it can easily be run behind Apache. The `server documentation <server.html>`_ has more information on how to better deploy Pydap.
 

docs/server.rst

@@ -1,44 +1,78 @@
 Running a server
 ================
 
-Pydap comes with a lightweight and scalable OPeNDAP server, implemented as a `WSGI <http://wsgi.org/>`_ application. Being a WSGI `application <http://wsgi.org/wsgi/Applications>`_, Pydap can run on a variety of `servers <http://wsgi.org/wsgi/Servers>`_, including Apache, IIS or even as a standalone Python process. It can also be seamless combined with different `middleware <http://wsgi.org/wsgi/Middleware_and_Utilities>`_ for authentication/authorization, GZip compression, and much more.
+Pydap comes with a lightweight and scalable OPeNDAP server, implemented as a `WSGI <http://wsgi.org/>`_ application. Being a WSGI `application <http://wsgi.org/wsgi/Applications>`_, Pydap can run on a variety of `servers <http://wsgi.org/wsgi/Servers>`_, and frameworks including Apache, `Nginx <https://www.nginx.com/>`_, IIS, `uWSGI <https://uwsgi-docs.readthedocs.io/en/latest/>`_, `Flask <http://flask.pocoo.org/>`_ or as a standalone Python process. It can also be seamless combined with different `middleware <http://wsgi.org/wsgi/Middleware_and_Utilities>`_ for authentication/authorization, GZip compression, and much more.
+
+There is no one right way to run Pydap server; your application requirements and software stack will inform your deployment decisions. In this chapter we provide a few examples to try and get you on the right track.
 
 In order to distribute your data first you need to install a proper `handler <handlers.html>`_, that will convert the data format to the Pydap data model. 
 
 Running standalone
 ------------------
 
-If you just want to quickly test the Pydap server, you can run it as a standalone Python application using the server that comes with `Python Paste <http://pythonpaste.org/>`_:
+If you just want to quickly test the Pydap server, you can run it as a standalone Python application using the server that comes with `Python Paste <http://pythonpaste.org/>`_ and `gunicorn <http://gunicorn.org/>`_. To run the server, first make sure that you have installed Pydap with the server extras dependencies:
+
+.. code-block:: bash
+
+    $ pip install Pydap[server]
+
+and then just run the ``pydap`` script that pip installs into your bin directory:
 
 .. code-block:: bash
 
-    $ paster create -t pydap myserver
+    $ pydap --data /path/to/my/data/files --port 8080
 
-To run the server just issue the command:
+To change the default directory listing, the help page and the HTML form, you can point a switch to your template directory
 
 .. code-block:: bash
 
-    $ paster serve ./myserver/server.ini
+    $ pydap --data /path/to/my/data/files --templates /path/to/my/templates.
+
+The HTML form template is fairly complex, since it contain some application logic and some Javascript code, so be careful to not break anything.
+
+.. _wsgi-application-section:
+
+WSGI Application
+----------------
+
+Pydap follows the `WSGI specification <https://www.fullstackpython.com/wsgi-servers.html>`_, and most web servers gateways simply require a WSGI callable and a small amount of boiler plate code. Pydap provides the ``DapServer`` class which is a WSGI callable located in the ``pydap.wsgi.app`` module. A simple WSGI application script file would be something like this:
+
+.. code-block:: python
+
+    from pydap.wsgi.app import DapServer
+    application = DapServer('/path/to/my/data/files')
+
+
+Flask
+-----
+
+The `Flask <http://flask.pocoo.org/>`_ framework simply requires a couple more function calls to spin up an application server. A simple server would looks something like this (your mileage may vary):
+
+.. code-block:: python
 
-This will run the server on http://localhost:8001/, serving files from ``./myserver/data/``. By default the server will listen only to local requests, ie, from the same machine. You can change this by editing the ``server.ini`` file; and you can also change the port number, though for ports lower than 1024 you will probably need to run the script as root.
+    from flask import Flask
+    from pydap.wsgi.app import DapServer
 
-To change the default directory listing, the help page and the HTML form, simply edit the corresponding templates in ``./myserver/templates/``. The HTML form template is fairly complex, since it contain some application logic and some Javascript code, so be careful to not break anything.
+    application = DapServer('/path/to/my/data/files')
+    app = Flask(__name__)
+    app.wsgi_app = application
+    app.run('0.0.0.0', 8000)
 
-Running Pydap with Apache
--------------------------
+Apache
+------
 
-For a robust deployment you should run Pydap with Apache, using `mod_wsgi <http://modwsgi.org/>`_. After `installing mod_wsgi <http://code.google.com/p/modwsgi/wiki/InstallationInstructions>`_, create a sandbox in a directory *outside* your DocumentRoot, say ``/var/www/pydap/``, using `virtualenv <http://pypi.python.org/pypi/virtualenv>`_:
+For a robust deployment you can run Pydap with Apache, using `mod_wsgi <http://modwsgi.org/>`_. After `installing mod_wsgi <http://code.google.com/p/modwsgi/wiki/InstallationInstructions>`_, create a sandbox in a directory *outside* your DocumentRoot, say ``/var/www/pydap/``, using `a virtual environment <https://docs.python.org/3/library/venv.html>`_:
 
 .. code-block:: bash
 
     $ mkdir /var/www/pydap
-    $ python virtualenv.py /var/www/pydap/env
+    $ python3 -m venv /var/www/pydap/env
 
 If you want the sandbox to use your system installed packages (like Numpy, e.g.) you can use the ``--system-site-packages`` flag:
 
 .. code-block:: bash
 
-    $ python virtualenv.py --system-site-packages /var/www/pydap/env
+    $ python3 -m venv --system-site-packages /var/www/pydap/env
 
 Now let's activate the sandbox and install Pydap -- this way the module and its dependencies can be isolated from the system libraries:
 
@@ -47,32 +81,16 @@ Now let's activate the sandbox and install Pydap -- this way the module and its
     $ source /var/www/pydap/env/bin/activate.sh
     (env)$ pip install Pydap
 
-And now we can install the basic server files:
-
-.. code-block:: bash
-
-    (env)$ cd /var/www/pydap
-    (env)$ paster create -t pydap server
-
-Now edit the file ``/var/www/pydap/server/apache/pydap.wsgi`` and insert the two following lines in the beginning of the file, forcing mod_wsgi to use the Python modules from the sandbox:
-
-.. code-block:: python
-
-    import site
-    site.addsitedir('/var/www/pydap/env/lib/python2.X/site-packages')
-    
-You'll need to insert the correct path (including Python version) to your sandbox ``site-packages`` directory, of course. After this, your file should look like this:
+Create a `WSGI script file <http://modwsgi.readthedocs.io/en/develop/user-guides/quick-configuration-guide.html#mounting-the-wsgi-application>`_ somewhere convenient (e.g. /var/www/pydap/server/apache/pydap.wsgi) that reads something like this:
 
 .. code-block:: python
 
     import site
-    site.addsitedir('/var/www/pydap/env/lib/python2.X/site-packages')
-
-    import os
-    from paste.deploy import loadapp
+    # force mod_wsgi to use the Python modules from the sandbox
+    site.addsitedir('/var/www/pydap/env/lib/pythonX.Y/site-packages')
 
-    config = os.path.join(os.path.dirname(__file__), '../server.ini')
-    application = loadapp('config:%s' % config)
+    from pydap.wsgi.app import DapServer
+    application = DapServer('/path/to/my/data/files')
 
 Now create an entry in your Apache configuration pointing to the ``pydap.wsgi`` file you just edited. To mount the server on the URL ``/pydap``, for example, you should configure it like this:
 
@@ -119,8 +137,9 @@ This is the file I use for the `test.pydap.org <http://test.pydap.org/>`_ virtua
 
 You can find more information on the `mod_wsgi configuration guide <http://code.google.com/p/modwsgi/wiki/QuickConfigurationGuide>`_. Just remember that Pydap is a WSGI application like any other else, so any information on WSGI applications applies to it as well.
 
-Running Pydap with uWSGI
-------------------------
+
+uWSGI
+-----
 
 `uWSGI <http://projects.unbit.it/uwsgi/>`_ is a "fast, self-healing and developer/sysadmin-friendly application container server coded in pure C" that can run Pydap. This is the recommended way to run Pydap if you don't have to integrate it with other web applications. Simply install uWSGI, follow the instructions in the last section in order to create a virtualenv and Pydap installation:
 
@@ -131,7 +150,8 @@ Running Pydap with uWSGI
     $ source /var/www/pydap/env/bin/activate.sh
     (env)$ pip install Pydap uWSGI
     (env)$ cd /var/www/pydap
-    (env)$ paster create -t pydap server
+
+Create a WSGI application file myapp.wsgi :ref:`as above <wsgi-application-section>`
 
 Now create a file in ``/etc/init/pydap.conf`` with the content:
 
@@ -148,11 +168,16 @@ Now create a file in ``/etc/init/pydap.conf`` with the content:
         --http-socket 0.0.0.0:80 \
         -H /var/www/pydap/env \
         --master --processes 4 \
-        --paste config:/var/www/pydap/server/server.ini
+        --wsgi-file /var/www/pydap/myapp.wsgi
 
 In order to make it run automatically during boot on Linux you can type:
 
 .. code-block:: bash
 
     $ sudo initctl reload-configuration
 
+
+Docker
+------
+
+Users have `reported success <https://github.com/pydap/pydap/issues/46>`_ deploying Pydap with a docker image built with nginx + uWSGI + Flask (based on https://hub.docker.com/r/tiangolo/uwsgi-nginx-flask/. A full configuration is somewhat beyond the scope of this documentation (since it will depend on your requirements and your software stack), but it is certainly possible.