Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Added documentation for new developers.

  • Loading branch information...
commit 8d715abeb47d4d69b229737d1da694c8bd521853 1 parent 7015223
@AdrianGaudebert AdrianGaudebert authored
View
126 docs/addaservice.rst
@@ -0,0 +1,126 @@
+.. index:: addaservice
+
+.. _addaservice-chapter:
+
+Add a service to the Middleware
+===============================
+
+Architecture overview
+---------------------
+
+The middleware is a simple REST API providing JSON data depending on the URL
+that is called. It is made of a list of services, each one binding a certain
+URL with parameters. Documentation for each service is available in the
+:ref:`middleware-chapter` page.
+
+Those services are not containing any code, but are only interfaces. They are
+using other resources from the external module. That external module is
+composed of one submodule for each external resource we are using. For example,
+there is a PostgreSQL submodule, an ElasticSearch submodule and a HBase
+submodule.
+
+You will also find some common code among external resources in socorro.lib.
+
+Class hierarchy
+---------------
+
+.. image:: images/middleware-hierarchy.png
+
+Create the service
+------------------
+
+First create a new file for your service in ``socorro/middleware/`` and call it
+``nameofservice_service.py``. This is a convention for the next version of our
+config manager. Then create a class inside as follow::
+
+ import logging
+
+ from socorro.middleware.service import DataAPIService
+
+ logger = logging.getLogger("webapi")
+
+
+ class MyService(DataAPIService):
+
+ default_service_order = [
+ "socorro.external.myresource",
+ "socorro.external.postgresql"
+ "socorro.external.elasticsearch"
+ ]
+ service_name = "myservice"
+ uri = "/myservice/(.*)"
+
+ def __init__(self, config):
+ super(MyService, self).__init__(config)
+ logger.debug('MyService service __init__')
+
+ def get(self, *args):
+ # Parse parameters
+ params = self.parse_query_string(args[0])
+
+ module = self.get_module(params)
+ impl = module.MyService(config=self.context)
+
+ return impl.mymethod(**params)
+
+``uri`` is the URL pattern you want to match. It is a regular expression, and
+the content of each part (``(.*)``) will be in ``args``.
+
+``service_name`` is the name of your service, and will be used to find the
+corresponding implementation resource. It has to match the filename of the
+module you need.
+
+``default_service_order`` is a configuration variable containing prefered
+implementation resources in case the global configuration value does not
+contain the needed module.
+
+If you want to add mandatory parameters, modify the URI and values will be
+passed in ``args``.
+
+Use external resources
+----------------------
+
+The ``socorro.external`` contains everything related to outer resources like
+databases. Each submodule has a base class and classes for specific
+functionalities. If the function you need for your service is not already in
+there, you create a new file and a new class to implement it. To do so,
+follow this pattern::
+
+ from socorro.external.myresource.base import MyResourceBase
+
+
+ class MyModule(MyResourceBase):
+
+ def __init__(self, *args, **kwargs):
+ super(MyModule, self).__init__(*args, **kwargs)
+
+ def mymethod(self, **kwargs):
+ do_stuff()
+ return my_json_result
+
+Configuration
+-------------
+
+Finally add your service to the list of running services in
+scripts/config/webapiconfig.py.dist as follow::
+
+ import socorro.middleware.search_service as search
+ import socorro.middleware.myservice_service as myservice # add
+
+ servicesList = cm.Option()
+ servicesList.doc = 'a python list of classes to offer as services'
+ servicesList.default = [myservice.MyService, search.Search, (...)] # add
+
+Then restart Apache and you should be good to go! If you're using a Vagrant VM,
+you can hit the middleware directly by calling
+http://socorro-api/bapi/myservice/params/.
+
+And then?
+---------
+
+Once you are done creating your service in the middleware, you might want to
+use it in the WebApp. If so, have a look at :ref:`ui-chapter`.
+
+You might also want to document it. We are keeping track of all existing
+services' documentation in our :ref:`middleware-chapter` page. Please add
+yours!
View
1  docs/development.rst
@@ -9,6 +9,7 @@ Development Discussions
:maxdepth: 2
codingconventions
+ newdeveloperguide
glossary
standalone
unittesting
View
74 docs/generalarchitecture.rst
@@ -0,0 +1,74 @@
+.. index:: generalarchitecture
+
+.. _generalarchitecture-chapter:
+
+General architecture of Socorro
+===============================
+
+If you clone our `git repository <https://github.com/mozilla/socorro>`_, you
+will find the following folders. Here is what each of them contains:
+
++--------------+-------------------------------------------------------------+
+| Folder | Description |
++==============+=============================================================+
+| analysis/ | Contains metrics jobs such as mapreduce. Will be moved. |
++--------------+-------------------------------------------------------------+
+| config/ | Contains the Apache configuration for the different parts |
+| | of the Socorro application. |
++--------------+-------------------------------------------------------------+
+| docs/ | Documentation of the Socorro project (the one you are |
+| | reading right now). |
++--------------+-------------------------------------------------------------+
+| scripts/ | Scripts for launching the different parts of the Socorro |
+| | application. |
++--------------+-------------------------------------------------------------+
+| socorro/ | Core code of the Socorro project. |
++--------------+-------------------------------------------------------------+
+| sql/ | SQL scripts related to our PostgreSQL database. Contains |
+| | schemas and update queries. |
++--------------+-------------------------------------------------------------+
+| thirparty/ | External libraries used by Socorro. |
++--------------+-------------------------------------------------------------+
+| tools/ | External tools used by Socorro. |
++--------------+-------------------------------------------------------------+
+| webapp-php/ | Front-end PHP application (also called UI). See |
+| | :ref:`ui-chapter`. |
++--------------+-------------------------------------------------------------+
+
+Socorro submodules
+------------------
+
+The core code module of Socorro, called ``socorro``, contains a lot of code.
+Here are descriptions of every submodule in there:
+
++-------------------+---------------------------------------------------------------+
+| Module | Description |
++===================+===============================================================+
+| collector | All code related to collectors. |
++-------------------+---------------------------------------------------------------+
+| cron | All cron jobs running around Socorro. |
++-------------------+---------------------------------------------------------------+
+| database | PostgreSQL related code. |
++-------------------+---------------------------------------------------------------+
+| deferredcleanup | Osolete. |
++-------------------+---------------------------------------------------------------+
+| external | Here are APIs related to external resources like databases. |
++-------------------+---------------------------------------------------------------+
+| integrationtest | Osolete. |
++-------------------+---------------------------------------------------------------+
+| lib | Different libraries used all over Socorro’s code. |
++-------------------+---------------------------------------------------------------+
+| middleware | New-style middleware services place. |
++-------------------+---------------------------------------------------------------+
+| monitor | All code related to monitors. |
++-------------------+---------------------------------------------------------------+
+| othertests | Some other tests? |
++-------------------+---------------------------------------------------------------+
+| services | Old-style middleware services place. |
++-------------------+---------------------------------------------------------------+
+| storage | HBase related code. |
++-------------------+---------------------------------------------------------------+
+| unittest | All our unit tests are here. |
++-------------------+---------------------------------------------------------------+
+| webapi | Contains a few tools used by web-based services. |
++-------------------+---------------------------------------------------------------+
View
BIN  docs/images/middleware-hierarchy.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View
15 docs/newdeveloperguide.rst
@@ -0,0 +1,15 @@
+.. index:: newdeveloperguide
+
+.. _newdeveloperguide-chapter:
+
+New Developer Guide
+===================
+
+If you are new to Socorro, you will find here good resources to start hacking:
+
+.. toctree::
+ :maxdepth: 2
+
+ generalarchitecture
+ setupdevenv
+ addaservice
View
51 docs/setupdevenv.rst
@@ -0,0 +1,51 @@
+.. index:: setupdevenv
+
+.. _setupdevenv-chapter:
+
+Setup a development environment
+===============================
+
+The best and easiest way to get started with a complete dev environment is to
+use Vagrant and our installation script. You can find all the instructions
+here: https://github.com/rhelmer/socorro-vagrant
+
+If you don't want to use a virtual machine, you can install everything in your
+own development environment. All steps are described in
+:ref:`standalone-chapter`.
+
+Use your own git repo
+---------------------
+
+If you forked our mozilla/socorro repository, you will want to make your repo
+the origin of the repository inside your VM. Once connected through SSH into
+the VM, execute the following commands::
+
+ sudo su - socorro
+ cd /home/socorro/dev/socorro
+ edit .git/config # change `url = git@github.com:mozilla/socorro.git` with your repo's URL
+ git fetch origin
+
+Apply your changes
+------------------
+
+After that, whenever you want to see changes you made in one of your branches,
+do the following::
+
+ cd /home/socorro/dev/socorro
+ git checkout my-dev-branch
+ make install
+ sudo /etc/init.d/apache restart
+ sudo /etc/init.d/supervisor force-stop && sudo /etc/init.d/supervisor start
+
+And then from your browser access http://crash-stats/ for the UI, or
+http://socorro-api/bpapi/ for the middleware API directly.
+
+Use a shared folder
+-------------------
+
+If you don't like vim or you want to use your favorite IDE, you can easily
+create a shared folder between your OS and your VM. You can then work in your
+OS and have all your changes automatically passed to the VM.
+
+The best solution is to use NFS. There is a good documentation on Vagrant's
+website that explain it all: http://vagrantup.com/docs/nfs.html
Please sign in to comment.
Something went wrong with that request. Please try again.