ION Capability Container and core framework (python) - an advanced fork is available at
Python Other
Latest commit 122c629 Jun 16, 2014 Michael Meisinger Merge pull request #513 from mmeisinger/license
Set license to BSD
Failed to load latest commit information.
examples Fix OOIION-938: Rolling window process saturation Jun 22, 2013
extern publish resource agent lifecycle events ("agent started", "agent stop… May 9, 2014
prototype Remove references to Apache license from source code headers Jun 16, 2014
putil Remove references to Apache license from source code headers Jun 16, 2014
pyon Remove references to Apache license from source code headers Jun 16, 2014
scripts Remove references to Apache license from source code headers Jun 16, 2014
.coveragerc Exclude coverage May 10, 2013
.gitignore Ignore coverage directory Feb 29, 2012
.gitmodules Switch back to https URL for submodules. Dec 8, 2011
.pyon_root_marker Directory cleanup Oct 14, 2011
CHANGES Changes, Todo updated; sorted list of messaging in shell API Oct 27, 2011
Doxyfile Reduce doxygen gen files Feb 9, 2012
LICENSE.txt Set license to BSD Jun 16, 2014
README Set license to BSD Jun 16, 2014
autolaunch.cfg Install numpy in autolaunch Mar 5, 2013 New 2.2.0 buildout bootstrap Jul 10, 2013
build.xml Include examples directory for ant clean Sep 26, 2012
buildout.cfg Pin gevent version buildout Dec 2, 2013
control_cc Add initial buildout with associated changes Oct 27, 2011 Add release script Jul 4, 2012
gcoverage_buildout.cfg Fix pyon buildout for coverage Jul 9, 2013
obj Converted obj to a submodule Oct 26, 2011
res Added res as symlink to extern/ion-definitions/res Nov 2, 2011
setup.cfg Add examples to coverage Dec 7, 2011 Set license to BSD Jun 16, 2014


Ocean Observatories Initiative Cyberinfrastructure (OOI CI)
Integrated Observatory Network (ION) - OOINet

pyon - Python Capability Container and Core Modules

(C) The Regents of the University of California, 2010-2014
Open source under BSD license. See LICENSE.txt for details.


Install the following if not yet present:

* OS Packages and package management:
For Mac, use homebrew
    > /usr/bin/ruby <(curl -fsSkL
- python 2.7.x
    > brew install python --framework --universal
    Note: see 'brew doctor' for any path issues
- git 1.7.7 or higher: Use brew or download the Mac or Linux installer and run it
    > brew install git

- Install core packages with Homebrew

    > brew install libevent libyaml zeromq rabbitmq hdf5 pkg-config netcdf udunits

    Linux: Note that many installs have much older versions installed by default.
    (If you have trouble with brew 'MD5 mismatch' errors, try running 'brew update'
    and try the install again)
    For any kind of installation trouble run 'brew doctor' and fix any issues.

- Install Postgres with Homebrew

    > brew install postgres postgis

    If you have trouble with the installation, please see:

    As user postgres:
    > createuser -s ion

* Python packages and environment management:
- pip
    > easy_install pip
- virtualenv and virtualenvwrapper modules for your python 2.7 installation
    > easy_install --upgrade virtualenv
    > easy_install --upgrade virtualenvwrapper
    Note: This require Mac's XCode with installed command line tools

* Setup a virtualenv to use for Pyon:
    > mkvirtualenv --no-site-packages pyon

* Pin setuptools in your virtualenv
    > pip install -U setuptools==0.8

* Install numpy to the virtualenv
    > pip install numpy==1.7.1



Use Buildout to install the python package dependencies with:
    > python -v 2.2.0
    > bin/buildout

** Add following to your .bashrc profile on Macs if buildout fails to
    compile for packages such as gevent, pyml, etc.
    export C_INCLUDE_PATH=/usr/local/include


Link git submodule ion-definitions, containing object and service definitions and resource files:
    git submodule update --init

Note: This git-submodule always references a specific commit of ion-definitions used by pyon.

Hint: The .gitmodules file  declares the submodule as read only GIT reference (https: instead of git@).
    If you have problems with a read only (anonymous) clone because of a missing public key,
    edit .git/config and change the last url to git://

    If you want to avoid entering your password for every push to the submodule,
    edit extern/ion-definitions/.git/config and change the remote reference to:

To update the code in pyon and the specific commit in the submodule:
    > git pull
    > git submodule update

- The submodule update does NOT happen automatically
- The submodule update does NOT pull the branch head of the submodule.
- To change the submodule commit within pyon, cd into the submodule and do a `git pull`, then commit pyon.

The obj/ subdir is a symlink to the checkout's extern/ion-definitions/objects.
The res/ subdir is a symlink to the checkout's extern/ion-definitions/res.

See more about submodules:


Generate service definition interfaces and abstract base classes files:
    > bin/generate_interfaces
    Note: These will be placed in the interfaces/ subdirectory (and not in GIT nor in the egg).



Run postgresql in second terminal before starting the container or running tests:
    > /usr/local/bin/postgres -D /usr/local/var/postgres

    > rabbitmq-server


Add the current directory to your PYTHONPATH for an easier time running
scripts. Add the following line to your ~/.profile (create it if it
doesn't exist, or use bash_profile if you already have that):
    export PYTHONPATH=.


Run the unittests:
    bin/nosetests -a UNIT

Run code coverage:
    bin/nosetests -w pyon --with-coverage --cover-package=pyon


Provide local configuration overrides in:

Start the capability container in interactive mode:
    > bin/pycc

List supported arguments:
    > bin/pycc --help

Start the CC with a deploy file:
    > bin/pycc --rel res/deploy/r2deploy.yml

Start the CC with a process to execute and then quit:
    > bin/pycc --proc examples.service.hello_service.HelloService

Command line configuration options for pycc:
    Use a different message broker:
    > bin/pycc

    Use a different database server:
    > bin/pycc

    Set the container's system name:
    > bin/pycc


USE Pycharm or any other Python IDE to debug; breakpoints and step-by-step are possible.
Set the executable to the bin/pycc and set the working directory to the repository root.
Use the -n option to avoid the interactive ipython shell. It does not work property with the debugger.

You will need to make sure the Pycharm debug egg is available in your path, something like:
    > export PYTHONPATH=.:/Applications/PyCharm\

Remote debugging of services in a running container is possible too. Setup a remote debugging
configuration to use the localhost and port 8585. First start the remote debuging server, then
start the container from the command line below (the -n seems to be required)
    > bin/pyccd -n --rel res/deploy/r2services.yml


Use psql


Make sure to look in the examples directory for a README and examples.