Find file
Fetching contributors…
Cannot retrieve contributors at this time
206 lines (141 sloc) 6.89 KB
Ocean Observatories Initiative Cyberinfrastructure (OOI CI)
Integrated Observatory Network (ION)
pyon - Python COI Capability Container and Core Modules R2
(C) UC Regents, 2010-2011
Install the following if not yet present:
* OS Packages and package management:
For Mac, use homebrew
> /usr/bin/ruby -e "$(/usr/bin/curl -fksSL"
- python 2.7.2
Download and use installer on a Mac.
If you need to build from source (like on some linux distributions), enable the following in the build:
- sqlite
- git 1.7.7: Use brew or download the Mac or Linux installer and run it
- couchdb 1.1.0 (optional if memory mockdb is used)
You will need to upgrade couchdb to at least 1.1.0.
- rabbitmq 2.6.1 or later (recommended, but can use rabbitmq on amoeba)
Use brew.
Alternative: download generic Linux version and unpack into a suitable directory.
- Install libevent, libyaml, couchdb, python, and rabbitmq with Homebrew
> brew install libevent libyaml couchdb python rabbitmq hdf5
You can even reinstall git using brew to clean up your /usr/local directory
(If you have trouble with brew 'MD5 mismatch' errors, try running 'brew update'
and try the install again)
Linux: Note that many installs have much older versions installed by default.
- libevent: Install with package manager
- libyaml: You need the -dev or -devel version. If you don't have such a package, use extern/
- couchdb: Make sure to have the appropriate version
* python packages and environment management:
- pip
> easy_install pip
- virtualenv and virtualenvwrapper modules for your python 2.7 installation
> easy_install --upgrade virtualenv
> easy_install --upgrade virtualenvwrapper
Note: This may require Mac's XCode (use XCode 3.3 free version)
* Setup a virtualenv to for Pyon (use any name you like):
> mkvirtualenv --no-site-packages --python=python2.7 ion27
> workon ion27
Use Buildout to install the python package dependencies with:
> python
> bin/buildout
Link git submodule ion-definitions, containing object and service definitions and resource files:
git submodule update --init
Note: This git-submodule always references a specific commit of ion-definitions used by pyon.
Hint: The .gitmodules file declares the submodule as read only GIT reference (https: instead of git@).
If you have problems with a read only (anonymous) clone because of a missing public key,
edit .git/config and change the last url to git://
If you want to avoid entering your password for every push to the submodule,
edit extern/ion-definitions/.git/config and change the remote reference to:
To update the code in pyon and the specific commit in the submodule:
> git pull
> git submodule update
- The submodule update does NOT happen automatically
- The submodule update does NOT pull the branch head of the submodule.
- To change the submodule commit within pyon, cd into the submodule and do a `git pull`, then commit pyon.
The obj/ subdir is a symlink to the checkout's extern/ion-definitions/objects.
The res/ subdir is a symlink to the checkout's extern/ion-definitions/res.
See more about submodules:
Generate service definition interfaces and abstract base classes files:
> bin/generate_interfaces
Note: These will be placed in the interfaces/ subdirectory (and not in GIT nor in the egg).
Run couchdb in second terminal before starting the container or running tests:
> couchdb
> sudo sbin/rabbitmq-server
Add the current directory to your PYTHONPATH for an easier time running scripts. Add the following
line to your ~/.profile (create it if it doesnt not exist, or use bash_profile if you already have that):
> export PYTHONPATH=.
Run the unittests:
bin/nosetests -a UNIT
Run code coverage:
bin/nosetests -w pyon --with-coverage --cover-package=pyon
Provide local configuration overrides in:
Start the capability container in interactive mode:
> bin/pycc
List supported arguments:
> bin/pycc --help
Start the CC with a deploy file:
> bin/pycc --rel res/deploy/r2deploy.yml
Start the CC with a process to execute and then quit:
> bin/pycc --proc examples.service.hello_service.HelloService
Command line configuration options for pycc:
Use a different message broker:
> bin/pycc
Use a different database server:
> bin/pycc
Set the container's system name:
> bin/pycc
USE Pycharm or any other Python IDE to debug; breakpoints and step-by-step are possible.
Set the executable to the bin/pycc and set the working directory to the repository root.
Use the -n option to avoid the interactive ipython shell. It does not work property with the debugger.
You will need to make sure the Pycharm debug egg is available in your path, something like:
> export PYTHONPATH=.:/Applications/PyCharm\
Remote debugging of services in a running container is possible too. Setup a remote debugging
configuration to use the localhost and port 8585. First start the remote debuging server, then
start the container from the command line below (the -n seems to be required)
> bin/pyccd -n --rel res/deploy/r2services.yml
View and manipulate the CouchDB datastore, by accessing http://localhost:5984/_utils
Make sure to look in the examples directory for a README and examples.