This document is intended for developers who wish to change Mastr-MS or see how it works.
Mastr-MS is Django web application and works with both PostgreSQL and MySQL.
The Mastr-MS code is hosted at BitBucket.
Git is required to check out the code.
All new bugs/feature requests should be submitted to the issue tracker:
If all significant changes have a ticket associated, then release notes can be accurately generated.
Mastr-MS was originally hosted on Google Code, where there still remains a small bug list:
We develop on Ubuntu. Installing the following packages is recommended:
sudo apt-get install postgresql-server \ ipython python-pip python-sphinx \ python-wxgtk2.8 xvfb xserver-xephyr
Packages helpful but not really required:
python-django \ python-werkzeug python-django-extensions \ python-selenium chromium-chromedriver \
Building a CentOS RPM
The RPM build is unlikely to work unless done under CentOS. Assuming
the Mastr-MS source code is checked out at
/usr/local/src, you can
build an RPM in more-or-less the normal way by running these
CCGSOURCEDIR=/usr/local/src export CCGSOURCEDIR cd $CCGSOURCEDIR && rpmbuild -bb centos/mastrms.spec
The spec file requires
CCGSOURCEDIR to be set. It will download
all the python dependencies with
pip, create the RPM, and output
~/rpmbuild/RPMS (or the location you have configured in
Building a Debian/Ubuntu Package
To get started, you need the build dependencies:
sudo apt-get install dpkg-dev sudo mk-build-deps -i debian/control
To rebuild the debian package, run:
dpkg-buildpackage -us -uc
The packaging method is standard Debian, except for the use of
Uploading to the CCG Apt Repo
To upload, you need the following in
[DEFAULT] default_host_main = ccg [ccg] fqdn = staging.ccgapps.com.au method = scp login = ubuntu incoming = /data/aptrepo/repo/incoming post_upload_command = ssh firstname.lastname@example.org 'aptrepo update'
Then run the
dput command to upload:
dput ccg ../mastr-ms_1.13.0_amd64.deb
Make sure you remember to include
ccg in the command line or it
will FTP upload your package into Debian incoming.
Upgrades and database migrations
dbconfig-common system will automatically migrate the database
for the sysadmin. To let that happen, you need to define which Debian
package version the migrations correspond to.
The mapping of versions is in the script
How to build the sync client
The support libraries and binaries are in the
You should be able to build the client on a 32-bit Windows XP box, by installing these resources as described below.
Initial Setup of build environment
- Copy client and supportwin32 over to the windows machine.
- Install Python27 (utf8 encoding for json is broken in earlier versions)
- Install wx for python 2.7
- Install py2exe for python 2.7
- Install NSIS
- Install 7Zip
- Extract the
- Copy the extension into
- Open a shell
- Change dir to the
Building the code
First, make sure you have updated the version number in
to be unique, and sequentially higher than previous ones.
On Windows box, get the latest client dir. Run:
c:\Python27\python.exe setup.py bdist_esky
Then unzip the build you just did (in
dist/) so that the files are
available to the installer.
If you then ran the
nsi file (right click on the file and choose
'Compile with NSIS'), it would make an installer, using the build you
just did (in
/dist) as a source. So that is how you make an
If you want to publish just the update, you take the
.zip that is
generated in the
dist directory, and
scp it to the
distribution URL. Currently, this is on S3, under
As long as the
version.py version was incremented, a new version
will be available to esky.
When building with Esky, if you get the message
../main.py. The file is in use by another process/, this probably
means your python code fails a syntax check or has some other runtime
- Testing requirements:
- chrome/firefox webdriver
- python xvfbwrapper
Command to run:
./manage.py test --exclude=yaphc --exclude=esky --exclude=httplib2
Mastr-MS contains some system tests as well as unit tests. The current test classes are:
.. autoclass:: mastrms.mdatasync_client.client.test.TestClient :members:
To generate sample data files like the lab instrument software would create, use the :class:`Simulator` class. Simulator can be run as a standalone WxPython GUI, or operated programmatically through :class:`Worklist`.
.. autoclass:: mastrms.mdatasync_client.client.Simulator.Simulator :members:
.. autoclass:: mastrms.mdatasync_client.client.Simulator.WorkList
For testing you usually don't want to do actual file transfers, but
you do want to check that
rsync was called. For this, we put a
mock command into the
PATH which can be queried from the test cases.
.. autoclass:: mastrms.mdatasync_client.client.test.fake_rsync.FakeRsync
The documentation is in :ref:`Sphinx <sphinx:contents>` format under
docs subdirectory of the source. To build it, simply run:
This list, written by Brad, may be useful in understanding the system.
Goals of MASTR
- To provide a web based tool for experimental design, sample metadata configuration, and sample data acquisition.
- To enable researchers and scientists from geographically separate institutions to work together on experiments, analysis, and to be able to share results and outcomes.
- To enable institutions to provide quotes for analysis work to third-parties, with automatic linkage through to the relevant projects and experiments.
Features of MASTR
- User / Group administration
- Experimental design, catering for:
- User roles and access control
- Sample origin metadata
- Sample timeline and treatment metadata
- Sample tracking
- Sample information import / export via CSV
- Standard Operating Procedure upload
- Run creation, generating worklists for the purposes of instrument automation.
- Fully customisable rules system for worklist generation
- Sample blocks, order, randomisation, and solvents/blanks can be specified as a programmable template.
- Worklist rulesets can be rolled out per individual, or shared with groups or the entire institution
- Rulesets can be branched and cloned
- Runs and Experiments can also be cloned for convenience.
- Data acquisition, consisting of:
- A program which runs on the computer connected to the instrument which processes MASTR worklists
- The program will check periodically for filesets related to experiment runs being performed for MASTR
- The sample data is compressed and uploaded to the MASTR-connected storage, and optionally archived on the client machine.
- The sample data is then securely available to relevant users through the MASTR web interface for viewing or download.
- Full end-to-end data acquisition, from experimental design to sample file access.
- Quote requests and tracking
- A system for third parties to request quotes for analysis work of any of the institutions in MASTR
- Institution Administrators or Node Representatives can review the requests and service replies, with optional PDF attachments.
- Full quote event history is maintained.
- Quotes in the system can be linked to Projects / Experiments
- Modern technologies
- Able to be accessed in all major web browsers
- Lightweight and powerful UI
- Open data formats and transports used (rsync, json)
- Open Source code repository
.. toctree:: tests