The Neptune Programming Language
Ruby C JavaScript Erlang Shell
Switch branches/tags
Nothing to show
Pull request Compare This branch is 1 commit ahead, 30 commits behind shatterednirvana:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


=begin rdoc
Neptune: A Domain Specific Language for Deploying HPC 
Software on Cloud Platforms

Neptune provides programmers with a simple interface
by which they can deploy MPI, X10, MapReduce, UPC, and Erlang jobs
to without needing to know the particulars of the underlying
cloud platform. You only need to give Neptune your code,
tell it how many machines to run on and where to put the output:
Neptune handles everything else. No more writing configuration
files, having to start up NFS on all your machines, yada yada yada.
Neptune works together with supported cloud platforms (currently
AppScale is recommended) and can deploy over anything AppScale
can - Xen or KVM virtual machines as well as Eucalyptus and
Amazon EC2. There's nothing virtualization specific in there,
so in theory any machine installed with the AppScale software
should work fine.

Although Neptune is designed to automate deploying HPC jobs,
it also can be used to deploy other types of software. For example,
Neptune has support for user-specified scaling of the underlying
cloud platform: users can write Neptune jobs that manually add
load balancers, application servers, or database servers to a running
AppScale deployment. Additionally, as of version 0.0.2, remote
compiling can also be performed: just give Neptune the path to
the directory you want to compile and be sure to include a Makefile
in it! Neptune will run 'make' on it (you can specify which target
to make as well) and return to you a folder containing the standard
out and standard error of the make command.

By default, Neptune jobs store their outputs in the underlying database
that AppScale is running over. As of Neptune 0.0.5, job outputs can
also be stored in Amazon S3, Eucalyptus Walrus, and Google Storage.

Sample Neptune job scripts can be found in samples. Test cases can
be found in the test folder, with the standard naming convention
- ts_neptune is the test suite runner, with tc_* containing test
cases for each type of job that Neptune offers. Before running
ts_neptune, you should export the environment variable APPSCALE_HEAD_NODE,
which should be set to the IP address of the AppScale machine that runs
the Shadow daemon (a.k.a. the Master AppController). Running in the top-level directory will run rcov
and generate the coverage reports automatically - currently this process
takes about an hour.

Developed by Chris Bunch as part of the AppScale project.
See LICENSE for the specifics of the New BSD License
by which Neptune is released.

Check us out on the web:

Contributors welcome! We would love to add support for other
cloud platforms and test out Neptune more on non-virtualized
deployments, as well as adding capabilities for other types
of computation. We would also like to refactor Neptune to use
symbols instead of instance variables for running jobs: this
will likely appear in a future release as well.

An academic paper describing Neptune is in the works - check
in for a link to that as it becomes available.

Version History:

April 8, 2011 - 0.0.8 released, fixing MapReduce support
for both regular Hadoop and Hadoop Streaming. Also increased
code coverage to cover a number of failure scenarios.

April 2, 2011 - 0.0.7 released, adding automatic test suite
and many bug fixes for all scenarios. rcov can also be used
to generate test coverage information: current coverage stats
can be found in coverage directory. MapReduce broken at the
moment - will fix in next release

March 28, 2011 - 0.0.6 released, adding support for input jobs,
so users can place data in the datastore without having to run
any computation

March 18, 2011 - 0.0.5 released, adding support for storage outside
of AppScale to be used. Tested and working with Amazon S3 and Google

February 10, 2011 - 0.0.4 released, adding UPC and Erlang support,
and restructuring syntax to pass in hashes to method calls instead
of passing in blocks

February 4, 2011 - 0.0.3 released, allowing users to use
Neptune properly as a gem within Ruby code

February 4, 2011 - 0.0.2 released, adding support for remote

January 27, 2011 - 0.0.1 released, with initial support for
MPI, X10, and MapReduce.