Skip to content
This repository

An API to provide an efficient distributed queue on a cluster. Libcircle is currently used in production to quickly traverse and perform operations on a file tree which contains several hundred-million file nodes.

branch: master

Merge pull request #23 from fintler/future-version-bump

Bump for future release.
latest commit f8714e6174
Jon Bringhurst fintler authored
Octocat-spinner-32 auxdir Rerun autogen to activate ottxor's merge request. Downgrade autotools… February 03, 2014
Octocat-spinner-32 doc Rerun autogen to activate ottxor's merge request. Downgrade autotools… February 03, 2014
Octocat-spinner-32 experimental Revert to 2a5de28. Before tompi. December 17, 2012
Octocat-spinner-32 libcircle Whitespace formatting. February 13, 2014
Octocat-spinner-32 m4 Rerun autogen to activate ottxor's merge request. Downgrade autotools… February 03, 2014
Octocat-spinner-32 tests Rerun autogen to activate ottxor's merge request. Downgrade autotools… February 03, 2014
Octocat-spinner-32 .travis.yml Fixup travis-ci. December 17, 2012
Octocat-spinner-32 AUTHORS Inital commit of a cleaner library build setup. September 17, 2011
Octocat-spinner-32 COPYING Clarified where libcircle is from. October 06, 2011
Octocat-spinner-32 HACKING.md Revert to 2a5de28. Before tompi. December 17, 2012
Octocat-spinner-32 Makefile.am Revert to 2a5de28. Before tompi. December 17, 2012
Octocat-spinner-32 Makefile.in Rerun autogen to activate ottxor's merge request. Downgrade autotools… February 03, 2014
Octocat-spinner-32 README.md Addition of travis-ci build status. February 03, 2014
Octocat-spinner-32 aclocal.m4 Rerun autogen to activate ottxor's merge request. Downgrade autotools… February 03, 2014
Octocat-spinner-32 astyle.options Style changes. October 16, 2011
Octocat-spinner-32 autogen.sh Updated configure and autogen to work with older autotools. October 03, 2011
Octocat-spinner-32 common.mk Fixed versioning. Fixed rpm build. January 09, 2013
Octocat-spinner-32 config.h.in Rerun autogen to activate ottxor's merge request. Downgrade autotools… February 03, 2014
Octocat-spinner-32 configure Regenerate configure file. February 13, 2014
Octocat-spinner-32 configure.ac Bump for future release. February 13, 2014
Octocat-spinner-32 libcircle.h Whitespace formatting. February 13, 2014
Octocat-spinner-32 libcircle.pc.in Fixed to compile on prod server. October 21, 2011
Octocat-spinner-32 libcircle.spec Bump for future release. February 13, 2014
README.md

libcircle

libcircle is an API for distributing embarrassingly parallel workloads using self-stabilization. Details on the algorithms used may be found at http://dl.acm.org/citation.cfm?id=2389114.

Dependencies

Compile and install

The current build status is: Build Status

./configure
make all check
sudo make install

To enable output from libcircle (including fatal errors), run configure with "--enable-loglevel=number" where "number" is one of the following options:

  • "1" fatal errors only.
  • "2" errors and lower log levels.
  • "3" warnings and lower log levels.
  • "4" info messages on internal operations and lower log levels.
  • "5" fine grained debug messages and lower log levels.

RPM Build and Install

To build an RPM, use the following instructions:

  1. rpmbuild -ta libcircle-<version>.tar.gz
  2. rpm --install <the appropriate RPM files>

Developer API Information

The basic program flow when using libcircle is the following:

  1. Define callbacks which enqueue or dequeue strings from the queue.
  2. Execute the program.
#include <libcircle.h>

/* An example of a create callback defined by your program */
void my_create_some_work(CIRCLE_handle *handle)
{
    /*
     * This is where you should generate work that needs to be processed.
     * For example, if your goal is to lstat files on a large filesystem,
     * this is where you would readdir() and and enqueue directory names.
     *
     * This should be a small amount of work. For example, only enqueue the
     * filenames from a single directory.
     */
    while((data_to_process = readdir(...)) != NULL)
    {
        handle->enqueue(data_to_process);
    }
}

/* An example of a process callback defined by your program. */
void my_process_some_work(CIRCLE_handle *handle)
{
    /*
     * This is where work should be processed. For example, this is where you
     * should lstat one of the files which was placed on the queue by your
     * create_some_work callback. Again, you should try to keep this short and
     * block as little as possible.
     */
    handle->dequeue(&my_data);
    ...
    finished_work = lstat(my_data, ...);
    ...
    store_in_database(finished_work);
}

/*
 * Initialize state required by libcircle. Arguments should be those passed in
 * by the launching process. argc is a pointer to the number of arguments,
 * argv is the argument vector. The return value is the MPI rank of the current
 * process.
 */
int rank = CIRCLE_init(&argc, argv);

/*
 * Processing and creating work is done through callbacks. Here's how we tell
 * libcircle about our function which creates the initial work. For MPI nerds,
 * this is your rank 0 process.
 */
CIRCLE_cb_create(&my_create_some_work);

/*
 * After you give libcircle a way to create work, you need to tell it how that
 * work should be processed.
 */
CIRCLE_cb_process(&my_process_some_work);

/*
 * Now that everything is setup, lets execute everything.
 */
CIRCLE_begin();

/*
 * Finally, give libcircle a chance to clean up after itself.
 */
CIRCLE_finalize();
Something went wrong with that request. Please try again.