Skip to content


Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Commits on May 28, 2008
  1. - Rewriting message_queue_adapter interface (and superclass). Makes m…

    phlapjack authored
    …uch more sense now.
    - Broke Skynet::Message into 3 classes, TaskMessage, ResultMessage and ErrorMessage (which inherits from ResultMessage)
    - MessageQueue adapters now assume if you pass :any as the task type you want, they should randomly pick from the master or task queues.
    - Added new SingleProcess message queue adapter to be the reference implementation and mock what a real message queue would work like.
    - Add new QuiQ queue, though I'm not using it yet.
    - Rafactor TupleSpace message queue adapter to use new superclass
    - Begin trying to move what was in Skynet::Task into Skynet::TaskMessage
    - Skynet::TaskMessage now runs the task.
    - Create a universal adapter tester. Just plug in your adapter and run the tests.
Commits on May 25, 2008
  1. Create a new type of Q class called qui_q

    phlapjack authored
    Create a new message q adapter for the qui_q MQ.   This will be the reference implementation for all q's
  2. Trying some different message queue architectures

    phlapjack authored
Commits on May 23, 2008
  1. update history

    phlapjack authored
  2. Make sure we don't die if we can't connect to a manager for stats

    phlapjack authored
    Some ruby installs Queue class does not have enq and deq, so I switched it to push/pop
    Fixed calculation of untaken_future_tasks in mysql message queue adaptor
  3. Fix to future tasks in stats

    phlapjack authored
  4. Add future tasks to stats

    phlapjack authored
Commits on May 22, 2008
  1. Make sure manager drb object accepts connections from any host (not j…

    phlapjack authored
    …ust localhost)
  2. Make sure to save the worker_queue_file every minute in Skynet Manage…

    phlapjack authored
    …r. Move start_worker_queue_thread to new method
  3. Remove worker queue adapters and skynet_worker_queue

    phlapjack authored
    Last commit missed the following message:
    Skynet Workers now communicate with skynet manager by grabbing the manager over DRB and passing hashes directly to an internal
    Queue object held by the manager.
    The manager has a thread that watches the queue and updates the @worker_queue when new items are in the queue.
    The worker/manager no longer use tuplespace to communicate
    Removed unecessary logging in a few places.
  4. give WorkerQueueAdapter::TupleSpace its own get_tuplespace method whi…

    phlapjack authored
    …ch just finds the ts connected to the manager on that machine
Commits on May 20, 2008
  1. - Add Launching of TupleSpace server to skynet_launcher instead of ha…

    phlapjack authored
    …ving it in the individually installed launchers. Much cleaner.
    - Make sure skynet_launcher passes ALL config options when it starts the tuple_space_server.
    - Remove starting of tuplespace server from installed skynet launcer
    - Make sure we verify if we have a valid tuplespace even if we're not using a ringserver.
    - Allow you to pass use_ringserver as a command line flag to skynet_tuplespace_server
  2. Fix manifest

    phlapjack authored
  3. remove docs from svn

    phlapjack authored
  4. add docs to svn

    phlapjack authored
  5. include information about examples in the readme

    phlapjack authored
Commits on May 18, 2008
  1. Clean up manifest and skynet directory

    phlapjack authored
Commits on May 17, 2008
  1. Have Skynet::Manager handle what happens when the worker queue dies f…

    phlapjack authored
    …or some reason
  2. Clean up debugging

    phlapjack authored
    Try and handle double kills better
  3. The major change is that now every skynet manager forks off its own t…

    phlapjack authored
    …uplespace server as a worker queue, which it kills when it dies. Workers communicate directly with that queue instead of the main message queue.
    Also, skynet shuts down more quickly.
    - SkynetMessageQueue now optionally takes a hash of options instead of a proxy object.
    - Renamed Skynet::Server as Skynet::TuplespaceServer
    - Config changes
      - renamed TS_DRBURIS to TS_DRBUSI which is now a string instead of an array
      - added TS_WQUEUE_URI
    - Skynet::Worker has a short circuit exit if its not processing a task at the moment.  It notifies_worker_stop when it does.
    - Skynet::WorkerQueue now assumes it's using a separate queue than the MessageQueue.  It passes the TS_WQUEUE_URI to the Worker Queue proxy class.
     - skynet runner that gets installed from skynet_install now passes some piddir and logdir flags to skynet_tuplespace_server start.
    - Skynet::Manager
      - the manager now forks off a tuplespace worker queue for its workers to communicate with it.   It kills that process once the manager is killed.
      - Fix bug in mark_worker_as_sopped
      - Signal workers now takes :idle as a tasktype where it sends the signal to workers that aren't reported as busy.  It also takes an array of tasktypes.
      - shutdown() passe an array of tasktypes including :idle to send_signal
Commits on May 9, 2008
  1. Added some debugging to Skynet::Manager. Added some new debugging met…

    phlapjack authored
    …hods to SkynetDebugger
Commits on May 4, 2008
  1. - Gracefully handles trying to start skynet more than once

    phlapjack authored
    - Dramatically improve shutdown time.   If a worker isn't doing anything it just dies right away regardless of where it is in the
      polling process
    - Skynet runner does not need 'boot.rb' in rails
Commits on May 3, 2008
  1. - Fix serious bug in Skynet::Partitioners::RecombineAndSplit where i…

    phlapjack authored
    …t wouldn't handle empty results well.
     - Fixed Skynet::Partitioners::ArrayDataSplitByFirstEntry to handle strings as keys better
  2. Update history

    phlapjack authored
  3. Update History

    phlapjack authored
  4. svn merge -r 1845:1860 svn+ssh://…

    phlapjack authored
    - Skynet manager now supports being daemonized by running ./script/skynet start and ./script/skynet stop!
    - Change mysql text fields to longtext in migration and schema files
    - Include some extras including our init.d script, our nagios monitoring script, rails controller and view for monitoring
    - Created a new skynet rails initializer which gets installed from skynet_install --rails
    - Modified the skynet_install skynet runner to take skynet initializer into account
    - Skynet::Manager - Work on the shutdown proceedure. Much more graceful now.
    - Changed some of the debug levels in Skynet::Worker and Skynet::Manager
    - Skynet::ActiveRecordExtensions Fixed a bug where it would fail if your table had fewer than 1000 rows.  It performs a count
     first now to make sure there are enough rows.
    - Fix a comment in Skynet::MapreduceHelper.  self.reduce should ahve been self.reduce_each
    - Fix some bugs in skynet_config where it would choke on Skynet::Config.logfile_location if the SKYNET_LOG_FILE was set to a filehandle like STDOUT.
    - Fix a skynet manager bug where it would get weird empty BigDecimals from the worker statuses
    - Added Skynet::Job.results_by_job_id to retrieve results from asyncronous jobs
    - Moved all worker stats into the manager class which now does stats no matter what worker_queue_adapter your using.  The stats are more detailed now as well.
    - Removed all stats code from worker queue classes
    - Added a stats_for_hosts method to Skynet::Manager which aggregates stats accross many managers.
    - Move worker methods out of the mysql message queue adapter into a new mysql worker queue adapter
    - Added a Skynet::Manager#stats method
    - Skynet::Manager now saves its stats to a file and reloads them when it starts.
    - Skynet::Manager.get returns the current local manager drb object
    - Workers now just add their statuses to the worker queue (without trying to take their old one off)
    - Managers just keep polling the queue, taking worker statuses off, and holding an internal hash of worker statuses.
    - The Manager doesn't pass around the worker_pids or worker_queue anymore, just updates when needed and uses the cached one.
    - Changed how the skynet pid file and log file are specified.  You can now specify the directory and file separately.
    - Lowered the WORKER_CHECK_DELAY
    - Made some API changes to Skynet::Config so you can call class or instance methods for all the CONFIG options.
    - Add new WORKER_QUEUE_ADAPTER config setting
    - Create Skynet::MessageQueue.start_or_connect
    - More work on Skynet::WorkerQueue class
Commits on Apr 11, 2008
  1. Rename skynet_schema.sql to skynet_mysql_schema.sql

    phlapjack authored
  2. Fix up the skynet script that installs with skynet_install. Make sure…

    phlapjack authored
    … it puts the log and pid file in the right place.
  3. update webiste

    phlapjack authored
  4. Add new SkynetDebugger method printlog which always prints to the log…

    phlapjack authored
    … regardless of the log level. Updated docs
Something went wrong with that request. Please try again.