Skip to content

Commit

Permalink
Tweaks to MongoQueue to optimize performance: 1) Get rid of the count…
Browse files Browse the repository at this point in the history
…() call, which runs very slowly when the queue is being populated/drained at high rates. 2) Add place holder locked, locked_at, and batch fields for documents. 3) Add indexing suggestions in README
  • Loading branch information
lunaru committed Sep 8, 2010
1 parent 43ff52f commit ec9f6da
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 3 deletions.
20 changes: 20 additions & 0 deletions README.textile
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,10 @@ You can also take advantage of @MongoJob@'s built in @later()@ method which auto

<pre>QueueTracer::later(500)->trace();</pre>

You can also batch calls which have the same parameter so that they'll only be run once by workers. The run time will be decided by the first job inserted and the subsequent jobs will not have their run times obeyed.

<pre>QueueTracer::batchLater(500)->trace();</pre>

h3. Running the jobs

Jobs are run via @MongoQueue::run()@. Before running job, you'll need to set two extra MongoQueue initializers: @environment@ and, optionally, @context@. MongoQueue currently does not include a daemon runner, but here is an example runner:
Expand All @@ -70,6 +74,22 @@ Any keys set in @$context@ are available as static variables inside the Job clas

h3. Administration

h4. Indexing/Performance

You'll want to add indices to MongoQueue to ensure high performance. You'll need at least the following index for basic operation:

<pre>
db.mongo_queue.ensureIndex({locked: 1, when: 1});
</pre>

If you plan to use the batching feature you want an index on the type of jobs already in the queue:

<pre>
db.mongo_queue.ensureIndex({object_class: 1, object_method: 1, parameters: 1, locked: 1});
</pre>

h4. Monitoring

All jobs are stored as Mongo documents in the mongo_queue collection. @MongoQueue@ does not come with any built in administrative scripts, but you can view the queue via your mongo console. @locked_at@ will tell you whether or not a job is currently being processed.


13 changes: 10 additions & 3 deletions lib/MongoQueue.php
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,14 @@ public static function push($className, $methodName, $parameters, $when, $batch
if (!$batch)
{
$collection = self::getCollection();
$collection->save(array('object_class' => $className, 'object_method' => $methodName, 'parameters' => $parameters, 'when' => $when));
$collection->save(array(
'object_class' => $className,
'object_method' => $methodName,
'parameters' => $parameters,
'when' => $when,
'locked' => null,
'locked_at' => null,
'batch' => 1));
}
else
{
Expand Down Expand Up @@ -44,7 +51,7 @@ public static function push($className, $methodName, $parameters, $when, $batch
}
}

public static function count($class_name = null)
public static function hasRunnable($class_name = null)
{
$collection = self::getCollection();

Expand All @@ -53,7 +60,7 @@ public static function count($class_name = null)
if ($class_name)
$query['object_class'] = $class_name;

return $collection->count($query);
return ($collection->findOne($query) != null);
}

public static function run($class_name = null)
Expand Down

0 comments on commit ec9f6da

Please sign in to comment.