public
Description: Gem for processing collections over a number of threads.
Homepage: http://peteonrails.lighthouseapp.com/projects/11058-threaded-collections/overview
Clone URL: git://github.com/peteonrails/threadedcollections.git
Peter Jackson (author)
Thu May 15 12:40:04 -0700 2008
commit  0b7162b92f430f0f85e0b2d3ed7f57dc67031b56
tree    7201ab34fb47589c2fbd92093fb2c3dbf5487805
parent  cc4b1ee4462a97a55c96fb27f91eb11a3808599c parent  f5e7b2ebf16ef4dd415ad071731ab0f37b87d005
name age message
file LICENSE Loading commit data...
file README
file Rakefile
file foo
directory lib/
directory pkg/
file threadedcollections.gemspec
README
Threaded Collections is a package for iterating through collections 
over multiple threads. With large collections, sometimes it can be 
more efficient to process a collection in parallel, provided that the 
collected items don't have a interdependencies, or need to be processed
in a specific order. 

Usage: 

require "threaded_collections"

threadcount = 2
arr = [1, 2, 3, 4, 5, 6, 7, 8, 9, 0]
tps = ThreadedCollectionProcessor.new(arr)
tps.process(2) { |thread_id, item| puts "Thread #{thread_id} processed item: #{item}" }

I abstracted this pattern from a web services client that posted items from a collection, but
each request took a second to process. The remote service had plenty of threads available, so
I parallelized the task with this pattern. 

THIS IS AN ALPHA RELEASE. 

I have no plans to break the interface but I do plan to make two major enhancements: 

1. Make it possible to mix this functionality in to the Ruby iterators, so you don't have create the 
ThreadedCollectionProcessor.
2. Make it work with fibers and processes in addition to threads. 

Note: Don't implement the Observer pattern inside the block! 

Submit bugs, enhancement requests, and other issues through Lighthouse: 

http://peteonrails.lighthouseapp.com/projects/11058-threaded-collections/overview