Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Version 0.1.0 -- first real release

  • Loading branch information...
commit 6f66c10a3f345caf9cf10ec88a69c3e29c67bd39 1 parent 18c7557
@jashkenas jashkenas authored
View
1  .gitignore
@@ -1,4 +1,5 @@
.autotest
+.yardoc
log
raws
tmp
View
3  .yardopts
@@ -0,0 +1,3 @@
+--no-highlight
+--protected
+--private
View
7 README
@@ -1,4 +1,4 @@
-
+=
_ _
( ` )_
( ) `)
@@ -39,9 +39,10 @@
* Web scraping.
- ~ Wiki ~
+ ~ Documentation ~
- http://wiki.github.com/documentcloud/cloud-crowd
+ Wiki: http://wiki.github.com/documentcloud/cloud-crowd
+ Rdoc: http://rdoc.info/projects/documentcloud/cloud-crowd
~ Getting started ~
View
2  Rakefile
@@ -13,7 +13,7 @@ namespace :gem do
desc 'Build and install cloud-crowd gem'
task :install do
sh "gem build cloud-crowd.gemspec"
- sh "sudo gem install #{Dir['*.gem'].join(' ')} --no-ri"
+ sh "sudo gem install #{Dir['*.gem'].join(' ')} --no-ri --no-rdoc"
end
desc 'Uninstall the cloud-crowd gem'
View
4 TODO
@@ -1,8 +1,6 @@
TODO:
- * Bump to version 0.1 and release on RubyForge.
* Use the email field to send mail when the Job's gone bad.
* Job creation UI. Pick an action, add inputs, set options, go.
- * Unit tests for lib.
* Look into Grand Central Dispatch for ideas about how to implement
Job ordering/piping.
@@ -23,3 +21,5 @@ DONE:
* Replace work_units_remaining with a percent_complete
* Monitor UI -- see queue of jobs / work units.
* Basic integration tests.
+ * Unit tests for lib.
+ * Bump to version 0.1 and release on RubyForge.
View
2  cloud-crowd.gemspec
@@ -1,6 +1,6 @@
Gem::Specification.new do |s|
s.name = 'cloud-crowd'
- s.version = '0.0.6' # Keep version in sync with cloud-cloud.rb
+ s.version = '0.1.0' # Keep version in sync with cloud-cloud.rb
s.date = '2009-09-01'
s.homepage = "http://wiki.github.com/documentcloud/cloud-crowd"
View
2  lib/cloud-crowd.rb
@@ -45,7 +45,7 @@ module CloudCrowd
ROOT = File.expand_path(File.dirname(__FILE__) + '/..')
# Keep the version in sync with the gemspec.
- VERSION = '0.0.6'
+ VERSION = '0.1.0'
# A Job is processing if its WorkUnits in the queue to be handled by workers.
PROCESSING = 1
View
4 lib/cloud_crowd/asset_store.rb
@@ -20,14 +20,14 @@ class AssetStore
case CloudCrowd.config[:storage]
when 's3' then include S3Store
when 'filesystem' then include FilesystemStore
- else raise StorageNotFound, "#{CloudCrowd.config[:storage]} is not a valid storage back end"
+ else raise Error::StorageNotFound, "#{CloudCrowd.config[:storage]} is not a valid storage back end"
end
# Creating the AssetStore ensures that its scratch directory exists.
def initialize
@use_auth = CloudCrowd.config[:use_s3_authentication]
FileUtils.mkdir_p temp_storage_path unless File.exists? temp_storage_path
- raise StorageNotWritable, "#{temp_storage_path} is not writable" unless File.writable?(temp_storage_path)
+ raise Error::StorageNotWritable, "#{temp_storage_path} is not writable" unless File.writable?(temp_storage_path)
end
# Get the path to CloudCrowd's temporary local storage. All actions run
View
1  lib/cloud_crowd/command_line.rb
@@ -17,6 +17,7 @@ class CommandLine
CloudCrowd is a MapReduce-inspired Parallel Processing System for Ruby.
Wiki: http://wiki.github.com/documentcloud/cloud-crowd
+Rdoc: http://rdoc.info/projects/documentcloud/cloud-crowd
Usage: crowd COMMAND OPTIONS
View
40 lib/cloud_crowd/exceptions.rb
@@ -1,26 +1,28 @@
module CloudCrowd
# Base Error class which all custom CloudCrowd exceptions inherit from.
- class Error < RuntimeError #:nodoc:
- end
-
- # ActionNotFound is raised when a job is created for an action that doesn't
- # exist.
- class ActionNotFound < Error #:nodoc:
- end
-
- # StorageNotFound is raised when config.yml specifies a storage back end that
- # doesn't exist.
- class StorageNotFound < Error #:nodoc:
- end
-
- # If the AssetStore can't write to its scratch directory.
- class StorageNotWritable < Error #:nodoc:
- end
+ # Rescuing CloudCrowd::Error (or RuntimeError) will get all custom exceptions.
+ class Error < RuntimeError
+
+ # ActionNotFound is raised when a job is created for an action that doesn't
+ # exist.
+ class ActionNotFound < Error
+ end
- # StatusUnspecified is raised when a WorkUnit returns without a valid
- # status code.
- class StatusUnspecified < Error #:nodoc:
+ # StorageNotFound is raised when config.yml specifies a storage back end that
+ # doesn't exist.
+ class StorageNotFound < Error
+ end
+
+ # If the AssetStore can't write to its scratch directory.
+ class StorageNotWritable < Error
+ end
+
+ # StatusUnspecified is raised when a WorkUnit returns without a valid
+ # status code.
+ class StatusUnspecified < Error
+ end
+
end
end
View
2  lib/cloud_crowd/helpers.rb
@@ -2,7 +2,7 @@
require 'cloud_crowd/helpers/resources'
module CloudCrowd
- module Helpers #:nodoc:
+ module Helpers
include Authorization, Resources #, Rack::Utils
end
end
View
2  lib/cloud_crowd/inflector.rb
@@ -1,7 +1,7 @@
module CloudCrowd
# Pilfered in parts from the ActiveSupport::Inflector.
- module Inflector #:nodoc:
+ module Inflector
def self.camelize(word)
word.to_s.gsub(/\/(.?)/) { "::#{$1.upcase}" }.gsub(/(?:^|_)(.)/) { $1.upcase }
View
2  lib/cloud_crowd/models/job.rb
@@ -94,7 +94,7 @@ def mergeable?
def action_class
klass = CloudCrowd.actions[self.action]
return klass if klass
- raise ActionNotFound, "no action named: '#{self.action}' could be found"
+ raise Error::ActionNotFound, "no action named: '#{self.action}' could be found"
end
# How complete is this Job?
View
2  lib/cloud_crowd/worker.rb
@@ -120,7 +120,7 @@ def run_work_unit
when PROCESSING then @action.process
when SPLITTING then @action.split
when MERGING then @action.merge
- else raise StatusUnspecified, "work units must specify their status"
+ else raise Error::StatusUnspecified, "work units must specify their status"
end
end
complete_work_unit({'output' => result}.to_json)
View
4 wiki/getting_started.textile
@@ -18,13 +18,13 @@ All of the CloudCrowd commands can be used through the @crowd@ executable, which
crowd install path/to/config
</pre>
-Inside the configuration folder, you'll find @config.yml@, @config.ru@, @database.yml@, and the @actions@ folder. More information about all of these can be found in [[The Configuration Folder]] as well as by reading the files. At this point, you'll want to edit @config.yml@ and @database.yml@ to add your AWS credentials, turn authentication on or off, and set up the backing database. CloudCrowd uses ActiveRecord on the central server, so any ActiveRecord compatible database will do nicely. Once your database has been created on your central server (which in this case is your laptop, if you're just giving it a spin) and configured in @database.yml@, you can load in the schema:
+Move into the configuration folder, and you'll find @config.yml@, @config.ru@, @database.yml@, and the @actions@ folder. More information about all of these can be found in [[The Configuration Folder]] as well as by reading the files. At this point, you'll want to edit @config.yml@ and @database.yml@ to add your AWS credentials, turn authentication on or off, and set up the backing database. CloudCrowd uses ActiveRecord on the central server, so any ActiveRecord compatible database will do nicely. Once your database has been created on your central server (which in this case is your laptop, if you're just giving it a spin) and configured in @database.yml@, you can load in the schema:
<pre>
crowd load_schema
</pre>
-With the database ready, let's start a single instance of the central server. In production, you can use the @config.ru@ rackup file to launch the desired number of server instances, and hide them behind a load balancer as you would any other Rack application.
+With the database ready, let's start a single instance of the central server. In production, you can use the @config.ru@ rackup file to launch the desired number of server instances, and hide them behind a load balancer as you would any other Rack application. But for now we'll just use @crowd@ to launch one.
<pre>
crowd server
View
2  wiki/home.textile
@@ -23,6 +23,8 @@ sudo gem install cloud-crowd
[[Wish List]]
+"RDoc":http://rdoc.info/projects/documentcloud/cloud-crowd
+
"Google Group":http://groups.google.com/group/cloud-crowd
</div>
Please sign in to comment.
Something went wrong with that request. Please try again.