Skip to content

Commit

Permalink
Sort commands by the order they are specified
Browse files Browse the repository at this point in the history
  • Loading branch information
iain committed Jul 22, 2012
1 parent b94326c commit ba67369
Show file tree
Hide file tree
Showing 7 changed files with 62 additions and 37 deletions.
3 changes: 3 additions & 0 deletions .travis.yml
Original file line number Original file line Diff line number Diff line change
Expand Up @@ -3,3 +3,6 @@ rvm:
- 1.9.3 - 1.9.3
- 1.9.2 - 1.9.2
- rbx-19mode - rbx-19mode
- rbx-18mode
- ree
- 1.8.7
8 changes: 5 additions & 3 deletions README.md
Original file line number Original file line Diff line number Diff line change
Expand Up @@ -84,6 +84,9 @@ run "some ruby code" do
end end
``` ```


Keep in mind that MRI has trouble running ruby and rake tasks in parallel due
to the GIL.

### Running scripts in parallel ### Running scripts in parallel


You can really win some time by running certain commands in parallel. Doing You can really win some time by running certain commands in parallel. Doing
Expand All @@ -105,7 +108,7 @@ that run in parallel have finished.


There are only a few caveats to this. The scripts must be able to run There are only a few caveats to this. The scripts must be able to run
simultaniously. If they both access the same global data, like a database or simultaniously. If they both access the same global data, like a database or
files on your hard disk, they will probably fail. Also, any output they produce files on your hard disk, they will probably fail. Any output they produce
will appear at the same time, possibly making it unreadable. will appear at the same time, possibly making it unreadable.


You can specify multiple parallel blocks. You can specify multiple parallel blocks.
Expand Down Expand Up @@ -400,11 +403,10 @@ contact me if you have any questions or ideas for improvements. Mention me on


### Known issues ### Known issues


* Works on MRI 1.9 and Rubinius in 1.9 mode. * Works on MRI and Rubinius.
* JRuby might have problems running shell commands. * JRuby might have problems running shell commands.
* JRuby doesn't always allow you to compile C extensions, so you cannot install * JRuby doesn't always allow you to compile C extensions, so you cannot install
Faye. Use a different Ruby implementation or use the Node.js version. Faye. Use a different Ruby implementation or use the Node.js version.
* Ruby 1.8 won't run commands in the right order, because of unsorted hashes.
* To get color in RSpec, use the `--tty` switch, or RSpec will not believe the * To get color in RSpec, use the `--tty` switch, or RSpec will not believe the
shell supports color. shell supports color.
* Use the `--color` switch for Cucumber. * Use the `--color` switch for Cucumber.
Expand Down
6 changes: 1 addition & 5 deletions lib/scripted/command.rb
Original file line number Original file line Diff line number Diff line change
Expand Up @@ -62,12 +62,8 @@ def forced?
!!@forced !!@forced
end end


def parallel?
!!@options[:parallel]
end

def parallel_id def parallel_id
@parallel_id ||= (@options[:parallel] || Object.new).object_id @options[:parallel_id]
end end


def run_in_parallel_with?(other) def run_in_parallel_with?(other)
Expand Down
24 changes: 21 additions & 3 deletions lib/scripted/group.rb
Original file line number Original file line Diff line number Diff line change
Expand Up @@ -7,25 +7,43 @@ class Group


def initialize(name) def initialize(name)
@name = name @name = name
@parallel = false
@parallel_id = (object_id ** 2)
end end


def define(&block) def define(&block)
instance_eval &block if block instance_eval &block if block
end end


def run(name, &block) def run(name, &block)
commands << Command.new(name, :parallel => @parallel, &block) next_parallel_id unless in_parallel?
commands << Command.new(name, :parallel_id => parallel_id, &block)
end end


def parallel(&block) def parallel(&block)
@parallel = Object.new @parallel = true
yield yield
@parallel = nil @parallel = false
next_parallel_id
end end


def commands def commands
@commands ||= [] @commands ||= []
end end


private

def parallel_id
@parallel_id
end

def next_parallel_id
@parallel_id += 1
end

def in_parallel?
@parallel
end

end end
end end
11 changes: 10 additions & 1 deletion lib/scripted/running/run_commands.rb
Original file line number Original file line Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ def initialize(logger)


def run(commands) def run(commands)
logged commands do logged commands do
commands.group_by(&:parallel_id).each do |parallel_id, parallel_commands| per_parallel_id commands do |parallel_commands|
threads = [] threads = []
parallel_commands.each do |command| parallel_commands.each do |command|
if should_execute?(command) if should_execute?(command)
Expand Down Expand Up @@ -61,6 +61,15 @@ def executed?


private private


# these are re-sorted, because in Ruby 1.8, hashes aren't sorted
def per_parallel_id(commands)
commands.group_by(&:parallel_id).values.sort_by { |commands|
commands.first.parallel_id
}.each { |commands|
yield commands
}
end

def logged(commands) def logged(commands)
@started_at = Time.now @started_at = Time.now
@running = true @running = true
Expand Down
4 changes: 2 additions & 2 deletions spec/scripted/command_spec.rb
Original file line number Original file line Diff line number Diff line change
Expand Up @@ -65,8 +65,8 @@
end end


it "can be parallel" do it "can be parallel" do
command = Scripted::Command.new("true", :parallel => Object.new) command = Scripted::Command.new("true", :parallel_id => 10)
expect(command).to be_parallel expect(command.parallel_id).to eq 10
end end


end end
43 changes: 20 additions & 23 deletions spec/scripted/group_spec.rb
Original file line number Original file line Diff line number Diff line change
Expand Up @@ -15,38 +15,35 @@
expect(group.commands.first.name).to eq "echo 1" expect(group.commands.first.name).to eq "echo 1"
end end


it "can define commands to be executed in parallel" do
parallel_command = also_parallel = not_parallel = nil

group.define do
parallel do
run("one") { parallel_command = self }
run("two") { also_parallel = self }
end
run("three") { not_parallel = self }
end

expect(parallel_command).to be_parallel
expect(also_parallel).to be_parallel
expect(not_parallel).not_to be_parallel
end

it "can define multiple parallel blocks" do it "can define multiple parallel blocks" do
command_one = command_two = command_three = self command1 = command2 = command3 = command4 = nil


group.define do group.define do
parallel do parallel do
run("one") { command_one = self } run("one") { command1 = self }
run("two") { command_two = self } run("two") { command2 = self }
end end
parallel do parallel do
run("three") { command_three = self } run("three") { command3 = self }
end end
run("four") { command4 = self }
end end


expect(command_one).to be_run_in_parallel_with(command_two) expect(command1).to be_run_in_parallel_with(command2)
expect(command_one).not_to be_run_in_parallel_with(command_three) expect(command2).to be_run_in_parallel_with(command1)
expect(command_two).not_to be_run_in_parallel_with(command_three)
expect(command3).not_to be_run_in_parallel_with(command1)
expect(command3).not_to be_run_in_parallel_with(command2)

expect(command1).not_to be_run_in_parallel_with(command3)
expect(command2).not_to be_run_in_parallel_with(command3)

expect(command4).not_to be_run_in_parallel_with(command1)
expect(command4).not_to be_run_in_parallel_with(command2)
expect(command4).not_to be_run_in_parallel_with(command3)
expect(command1).not_to be_run_in_parallel_with(command4)
expect(command2).not_to be_run_in_parallel_with(command4)
expect(command3).not_to be_run_in_parallel_with(command4)
end end


end end

0 comments on commit ba67369

Please sign in to comment.