Skip to content
Permalink
Browse files

Merge pull request #791 from ruby-concurrency/ruby-association

Ruby association project
  • Loading branch information...
pitr-ch committed Mar 11, 2019
2 parents 2307958 + 3ff1cb0 commit f72141ee20c8f733f2d268b4b259e3600d186a12
Showing with 37,307 additions and 12,809 deletions.
  1. +1 −0 .rspec
  2. +13 −0 CHANGELOG.md
  3. +8 −6 Gemfile
  4. +31 −0 README.md
  5. +20 −46 Rakefile
  6. +1 −1 concurrent-ruby-edge.gemspec
  7. +158 −0 docs-source/cancellation.in.md
  8. +6 −0 docs-source/cancellation.init.rb
  9. +192 −0 docs-source/cancellation.out.md
  10. +225 −0 docs-source/channel.in.md
  11. +6 −0 docs-source/channel.init.rb
  12. +313 −0 docs-source/channel.out.md
  13. +278 −0 docs-source/erlang_actor.in.md
  14. +9 −0 docs-source/erlang_actor.init.rb
  15. +300 −0 docs-source/erlang_actor.out.md
  16. +273 −0 docs-source/medium-example.in.rb
  17. +13 −0 docs-source/medium-example.init.rb
  18. +707 −0 docs-source/medium-example.out.rb
  19. +51 −138 docs-source/promises.in.md
  20. +110 −202 docs-source/promises.out.md
  21. +208 −0 docs-source/ruby-association-final-report.md
  22. +99 −0 docs-source/ruby-association-intermediate-report.md
  23. +141 −0 docs-source/throttle.in.md
  24. +6 −0 docs-source/throttle.init.rb
  25. +145 −0 docs-source/throttle.out.md
  26. +31 −380 docs/master/Concurrent.html
  27. +2 −2 docs/master/Concurrent/Actor.html
  28. +560 −560 docs/master/Concurrent/Actor/AbstractContext.html
  29. +12 −12 docs/master/Concurrent/Actor/ActorTerminated.html
  30. +345 −345 docs/master/Concurrent/Actor/Behaviour/Abstract.html
  31. +185 −185 docs/master/Concurrent/Actor/Core.html
  32. +108 −108 docs/master/Concurrent/Actor/Envelope.html
  33. +188 −188 docs/master/Concurrent/Actor/InternalDelegations.html
  34. +307 −301 docs/master/Concurrent/Actor/Reference.html
  35. +12 −12 docs/master/Concurrent/Actor/UnknownMessage.html
  36. +318 −318 docs/master/Concurrent/Agent.html
  37. +348 −348 docs/master/Concurrent/Atom.html
  38. +8 −4 docs/master/Concurrent/AtomicFixnum.html
  39. +578 −112 docs/master/Concurrent/Cancellation.html
  40. +0 −814 docs/master/Concurrent/Cancellation/Token.html
  41. +507 −507 docs/master/Concurrent/Delay.html
  42. +2 −2 docs/master/Concurrent/Edge/LockFreeLinkedSet.html
  43. +1 −1 docs/master/Concurrent/Edge/LockFreeLinkedSet/Node.html
  44. +1 −1 docs/master/Concurrent/Edge/LockFreeLinkedSet/Tail.html
  45. +1,010 −0 docs/master/Concurrent/ErlangActor.html
  46. +638 −0 docs/master/Concurrent/ErlangActor/Down.html
  47. +2,094 −0 docs/master/Concurrent/ErlangActor/Environment.html
  48. +196 −0 docs/master/Concurrent/ErlangActor/EnvironmentConstants.html
  49. +299 −0 docs/master/Concurrent/ErlangActor/EnvironmentConstants/AbstractLogicOperationMatcher.html
  50. +250 −0 docs/master/Concurrent/ErlangActor/EnvironmentConstants/And.html
  51. +251 −0 docs/master/Concurrent/ErlangActor/EnvironmentConstants/Or.html
  52. +33 −82 docs/master/Concurrent/{Synchronization/TruffleRubyAttrVolatile.html → ErlangActor/Error.html}
  53. +298 −0 docs/master/Concurrent/ErlangActor/FunctionShortcuts.html
  54. +618 −0 docs/master/Concurrent/ErlangActor/Functions.html
  55. +492 −0 docs/master/Concurrent/ErlangActor/NoActor.html
  56. +33 −83 docs/master/Concurrent/{Synchronization/MriAttrVolatile.html → ErlangActor/NoReply.html}
  57. +903 −0 docs/master/Concurrent/ErlangActor/Pid.html
  58. +32 −16 docs/master/Concurrent/{Synchronization/JRubyAttrVolatile.html → ErlangActor/Reference.html}
  59. +557 −0 docs/master/Concurrent/ErlangActor/Terminated.html
  60. +416 −416 docs/master/Concurrent/IVar.html
  61. +1 −1 docs/master/Concurrent/LazyRegister.html
  62. +432 −2 docs/master/Concurrent/LockFreeQueue/Node.html
  63. +4 −4 docs/master/Concurrent/LockFreeStack.html
  64. +16 −16 docs/master/Concurrent/Map.html
  65. +229 −274 docs/master/Concurrent/ProcessingActor.html
  66. +215 −135 docs/master/Concurrent/Promises.html
  67. +110 −226 docs/master/Concurrent/Promises/AbstractEventFuture.html
  68. +0 −327 docs/master/Concurrent/Promises/AbstractEventFuture/ThrottleIntegration.html
  69. +3,042 −143 docs/master/Concurrent/Promises/Channel.html
  70. +40 −42 docs/master/Concurrent/Promises/Event.html
  71. +3,885 −397 docs/master/Concurrent/Promises/FactoryMethods.html
  72. +3 −3 docs/master/Concurrent/Promises/FactoryMethods/Configuration.html
  73. +0 −261 docs/master/Concurrent/Promises/FactoryMethods/NewChannelIntegration.html
  74. +775 −742 docs/master/Concurrent/Promises/Future.html
  75. +1 −1 docs/master/Concurrent/Promises/Future/ActorIntegration.html
  76. +9 −9 docs/master/Concurrent/Promises/Future/NewChannelIntegration.html
  77. +0 −325 docs/master/Concurrent/Promises/Future/ThrottleIntegration.html
  78. +459 −2 docs/master/Concurrent/Promises/Resolvable.html
  79. +306 −13 docs/master/Concurrent/Promises/ResolvableEvent.html
  80. +1,133 −66 docs/master/Concurrent/Promises/ResolvableFuture.html
  81. +3 −3 docs/master/Concurrent/ReadWriteLock.html
  82. +4 −4 docs/master/Concurrent/ReentrantReadWriteLock.html
  83. +6 −8 docs/master/Concurrent/Synchronization.html
  84. +0 −648 docs/master/Concurrent/Synchronization/Condition.html
  85. +4 −4 docs/master/Concurrent/Synchronization/JRubyAttrVolatile/ClassMethods.html
  86. +0 −369 docs/master/Concurrent/Synchronization/Lock.html
  87. +4 −4 docs/master/Concurrent/Synchronization/MriAttrVolatile/ClassMethods.html
  88. +252 −183 docs/master/Concurrent/Synchronization/Object.html
  89. +0 −197 docs/master/Concurrent/Synchronization/RbxAttrVolatile.html
  90. +4 −4 docs/master/Concurrent/Synchronization/RbxAttrVolatile/ClassMethods.html
  91. +4 −4 docs/master/Concurrent/Synchronization/TruffleRubyAttrVolatile/ClassMethods.html
  92. +2 −2 docs/master/Concurrent/TVar.html
  93. +3,793 −481 docs/master/Concurrent/Throttle.html
  94. +0 −381 docs/master/Concurrent/Throttle/PromisesIntegration.html
  95. +8 −12 docs/master/Concurrent/TimerSet.html
  96. +176 −176 docs/master/Concurrent/TimerTask.html
  97. +147 −74 docs/master/_index.html
  98. +1 −1 docs/master/class_list.html
  99. +17 −0 docs/master/file.CHANGELOG.html
  100. +36 −0 docs/master/file.README.html
  101. +782 −0 docs/master/file.medium-example.out.html
  102. +125 −213 docs/master/file.promises.out.html
  103. +7 −2 docs/master/file_list.html
  104. +36 −0 docs/master/index.html
  105. +1,599 −1,071 docs/master/method_list.html
  106. +4 −0 lib-edge/concurrent-edge.rb
  107. +3 −0 lib-edge/concurrent/actor/reference.rb
  108. +78 −112 lib-edge/concurrent/edge/cancellation.rb
  109. +450 −0 lib-edge/concurrent/edge/channel.rb
  110. +1,545 −0 lib-edge/concurrent/edge/erlang_actor.rb
  111. +83 −64 lib-edge/concurrent/edge/processing_actor.rb
  112. +80 −110 lib-edge/concurrent/edge/promises.rb
  113. +167 −141 lib-edge/concurrent/edge/throttle.rb
  114. +3 −0 lib-edge/concurrent/edge/version.rb
  115. +2 −2 lib/concurrent/atomic/atomic_fixnum.rb
  116. +1 −1 lib/concurrent/collection/lock_free_stack.rb
  117. +13 −15 lib/concurrent/executor/timer_set.rb
  118. +348 −117 lib/concurrent/promises.rb
  119. +1 −0 lib/concurrent/synchronization/abstract_struct.rb
  120. +2 −0 lib/concurrent/synchronization/condition.rb
  121. +1 −0 lib/concurrent/synchronization/jruby_object.rb
  122. +2 −0 lib/concurrent/synchronization/lock.rb
  123. +1 −0 lib/concurrent/synchronization/mri_object.rb
  124. +46 −20 lib/concurrent/synchronization/object.rb
  125. +1 −0 lib/concurrent/synchronization/rbx_object.rb
  126. +1 −0 lib/concurrent/synchronization/truffleruby_object.rb
  127. +1 −2 lib/concurrent/version.rb
  128. +1 −4 spec/concurrent/agent_spec.rb
  129. +91 −0 spec/concurrent/cancellation_spec.rb
  130. +1 −1 spec/concurrent/channel/buffer/base_spec.rb
  131. +1 −1 spec/concurrent/channel/buffer/buffered_spec.rb
  132. +1 −1 spec/concurrent/channel/buffer/dropping_spec.rb
  133. +1 −1 spec/concurrent/channel/buffer/sliding_spec.rb
  134. +1 −1 spec/concurrent/channel/buffer/ticker_spec.rb
  135. +1 −1 spec/concurrent/channel/buffer/timer_spec.rb
  136. +1 −1 spec/concurrent/channel/buffer/unbuffered_spec.rb
  137. +3 −4 spec/concurrent/channel/tick_spec.rb
  138. +387 −0 spec/concurrent/edge/channel_spec.rb
  139. +1,027 −0 spec/concurrent/edge/erlang_actor_spec.rb
  140. +6 −6 spec/concurrent/executor/safe_task_executor_spec.rb
  141. +1 −1 spec/concurrent/maybe_spec.rb
  142. +4 −4 spec/concurrent/mvar_spec.rb
  143. +204 −169 spec/concurrent/promises_spec.rb
  144. +0 −1 spec/concurrent/scheduled_task_spec.rb
  145. +94 −0 spec/concurrent/throttle_spec.rb
  146. +1 −1 spec/concurrent/timer_task_spec.rb
  147. +0 −1 spec/spec_helper.rb
  148. +3 −3 spec/support/example_group_extensions.rb
  149. +5 −4 support/yard_full_types.rb
  150. +2 −0 yard-template/default/layout/html/objects.erb
  151. +21 −0 yard-template/default/module/setup.rb
1 .rspec
@@ -1,3 +1,4 @@
-I lib-edge
--require spec_helper
--color
--warnings
@@ -1,5 +1,18 @@
## Current

## Release v1.1.5, edge v0.5.0 (10 mar 2019)

concurrent-ruby:

* fix potential leak of context on JRuby and Java 7

concurrent-ruby-edge:

* Add finalized Concurrent::Cancellation
* Add finalized Concurrent::Throttle
* Add finalized Concurrent::Promises::Channel
* Add new Concurrent::ErlangActor

## Release v1.1.4 (14 Dec 2018)

* (#780) Remove java_alias of 'submit' method of Runnable to let executor service work on java 11
14 Gemfile
@@ -1,6 +1,7 @@
source 'https://rubygems.org'

require File.join(File.dirname(__FILE__ ), 'lib/concurrent/version')
require File.join(File.dirname(__FILE__), 'lib/concurrent/version')
require File.join(File.dirname(__FILE__ ), 'lib-edge/concurrent/edge/version')

no_path = ENV['NO_PATH']
options = no_path ? {} : { path: '.' }
@@ -11,26 +12,27 @@ gem 'concurrent-ruby-ext', Concurrent::VERSION, options.merge(platform: :mri)

group :development do
gem 'rake', '~> 12.0'
gem 'rake-compiler', '~> 1.0'
gem 'rake-compiler', '~> 1.0', '>= 1.0.7'
gem 'rake-compiler-dock', '~> 0.7.0'
gem 'pry', '~> 0.11', platforms: :mri
end

group :documentation, optional: true do
gem 'yard', '~> 0.9.0', :require => false
gem 'yard', '~> 0.9.0', require: false
gem 'redcarpet', '~> 3.0', platforms: :mri # understands github markdown
gem 'md-ruby-eval', '~> 0.3'
gem 'md-ruby-eval', '~> 0.6'
end

group :testing do
gem 'rspec', '~> 3.7'
gem 'timecop', '~> 0.7.4'
gem 'sigdump', require: false
end

# made opt-in since it will not install on jruby 1.7
group :coverage, optional: !ENV['COVERAGE'] do
gem 'simplecov', '~> 0.10.0', :require => false
gem 'coveralls', '~> 0.8.2', :require => false
gem 'simplecov', '~> 0.16.0', require: false
gem 'coveralls', '~> 0.8.2', require: false
end

group :benchmarks, optional: true do
@@ -224,6 +224,34 @@ be obeyed though. Features developed in `concurrent-ruby-edge` are expected to m
*Status: will be moved to core soon.*
* [LockFreeStack](http://ruby-concurrency.github.io/concurrent-ruby/master/Concurrent/LockFreeStack.html)
*Status: missing documentation and tests.*
* [Promises::Channel](http://ruby-concurrency.github.io/concurrent-ruby/master/Concurrent/Promises/Channel.html)
A first in first out channel that accepts messages with push family of methods and returns
messages with pop family of methods.
Pop and push operations can be represented as futures, see `#pop_op` and `#push_op`.
The capacity of the channel can be limited to support back pressure, use capacity option in `#initialize`.
`#pop` method blocks ans `#pop_op` returns pending future if there is no message in the channel.
If the capacity is limited the `#push` method blocks and `#push_op` returns pending future.
* [Cancellation](http://ruby-concurrency.github.io/concurrent-ruby/master/Concurrent/Cancellation.html)
The Cancellation abstraction provides cooperative cancellation.

The standard methods `Thread#raise` of `Thread#kill` available in Ruby
are very dangerous (see linked the blog posts bellow).
Therefore concurrent-ruby provides an alternative.

* <https://jvns.ca/blog/2015/11/27/why-rubys-timeout-is-dangerous-and-thread-dot-raise-is-terrifying/>
* <http://www.mikeperham.com/2015/05/08/timeout-rubys-most-dangerous-api/>
* <http://blog.headius.com/2008/02/rubys-threadraise-threadkill-timeoutrb.html>

It provides an object which represents a task which can be executed,
the task has to get the reference to the object and periodically cooperatively check that it is not cancelled.
Good practices to make tasks cancellable:
* check cancellation every cycle of a loop which does significant work,
* do all blocking actions in a loop with a timeout then on timeout check cancellation
and if ok block again with the timeout
* [Throttle](http://ruby-concurrency.github.io/concurrent-ruby/master/Concurrent/Throttle.html)
A tool managing concurrency level of tasks.
* [ErlangActor](http://ruby-concurrency.github.io/concurrent-ruby/master/Concurrent/ErlangActor.html)
Actor implementation which matches Erlang actor behaviour.

## Supported Ruby versions

@@ -339,6 +367,9 @@ and to the past maintainers
* [Paweł Obrok](https://github.com/obrok)
* [Lucas Allan](https://github.com/lucasallan)

and to [Ruby Association](https://www.ruby.or.jp/en/) for sponsoring a project
["Enhancing Ruby’s concurrency tooling"](https://www.ruby.or.jp/en/news/20181106) in 2018.

## License and Copyright

*Concurrent Ruby* is free software released under the
@@ -16,43 +16,7 @@ edge_gemspec = Gem::Specification.load File.join(__dir__, 'concurrent-ruby-edge.

require 'rake/javaextensiontask'

class ConcurrentRubyJavaExtensionTask < Rake::JavaExtensionTask
def java_classpath_arg(*args)
jruby_cpath = nil

if RUBY_PLATFORM =~ /java/
begin
cpath = Java::java.lang.System.getProperty('java.class.path').split(File::PATH_SEPARATOR)
cpath += Java::java.lang.System.getProperty('sun.boot.class.path').split(File::PATH_SEPARATOR)
jruby_cpath = cpath.compact.join(File::PATH_SEPARATOR)
rescue => e
end

unless jruby_cpath
libdir = RbConfig::CONFIG['libdir']
if libdir.start_with? "classpath:"
raise 'Cannot build with jruby-complete'
end
jruby_cpath = File.join(libdir, "jruby.jar")
end
end

unless jruby_cpath
jruby_home = ENV['JRUBY_HOME']
if jruby_home
candidate = File.join(jruby_home, 'lib', 'jruby.jar')
jruby_cpath = candidate if File.exist? candidate
end
end

raise "jruby.jar path not found" unless jruby_cpath

jruby_cpath += File::PATH_SEPARATOR + args.join(File::PATH_SEPARATOR) unless args.empty?
jruby_cpath ? "-cp \"#{jruby_cpath}\"" : ""
end
end

ConcurrentRubyJavaExtensionTask.new('concurrent_ruby', core_gemspec) do |ext|
Rake::JavaExtensionTask.new('concurrent_ruby', core_gemspec) do |ext|
ext.ext_dir = 'ext/concurrent-ruby'
ext.lib_dir = 'lib/concurrent'
end
@@ -79,7 +43,8 @@ namespace :repackage do
sh 'bundle package'

# build only the jar file not the whole gem for java platform, the jar is part the concurrent-ruby-x.y.z.gem
RakeCompilerDock.sh 'bundle install --local && bundle exec rake lib/concurrent/concurrent_ruby.jar --trace', rubyvm: :jruby
Rake::Task['lib/concurrent/concurrent_ruby.jar'].invoke

# build all gem files
RakeCompilerDock.sh 'bundle install --local && bundle exec rake cross native package --trace'
end
@@ -101,15 +66,14 @@ begin

RSpec::Core::RakeTask.new(:spec)

options = %w[ --color
--backtrace
--seed 1
--format documentation
--tag ~notravis ]

namespace :spec do
desc '* Configured for ci'
RSpec::Core::RakeTask.new(:ci) do |t|
options = %w[ --color
--backtrace
--order defined
--format documentation
--tag ~notravis ]
t.rspec_opts = [*options].join(' ')
end

@@ -184,10 +148,19 @@ begin
end

define_yard_task = -> name do
output_dir = "docs/#{name}"

removal_name = "remove.#{name}"
task removal_name do
Dir.chdir __dir__ do
FileUtils.rm_rf output_dir
end
end

desc "* of #{name} into subdir #{name}"
YARD::Rake::YardocTask.new(name) do |yard|
yard.options.push(
'--output-dir', "docs/#{name}",
'--output-dir', output_dir,
'--main', 'tmp/README.md',
*common_yard_options)
yard.files = ['./lib/**/*.rb',
@@ -196,10 +169,11 @@ begin
'-',
'docs-source/thread_pools.md',
'docs-source/promises.out.md',
'docs-source/medium-example.out.rb',
'LICENSE.md',
'CHANGELOG.md']
end
Rake::Task[name].prerequisites.push 'yard:eval_md', 'yard:update_readme'
Rake::Task[name].prerequisites.push removal_name, 'yard:eval_md', 'yard:update_readme'
end

define_yard_task.call current_yard_version_name
@@ -1,4 +1,4 @@
require File.join(File.dirname(__FILE__ ), 'lib/concurrent/version')
require File.join(File.dirname(__FILE__ ), 'lib-edge/concurrent/edge/version')

Gem::Specification.new do |s|
git_files = `git ls-files`.split("\n")
@@ -0,0 +1,158 @@

## Examples

**Run async task until cancelled**

Create cancellation and then run work in a background thread until it is cancelled.

```ruby
cancellation, origin = Concurrent::Cancellation.new
# - origin is used for cancelling, resolve it to cancel
# - cancellation is passed down to tasks for cooperative cancellation
async_task = Concurrent::Promises.future(cancellation) do |cancellation|
# Do work repeatedly until it is cancelled
do_stuff until cancellation.canceled?
:stopped_gracefully
end
sleep 0.01
# Wait a bit then stop the thread by resolving the origin of the cancellation
origin.resolve
async_task.value!
```

Or let it raise an error.

```ruby
cancellation, origin = Concurrent::Cancellation.new
async_task = Concurrent::Promises.future(cancellation) do |cancellation|
# Do work repeatedly until it is cancelled
while true
cancellation.check!
do_stuff
end
end
sleep 0.01
# Wait a bit then stop the thread by resolving the origin of the cancellation
origin.resolve
async_task.result
```

**Run additional tasks on Cancellation**

Cancellation can also be used to log or plan re-execution.

```ruby
cancellation.origin.chain do
# This block is executed after the Cancellation is cancelled
# It can then log cancellation or e.g. plan new re-execution
end
```

**Run only for limited time – Timeout replacement**

Execute task for a given time then finish.
Instead of letting Cancellation crate its own origin, it can be passed in as argument.
The passed in origin is scheduled to be resolved in given time which then cancels the Cancellation.

```ruby
timeout = Concurrent::Cancellation.new Concurrent::Promises.schedule(0.02)
# or using shortcut helper method
timeout = Concurrent::Cancellation.timeout 0.02
count = Concurrent::AtomicFixnum.new
Concurrent.global_io_executor.post(timeout) do |timeout|
# do stuff until cancelled
count.increment until timeout.canceled?
end #
timeout.origin.wait
count.value # => 177576
```

**Parallel background processing with single cancellation**

Each task tries to count to 1000 but there is a randomly failing test. The
tasks share single cancellation, when one of them fails it cancels the others.
The failing tasks ends with an error, the other tasks are gracefully cancelled.

```ruby
cancellation, origin = Concurrent::Cancellation.new
tasks = 4.times.map do |i|
Concurrent::Promises.future(cancellation, origin, i) do |cancellation, origin, i|
count = 0
100.times do
break count = :cancelled if cancellation.canceled?
count += 1
sleep 0.001
if rand > 0.95
origin.resolve # cancel
raise 'random error'
end
count
end
end
end
Concurrent::Promises.zip(*tasks).result #
# => [false,
# [:cancelled, nil, :cancelled, :cancelled],
# [nil, #<RuntimeError: random error>, nil, nil]]
```

Without the randomly failing part it produces following.

```ruby
cancellation, origin = Concurrent::Cancellation.new
tasks = 4.times.map do |i|
Concurrent::Promises.future(cancellation, origin, i) do |cancellation, origin, i|
count = 0
100.times do
break count = :cancelled if cancellation.canceled?
count += 1
sleep 0.001
# if rand > 0.95
# origin.resolve
# raise 'random error'
# end
count
end
end
end
Concurrent::Promises.zip(*tasks).result
```

**Combine cancellations**

The combination created by joining two cancellations cancels when the first or the other does.

```ruby
cancellation_a, origin_a = Concurrent::Cancellation.new
cancellation_b, origin_b = Concurrent::Cancellation.new
combined_cancellation = cancellation_a.join(cancellation_b)
origin_a.resolve
cancellation_a.canceled?
cancellation_b.canceled?
combined_cancellation.canceled?
```

If a different rule for joining is needed, the source can be combined manually.
The manually created cancellation cancels when both the first and the other cancels.

```ruby
cancellation_a, origin_a = Concurrent::Cancellation.new
cancellation_b, origin_b = Concurrent::Cancellation.new
# cancels only when both a and b is cancelled
combined_cancellation = Concurrent::Cancellation.new origin_a & origin_b
origin_a.resolve
cancellation_a.canceled? #=> true
cancellation_b.canceled? #=> false
combined_cancellation.canceled? #=> false
origin_b.resolve
combined_cancellation.canceled? #=> true
```

@@ -0,0 +1,6 @@
require 'concurrent-edge'

def do_stuff(*args)
sleep 0.01
:stuff
end
Oops, something went wrong.

0 comments on commit f72141e

Please sign in to comment.
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.