Deadlock using Edge Future chain & zip

[ms-ati]

I am seeing this from irb -rconcurrent-edge. Is this something I am doing wrong, or an error?

Array.new(4) do 
  Concurrent.zip(
    Array.new(5) do |i|
      Concurrent.delay do 
        puts "starting #{i}"
        sleep(i)
        puts "done #{i}"
        i
      end
    end
  )
end.inject { |a,b| a.chain { b }.flat }.value!
starting 0
done 0
starting 1
starting 2
starting 3
starting 4
done 1
done 2
done 3
done 4
fatal: No live threads left. Deadlock?
	from /Users/marcsiegel/.gem/ruby/2.3.1/gems/concurrent-ruby-edge-0.2.3.pre3/lib/concurrent/edge/future.rb:403:in `sleep'
	from /Users/marcsiegel/.gem/ruby/2.3.1/gems/concurrent-ruby-edge-0.2.3.pre3/lib/concurrent/edge/future.rb:403:in `wait'
	from /Users/marcsiegel/.gem/ruby/2.3.1/gems/concurrent-ruby-edge-0.2.3.pre3/lib/concurrent/edge/future.rb:403:in `block in wait_until_complete'
	from /Users/marcsiegel/.gem/ruby/2.3.1/gems/concurrent-ruby-edge-0.2.3.pre3/lib/concurrent/edge/future.rb:400:in `synchronize'
	from /Users/marcsiegel/.gem/ruby/2.3.1/gems/concurrent-ruby-edge-0.2.3.pre3/lib/concurrent/edge/future.rb:400:in `wait_until_complete'
	from /Users/marcsiegel/.gem/ruby/2.3.1/gems/concurrent-ruby-edge-0.2.3.pre3/lib/concurrent/edge/future.rb:784:in `wait_until_complete!'
	from /Users/marcsiegel/.gem/ruby/2.3.1/gems/concurrent-ruby-edge-0.2.3.pre3/lib/concurrent/edge/future.rb:620:in `value!'
	from (irb):1
	from /Users/marcsiegel/.rubies/ruby-2.3.1/bin/irb:11:in `<main>'

---

• Operating system: Mac OS X 10.11.6 (El Capitan)
• concurrent-ruby version: 1.0.3.pre3
• concurrent-ruby-ext installed: no
• concurrent-ruby-edge used: yes: 0.2.3.pre3

[ms-ati]

Note -- I am a Scala user as well, so my expectation and hope was I could use these in somewhat analogous ways to Scala Futures, but with the fully lazy .delay case

[pitr-ch]

Thanks for reporting! I may see the issue, the flat operation may not correctly propagate requests for value to delayed futures which are being flatted. We are about to release 1.0.3, I'll include a fix for this.

[ms-ati]

@pitr-ch Thanks so much! Looking forward to seeing a fix in 1.0.3

[ms-ati]

FYI, here's a workaround until this is fixed (gist here)

In case anyone else needs to compose sequential lazy Futures, each of which are a zipped set of parallel Futures:

@m = Mutex.new

def sync_puts(*args)
  @m.synchronize { puts(*args) }
end  

procs_to_delay_steps =
  Array.new(4) do |x|
    -> do
      sync_puts "\n-- starting step #{x}"

      Concurrent.zip(
        Array.new(5) do |i|
          Concurrent.future do 
            sync_puts "starting #{i}"
            sleep(i)
            sync_puts "done #{i}"
            i
          end
        end
      )
    end
  end  

proc_to_delay_chaining = -> do
  procs_to_delay_steps.inject do |a, b|
    f = a.respond_to?(:call) ? a.call : a
    f.chain { b.call }.flat
  end
end

sync_puts "Lazy. Nothing has happened yet."
sleep(1)

sync_puts "Invoking composed Proc of Futures"
proc_to_delay_chaining.call.value!

This correctly prints out:

Lazy. Nothing has happened yet.
Invoking composed Proc of Futures

-- starting step 0
starting 0
done 0
starting 1
starting 2
starting 3
starting 4
done 1
done 2
done 3
done 4

-- starting step 1
starting 0
done 0
starting 1
starting 3
starting 2
starting 4
done 1
done 2
done 3
done 4

-- starting step 2
starting 0
done 0
starting 4
starting 2
starting 1
starting 3
done 1
done 2
done 3
done 4

-- starting step 3
starting 0
done 0
starting 1
starting 4
starting 3
starting 2
done 1
done 2
done 3
done 4
=> [0, 1, 2, 3, 4]

[ms-ati]

Hi @pitr-ch: do you have any feeling on when 1.0.3 would be released with this?