Use parallel for nested loops in VirtualMachineImageService#list_all method #363
Conversation
|
Checked commit djberg96@f74cefb with ruby 2.3.3, rubocop 0.52.1, haml-lint 0.20.0, and yamllint 1.10.0 |
| Parallel.each(skus(offer.name, location, publisher.name), :in_threads => configuration.max_threads) do |sku| | ||
| Parallel.each(versions(sku.name, offer.name, location, publisher.name), :in_threads => configuration.max_threads) do |version| | ||
| mutex.synchronize do | ||
| images << Azure::Armrest::VirtualMachineImage.new( |
There was a problem hiding this comment.
Where is the place that the code needs to make API calls to the provider? Is it in the constructor? Maybe we can use multiple loops to collect the hashes, and finally use one Parallel.each to construct images.
There was a problem hiding this comment.
@bzwei There's more than one way to skin a cat, sure, but I don't think it's any cleaner or easier to read. I did something like that when I was fiddling with Excon:
https://gist.github.com/djberg96/f44d2cb7b930c2898ce7a8466bd54956
There was a problem hiding this comment.
The versions, skus and offers and publishers methods are where the actual API calls happen.
This updates the
VirtualMachineImageService#list_allmethod to use multiple loops via the Parallel library in order to significantly speed up the results. Currently this method takes over 20 minutes without threading. With threading the time drops down to about 1 minute to collect all results.Addresses #306