:awesome = Elixir's Task ++ Basho's sidejob library
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config
lib
test
.gitignore
.travis.yml
CHANGELOG.md
LICENSE
README.md
mix.exs
mix.lock

README.md

SideTask

Build Status

SideTask is an alternative to Elixir's Task.Supervisor that uses Basho's sidejob library for better parallelism and to support capacity limiting of Tasks.

Elixir's Task.Supervisor is implemented as a single :simple_one_for_one supervisor with the individual Tasks as children. This means starting a new task has to go through this single supervisor. Furthermore there is no limit to the number of workers that can be running at the same time.

Basho's sidejob library spawns multiple supervisors (one for each scheduler by default) and uses ETS tables to keep track of the number of workers, thereby making it possible to start workers in parallel while also putting an upper bound on the number of running workers.

SideTask provides an API similar to Task.Supervisor, with the addition that all calls that start a new task require a sidejob resource as argument and can return {:error, :overload}.

Convenience functions for adding and deleting sidejob resources are provided.

Example 1

SideTask.add_resource(:example1, 50)
potential_tasks = for i <- 1..100 do
  case SideTask.async(:example1, fn -> :timer.sleep(1000); i end) do
    {:ok, task} ->
      IO.puts "Task #{i} created"
      task
    {:error, :overload} ->
      IO.puts "Task #{i} not created, overloaded"
      nil
  end
end
IO.inspect for task = %Task{} <- potential_tasks, do: Task.await(task)

Example 2

# Erlang spawns one scheduler per CPU core by default
schedulers = :erlang.system_info(:schedulers)
SideTask.add_resource(:example2, schedulers * 2)
for scheduler <- 1..schedulers do
  spawn_link fn ->
    for i <- Stream.iterate(0, &(&1+1)) do
      SideTask.start_child :example2, fn ->
        :timer.sleep(250);
        IO.inspect scheduler: scheduler, count: i
      end
    end
  end
end

Observer while running example 2

License

This software is licensed under the MIT license.

About

This project is maintained and funded by PSPDFKit.

See our other open source projects, read our blog or say hello on Twitter (@PSPDFKit).