Run your Python algorithms in parallel and avoid the GIL
Branch: master
Clone or download
gonzalo
gonzalo Elixir 1.7
Latest commit 5ab3ae4 Aug 17, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config First commit May 31, 2017
lib elixir 1.6 and documentation Mar 23, 2018
test removing piton doc test Mar 23, 2018
.formatter.exs elixir 1.6 and documentation Mar 23, 2018
.gitignore Elixir 1.7 Aug 17, 2018
.travis.yml Elixir 1.7 Aug 17, 2018
LICENSE Initial commit May 31, 2017
README.md
mix.exs Elixir 1.7 Aug 17, 2018
mix.lock Elixir 1.7 Aug 17, 2018

README.md

Piton

hex.pm hexdocs.pm Build Status

Piton is a library which will help you to run your Python code.

You can implement your own Piton.Port and run your python code but I highly recommend to use Piton.Pool, a pool which will allow to run Python code in parallel, a way of avoiding the GIL, and it will protect you from python exceptions.

Installation

Add piton to your list of dependencies in mix.exs:

def deps do
  [{:piton, "~> 0.4.0"}]
end

How to use it

Define your own port

  • The Easiest one
defmodule MySimplePort do
  use Piton.Port
end
  • A port with some wrapper functions which will help you to call the python function: YOUR_MODULE.execute(pid, python_module, python_function, list_of_arguments)
defmodule MyCustomPort do
  use Piton.Port
  def start_link(), do: MyCustomPort.start_link([path: Path.expand("python_folder"), python: "python"], [name: __MODULE__])
  def fun(n), do: MyCustomPort.execute(__MODULE__, :functions, :fun, [n])
end
  • A port prepared to be run by Piton.Pool They have to have a function start() and it has not to be linked.
defmodule MyPoolPort do
  use Piton.Port
  def start(), do: MyPoolPort.start([path: Path.expand("python_folder"), python: "python"], [])
  def fun(pid, n), do: MyPoolPort.execute(pid, :functions, :fun, [n])
end

Run a Pool

Pay attention to the number of Pythons you want to run in parallel. It does not exist an optimal number, maybe it is the number of cores, maybe half or maybe double. Test it with your application.

{:ok, pool} = Piton.Pool.start_link([module: MyPoolPort, pool_number: pool_number], [])

Call a Port (No pool)

iex> MyCustomPort.execute(pid_of_the_port, python_module, python_function, list_of_arguments_of_python_function)

Call a Pool

iex> Piton.Pool.execute(pid_of_the_pool, elixir_function, list_of_arguments_of_elixir_function)

Tasks

Some Mix.Tasks have been included in order to facilitate the integration of a python project

  • Mix.Tasks.Piton.Venv: It creates a Python Virtual Environment.
  • Mix.Tasks.Piton.Pip: It upgrades the Python pip.
  • Mix.Tasks.Piton.Requirements: It gets the dependencies of the Python project.

Test

Run the tests.

mix test 

Name

Pitón is only Python in Spanish 😜 🐍