Skip to content
JUN - python pandas, plotly, seaborn support & dataframes manipulation over erlang
Erlang Python Other
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config add config Jul 12, 2017
docs upgrade docs Jul 22, 2019
priv return inplace value instead undefined Jul 17, 2019
src bump version 0.1.19 Aug 27, 2019
test add test for complex legacy query using or into str Aug 27, 2019
.gitignore add pyc into gitignore Jul 21, 2017
.travis.yml use specific version for cufflinks in travis Aug 3, 2018
LICENSE Update LICENSE Jul 13, 2017
Makefile add covertool script and fix test target in makefile to generate cobe… Jul 23, 2017
README.md [skip travis-ci] update readme.md (ready for prod) Jul 4, 2019
codecov.yml setup codecov Jul 23, 2017
covertool
rebar.config move dependencies from git to hex & add rebar3_hex plugin Apr 11, 2018
rebar.lock upgrade rebar.lock Apr 11, 2018
rebar3 add script rebar3 Jul 12, 2017
relx.config add relx Jul 12, 2017

README.md

jun

Jun

JUN - python pandas, plotly, seaborn support & dataframes manipulation over erlang

Hex.pm Build Status Codecov License: MIT

JUN is a wrapper written in erlang to execute pandas, plotly, seaborn functions and manipuling dataframes creating an isolated environment, so you can handle multiple environments to treat dataframes in each of one them.

Creating an environment

To create an environment you need start a jun worker:

(jun@hurakann)1> {ok, Pid} = jun_worker:start_link().
{ok, <0.338.0>}

This will create and hold your environment in the Pid so you can start with the pandas usage from there.

Loading data frames

For now the only way to load a dataframe into jun is reading a csv (in later version we are planning support all methods of pandas api):

(jun@hurakann)2> {ok, {_, DataFrame}} = jun_pandas:read_csv(Pid, '/file.csv', []).
{ok,{'pandas.core.frame.DataFrame', {'$erlport.opaque',python,
                       <<128,2,99,112,97,110,100,97,115,46,99,111,114,101,46,
                         102,114,97,109,101,10,68,97,116,...>>}}}

Ensure that path exists and is in atom data type. The above code should return a dataframe in an opaque format, so this object is a serializable of the original in py env, this dataframe is stored into DataFrame variable, so you can use for other purposes such execute a query, getting max, min etc.

Manipulating dataframe

Now it's time to use some functions over our dataframe previously loaded, for example sum all values of age column:

(jun@hurakann)3> jun_pandas:sum(Pid, DataFrame, age, []).
{ok,13}

As you can see, the functions is executed through our environment and the response is delivered via the wrapper.

Handling errors

All errors will raise as a python errors, describing the class and arguments:

(jun@hurakann)4> jun_pandas:sum(Pid, DataFrame, id, []). 
{error,{'exceptions.KeyError',"Atom('id')"}}

Readable Data Frames into Erlang VM

If you noticed, we are working with opaque terms (serialization), this is because if we design a encoder/decoder for an erlang term to data frame in every execution this will be a very expensive task, so we decide use opaque terms, however sometimes you need to know the data frame in a readable syntax, for that purpose we design a single encoder:

(jun@hurakann)5> jun_pandas:to_erl(Pid, DataFrame).
{ok,{'pandas.core.frame.DataFrame',[<<"name">>,<<"age">>],
                                   [[<<"A">>,1],
                                    [<<"B">>,2],
                                    [<<"C">>,3],
                                    [<<"D">>,3],
                                    [<<"R">>,4]]}}

In the above snippet you can see how is a pandas dataframe represented in erlang term, the first is an atom with the class of the dataframe, the second are a list with the column names, the third is a list of lists with the values of each row.

Modules Index

Modules
jun_worker
jun_pandas
jun_pandas_series
jun_plotly
jun_seaborn

See also

KAA: support to execute jun tasks using protocol buffers

Authors

@zgbjgg Jorge Garrido zgbjgg@gmail.com

You can’t perform that action at this time.