Unable to install to install Optimus

[cpatte7372]

Hello all,

I'm really excited about what I've read about Optimus. Unfortunately, I'm coming across a number of issues getting it up and running.

I have followed the instructions here:

https://medium.com/hi-optimus/how-to-install-jupyter-notebook-4-4-0-and-optimus-on-ubuntu-18-04-92ff5ef30ea4

However, whenever i try to issue the following script to my notebook I get the following error:

import optimus as op

ModuleNotFoundError Traceback (most recent call last)
in ()
----> 1 import optimus as op

ModuleNotFoundError: No module named 'optimus'

Im running python 3.6

I feel that this is a simple error that I have made, but I'm at a loss.

Any help will be greatly appreciated.

Cheers

[FavioVazquez]

Hi! Thanks for the interest. That post it’s a little outdated. You can run:

pip install optimuspyspark

And then check the examples from the master branch!

You have to do a different import now.

Please let me know if this work for you! :)

[cpatte7372]

Hi FavioVazquez,

Thanks for reaching out. I'm sure I'm tried your suggestion and it didn't work out ... going to try it again now and let you know.

[cpatte7372]

OK,

I'm not getting the following error from the terminal when I issue the following command after installing pip install optimuspyspark from https://github.com/ironmussa/Optimus

from optimus import Optimus
op = Optimus()

packt@ubuntu-c:/.local/lib/python3.5/site-packages/optimus$ from optimus import Optimus
from: can't read /var/mail/optimus
packt@ubuntu-c:/.local/lib/python3.5/site-packages/optimus$ op = Optimus()
bash: syntax error near unexpected token `('
packt@ubuntu-c:~/.local/lib/python3.5/site-packages/optimus$

[cpatte7372]

Sorry, I meant to say I'm NOW getting the following error from the terminal when I issue the following command after installing pip install optimuspyspark from https://github.com/ironmussa/Optimus

from optimus import Optimus
op = Optimus()

packt@ubuntu-c:/.local/lib/python3.5/site-packages/optimus$ from optimus import Optimus
from: can't read /var/mail/optimus
packt@ubuntu-c:/.local/lib/python3.5/site-packages/optimus$ op = Optimus()
bash: syntax error near unexpected token `('
packt@ubuntu-c:~/.local/lib/python3.5/site-packages/optimus$

[cpatte7372]

Any thoughts?

[cpatte7372]

I tried to install on another unbuntu with the command sudo pip install optimuspyspark and I got the following error:

packt@ubuntu2:~$ sudo pip install optimuspyspark
The directory '/home/packt/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/packt/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting optimuspyspark
  Downloading https://files.pythonhosted.org/packages/56/6a/93e4f3cedb245028820915173d932e2002cb6121a60c2cab6b013963cc91/optimuspyspark-2.1.3.tar.gz (82kB)
    100% |████████████████████████████████| 92kB 269kB/s 
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-ust1jm/optimuspyspark/setup.py", line 14, in <module>
        with open('requirements.txt') as f:
    IOError: [Errno 2] No such file or directory: 'requirements.txt'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-ust1jm/optimuspyspark/
packt@ubuntu2:~$ 

[cpatte7372]

Any help will be greatly appreciated

[cpatte7372]

Hi, I managed to get this up and running. However, when I go through the example shown here
https://github.com/ironmussa/Optimus

I run the command df.cols.replace("product","taaaccoo","taco")\ but there isn't any change to the table

Any reasons why?

[argenisleon]

hi @cpatte7372
It should work, I just execute

df =op.load.url("https://raw.githubusercontent.com/ironmussa/Optimus/master/examples/data/foo.csv")
df.table()

def func(value, arg):
    return "this was a number"
    
df\
    .rows.sort("product","desc")\
    .cols.lower(["firstName","lastName"])\
    .cols.date_transform("birth", "yyyy/MM/dd", "dd-MM-YYYY")\
    .cols.years_between("birth", "yyyy/MM/dd")\
    .cols.remove_accents("lastName")\
    .cols.remove_special_chars("lastName")\
    .cols.replace("product","taaaccoo","taco")\
    .cols.replace("product",["piza","pizzza"],"pizza")\
    .rows.drop(df["id"]<7)\
    .cols.drop("dummyCol")\
    .cols.rename(str.lower)\
    .cols.apply_by_dtypes("product",func,"string", data_type="integer")\
    .cols.trim("*")\
    .table()

Is this what you expect?

[cpatte7372]

Hi ironmussa, This worked perfectly, thanks. I just have a few learning questions: 1. Can you explain what exactly the following command set out to achieve: def func(value, arg): return "this was a number 2. Is there place where I can find all the switches e.g. remove_special_chars, remove_accents, years_between. Thanks Carlton From: Argenis Leon <notifications@github.com> Sent: 10 November 2018 23:56 To: ironmussa/Optimus <Optimus@noreply.github.com> Cc: cpatte7372 <carlton@keyloop.co.uk>; Mention <mention@noreply.github.com> Subject: Re: [ironmussa/Optimus] Unable to install to install Optimus (#391) hi @cpatte7372 <https://github.com/cpatte7372> It should work, I just execute df =op.load.url("https://raw.githubusercontent.com/ironmussa/Optimus/master/examples/data/foo.csv") df.table() <https://user-images.githubusercontent.com/37144/48307388-5353b680-e511-11e8-95bc-eec2aaf6bf09.png> def func(value, arg): return "this was a number" df\ .rows.sort("product","desc")\ .cols.lower(["firstName","lastName"])\ .cols.date_transform("birth", "yyyy/MM/dd", "dd-MM-YYYY")\ .cols.years_between("birth", "yyyy/MM/dd")\ .cols.remove_accents("lastName")\ .cols.remove_special_chars("lastName")\ .cols.replace("product","taaaccoo","taco")\ .cols.replace("product",["piza","pizzza"],"pizza")\ .rows.drop(df["id"]<7)\ .cols.drop("dummyCol")\ .cols.rename(str.lower)\ .cols.apply_by_dtypes("product",func,"string", data_type="integer")\ .cols.trim("*")\ .table() <https://user-images.githubusercontent.com/37144/48307404-91e97100-e511-11e8-91b5-971813dd4fc5.png> Is this what you expect? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#391 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AXd82cCazDR7FAqYkVfjXYkeRu05AFGQks5ut2ergaJpZM4YXoY4> . <https://github.com/notifications/beacon/AXd82a1qwnz0omoyxC1gCOmuY3qXHlfWks5ut2ergaJpZM4YXoY4.gif>

[cpatte7372]

Hi argenisleon,

Any idea where I can find more cleansing examples?

Can you let me know where I can find a list of switches as shown below?
.cols.remove_accents
.cols.replace
.cols.rename

Thanks

[argenisleon]

@cpatte7372
About the first question:

def func(value, arg):
    return "this was a number"

Is a function applied all the integers in the 'product' column. More especicaflly in transform all integers to the string "this was a number"

.cols.apply_by_dtypes("product",func,"string", data_type="integer")

2. You can find all the columns and row operacions here and here

You also can find a lot of examples and uses cases in the examples folder

Hope this help

[cpatte7372]

@argenisleon

thanks for the information, that's great.

I'm trying to run the same commands from my Databricks workspace notebook (as you probably know Databricks is built on Spark and uses the same principle as Jupyter Notebook).

I issue the following commands:

!pip install optimuspyspark
!from optimus import Optimus
!op= Optimus()
!df =op.load.url("https://raw.githubusercontent.com/ironmussa/Optimus/master/examples/data/foo.csv")
!table()

However, there isn't any output. Do you have any idea why?

Cheers

[argenisleon]

@FavioVazquez can you please look at this?

[FavioVazquez]

Hi @cpatte7372! The table is a method of the df, so you have to run

df.table()

Please let me know if this helps :)

[argenisleon]

@cpatte7372 Did you solve the problem?

[cpatte7372]

@argenisleon this fixed the problem.

Thank you