Numpy again (and NetworkX) in DrawBotPy3

[MauriceMeilleur]

(PS: since Just closed the last version of this issue, I can't re-open it—sorry.)

Hate to dig up an old issue, but I'm at a point in a current project in DBPy3 where I am ahead of my skis regarding graph and network functions, and could sure use numpy and NetworkX. I can import numpy in the 2.7 version of DrawBot thanks to last year's thread and Just/Frederik's help (thanks again, guys), but not NetworkX—and I can't import either one in Py3.

I'm happy to download and install these modules by hand and update directories myself—for whatever reason, perhaps because of permissions issues with a school machine, who knows—I can't install these modules with pip (either pip can't find them or appropriate versions of them, or DB can't find them once they're installed, and sudo runs me into permissions errors). I just need to know where to put them and what edits to what scripts need making. (Again feeling the shame of being a Python dilettante, sorry.)

[typemytype]

do you have python 3.6 installed on your local system? and is numpy installed for python3?

DrawBot finds the local site packages the python version and makes all those packages available inside DrawBot

[justvanrossum]

Also: Python 3.6 needs to be from python.org, not homebrew.

[MauriceMeilleur]

Frederik, I installed 3.6 with the macOS 64-bit/32-bit installer at https://www.python.org/downloads/release/python-365/. I see the version in Macintosh HD/Library/Frameworks/Python.framework/Versions/3.6

From Terminal I was able to install both numpy and NetworkX with
python3 -m pip install --user numpy scipy matplotlib ipython jupyter pandas sympy nose
and
python3 -m pip install --user networkx==2.1

I see Python 3.6 in the package contents for DrawBotPy3, too (Contents/Frameworks/Python.framework/Versions). But nothing in my user account.

[typemytype]

so this works in terminal

python3
import numpy
numpy

drawBot add these paths available https://github.com/typemytype/drawbot/blob/master/drawBot/scriptTools.py#L94-L97

if the packages are not available there, drawBot will not find it

[MauriceMeilleur]

Seems so, here's the Terminal log:

Last login: Tue Jun 5 15:01:25 on ttys000
ART-C02PV4VHFVH8:~ meilleurmj$ python3
Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 03:03:55)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import numpy
numpy
<module 'numpy' from '/Users/meilleurmj/Library/Python/3.6/lib/python/site-packages/numpy/init.py'>

[typemytype]

mmmm....

its installed in your user...

must be available in /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages

[MauriceMeilleur]

Do these lines:

if PY3:
        paths = [
            # add local stdlib and site-packages; TODO: this needs editing once we embed the full stdlib
            '/Library/Frameworks/Python.framework/Versions/%s/lib/python%s' % (version, version),
            '/Library/Frameworks/Python.framework/Versions/%s/lib/python%s/lib-dynload' % (version, version),
            '/Library/Frameworks/Python.framework/Versions/%s/lib/python%s/site-packages' % (version, version),
        ]

reference MacintoshHD/Library or my user library?

[MauriceMeilleur]

Just checked: nothing in the fallback System/Library/etc. …

[MauriceMeilleur]

So, I know this is like asking how you sharpen a pencil, but: if I want these packages installed globally, how would I do that in Terminal? It looks to me like I can't even find Python 3.x.x in the root Library/Python, though I do see it in the root Library/Frameworks/Versions folder …

[justvanrossum]

Try to leave out the --user option when running pip.

[MauriceMeilleur]

Here's the log without the --user flag. Like I wrote, the funny thing is that I don't even see Python 3.6 in ./Library/Python at all, nor in ./System/Library/Frameworks/Python.framework/Versions—though I do see it in ./Library/Frameworks/Python.framework/Versions.

ART-C02PV4VHFVH8:~ meilleurmj$ python3 -m pip install numpy scipy matplotlib ipython jupyter pandas sympy nose
Requirement already satisfied: numpy in ./Library/Python/3.6/lib/python/site-packages
Requirement already satisfied: scipy in ./Library/Python/3.6/lib/python/site-packages
Requirement already satisfied: matplotlib in ./Library/Python/3.6/lib/python/site-packages
Requirement already satisfied: ipython in ./Library/Python/3.6/lib/python/site-packages
Requirement already satisfied: jupyter in ./Library/Python/3.6/lib/python/site-packages
Requirement already satisfied: pandas in ./Library/Python/3.6/lib/python/site-packages
Requirement already satisfied: sympy in ./Library/Python/3.6/lib/python/site-packages
Requirement already satisfied: nose in ./Library/Python/3.6/lib/python/site-packages
Requirement already satisfied: python-dateutil>=2.1 in ./Library/Python/3.6/lib/python/site-packages (from matplotlib)
Requirement already satisfied: kiwisolver>=1.0.1 in ./Library/Python/3.6/lib/python/site-packages (from matplotlib)
Requirement already satisfied: six>=1.10 in ./Library/Python/3.6/lib/python/site-packages (from matplotlib)
Requirement already satisfied: cycler>=0.10 in ./Library/Python/3.6/lib/python/site-packages (from matplotlib)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in ./Library/Python/3.6/lib/python/site-packages (from matplotlib)
Requirement already satisfied: pytz in ./Library/Python/3.6/lib/python/site-packages (from matplotlib)
Requirement already satisfied: pexpect; sys_platform != "win32" in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: backcall in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: prompt-toolkit<2.0.0,>=1.0.15 in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: decorator in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: pickleshare in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: traitlets>=4.2 in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: appnope; sys_platform == "darwin" in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: pygments in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: setuptools>=18.5 in /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages (from ipython)
Requirement already satisfied: jedi>=0.10 in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: simplegeneric>0.8 in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: nbconvert in ./Library/Python/3.6/lib/python/site-packages (from jupyter)
Requirement already satisfied: ipywidgets in ./Library/Python/3.6/lib/python/site-packages (from jupyter)
Requirement already satisfied: jupyter-console in ./Library/Python/3.6/lib/python/site-packages (from jupyter)
Requirement already satisfied: notebook in ./Library/Python/3.6/lib/python/site-packages (from jupyter)
Requirement already satisfied: qtconsole in ./Library/Python/3.6/lib/python/site-packages (from jupyter)
Requirement already satisfied: ipykernel in ./Library/Python/3.6/lib/python/site-packages (from jupyter)
Requirement already satisfied: mpmath>=0.19 in ./Library/Python/3.6/lib/python/site-packages (from sympy)
Requirement already satisfied: ptyprocess>=0.5 in ./Library/Python/3.6/lib/python/site-packages (from pexpect; sys_platform != "win32"->ipython)
Requirement already satisfied: wcwidth in ./Library/Python/3.6/lib/python/site-packages (from prompt-toolkit<2.0.0,>=1.0.15->ipython)
Requirement already satisfied: ipython-genutils in ./Library/Python/3.6/lib/python/site-packages (from traitlets>=4.2->ipython)
Requirement already satisfied: parso>=0.2.0 in ./Library/Python/3.6/lib/python/site-packages (from jedi>=0.10->ipython)
Requirement already satisfied: jupyter-core in ./Library/Python/3.6/lib/python/site-packages (from nbconvert->jupyter)
Requirement already satisfied: mistune>=0.7.4 in ./Library/Python/3.6/lib/python/site-packages (from nbconvert->jupyter)
Requirement already satisfied: jinja2 in ./Library/Python/3.6/lib/python/site-packages (from nbconvert->jupyter)
Requirement already satisfied: pandocfilters>=1.4.1 in ./Library/Python/3.6/lib/python/site-packages (from nbconvert->jupyter)
Requirement already satisfied: bleach in ./Library/Python/3.6/lib/python/site-packages (from nbconvert->jupyter)
Requirement already satisfied: entrypoints>=0.2.2 in ./Library/Python/3.6/lib/python/site-packages (from nbconvert->jupyter)
Requirement already satisfied: testpath in ./Library/Python/3.6/lib/python/site-packages (from nbconvert->jupyter)
Requirement already satisfied: nbformat>=4.4 in ./Library/Python/3.6/lib/python/site-packages (from nbconvert->jupyter)
Requirement already satisfied: widgetsnbextension~=3.2.0 in ./Library/Python/3.6/lib/python/site-packages (from ipywidgets->jupyter)
Requirement already satisfied: jupyter-client in ./Library/Python/3.6/lib/python/site-packages (from jupyter-console->jupyter)
Requirement already satisfied: pyzmq>=17 in ./Library/Python/3.6/lib/python/site-packages (from notebook->jupyter)
Requirement already satisfied: terminado>=0.8.1 in ./Library/Python/3.6/lib/python/site-packages (from notebook->jupyter)
Requirement already satisfied: Send2Trash in ./Library/Python/3.6/lib/python/site-packages (from notebook->jupyter)
Requirement already satisfied: tornado>=4 in ./Library/Python/3.6/lib/python/site-packages (from notebook->jupyter)
Requirement already satisfied: MarkupSafe>=0.23 in ./Library/Python/3.6/lib/python/site-packages (from jinja2->nbconvert->jupyter)
Requirement already satisfied: html5lib!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8,>=0.99999999pre in ./Library/Python/3.6/lib/python/site-packages (from bleach->nbconvert->jupyter)
Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in ./Library/Python/3.6/lib/python/site-packages (from nbformat>=4.4->nbconvert->jupyter)
Requirement already satisfied: webencodings in ./Library/Python/3.6/lib/python/site-packages (from html5lib!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8,>=0.99999999pre->bleach->nbconvert->jupyter)
You are using pip version 9.0.3, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
ART-C02PV4VHFVH8:~ meilleurmj$

[justvanrossum]

Ugh, maybe you need to uninstall the packages first, before they properly get installed without the --user option. I assume pip uninstall <name> should do it.

(We really need to bundle pip with DrawBot and have a proper way to do these things.)

[MauriceMeilleur]

I'll give it a try—uninstalling went without anything that looked like a problem. Sorry to be bothering you guys with such low-level stuff …

[MauriceMeilleur]

Okay, partway there: here's the log for both new installs:

Last login: Tue Jun  5 15:57:20 on ttys000
ART-C02PV4VHFVH8:~ meilleurmj$ python3 -m pip install numpy scipy matplotlib ipython jupyter pandas sympy nose
Collecting numpy
  Using cached https://files.pythonhosted.org/packages/8e/75/7a8b7e3c073562563473f2a61bd53e75d0a1f5e2047e576ee61d44113c22/numpy-1.14.3-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Collecting scipy
  Using cached https://files.pythonhosted.org/packages/a0/b6/70bf61c1badb5fea82d4c558e05e76c2dee5e77bb072fe465d7c7a87287d/scipy-1.1.0-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Collecting matplotlib
  Using cached https://files.pythonhosted.org/packages/8a/d5/5337662b714c65100f3545ed3909e9478614d1ebf1f692a52981f3f5167b/matplotlib-2.2.2-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Collecting ipython
  Using cached https://files.pythonhosted.org/packages/b1/7f/91d50f28af3e3a24342561983a7857e399ce24093876e6970b986a0b6677/ipython-6.4.0-py3-none-any.whl
Collecting jupyter
  Using cached https://files.pythonhosted.org/packages/83/df/0f5dd132200728a86190397e1ea87cd76244e42d39ec5e88efd25b2abd7e/jupyter-1.0.0-py2.py3-none-any.whl
Collecting pandas
  Using cached https://files.pythonhosted.org/packages/59/07/a6c964fe735572139f270983896fb98a639ff7cedc9bb6568868ab75db6b/pandas-0.23.0-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Collecting sympy
  Using cached https://files.pythonhosted.org/packages/91/26/4e477dbd1f9260eb743d9f221af3044648a8fb2fcf3f2f23daee4dc831a4/sympy-1.1.1.tar.gz
Collecting nose
  Using cached https://files.pythonhosted.org/packages/15/d8/dd071918c040f50fa1cf80da16423af51ff8ce4a0f2399b7bf8de45ac3d9/nose-1.3.7-py3-none-any.whl
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in ./Library/Python/3.6/lib/python/site-packages (from matplotlib)
Requirement already satisfied: python-dateutil>=2.1 in ./Library/Python/3.6/lib/python/site-packages (from matplotlib)
Requirement already satisfied: six>=1.10 in ./Library/Python/3.6/lib/python/site-packages (from matplotlib)
Requirement already satisfied: pytz in ./Library/Python/3.6/lib/python/site-packages (from matplotlib)
Requirement already satisfied: kiwisolver>=1.0.1 in ./Library/Python/3.6/lib/python/site-packages (from matplotlib)
Requirement already satisfied: cycler>=0.10 in ./Library/Python/3.6/lib/python/site-packages (from matplotlib)
Requirement already satisfied: pickleshare in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: pexpect; sys_platform != "win32" in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: appnope; sys_platform == "darwin" in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: pygments in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: prompt-toolkit<2.0.0,>=1.0.15 in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: simplegeneric>0.8 in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: backcall in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: setuptools>=18.5 in /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages (from ipython)
Requirement already satisfied: traitlets>=4.2 in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: decorator in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: jedi>=0.10 in ./Library/Python/3.6/lib/python/site-packages (from ipython)
Requirement already satisfied: notebook in ./Library/Python/3.6/lib/python/site-packages (from jupyter)
Requirement already satisfied: ipykernel in ./Library/Python/3.6/lib/python/site-packages (from jupyter)
Requirement already satisfied: nbconvert in ./Library/Python/3.6/lib/python/site-packages (from jupyter)
Requirement already satisfied: qtconsole in ./Library/Python/3.6/lib/python/site-packages (from jupyter)
Requirement already satisfied: jupyter-console in ./Library/Python/3.6/lib/python/site-packages (from jupyter)
Requirement already satisfied: ipywidgets in ./Library/Python/3.6/lib/python/site-packages (from jupyter)
Requirement already satisfied: mpmath>=0.19 in ./Library/Python/3.6/lib/python/site-packages (from sympy)
Requirement already satisfied: ptyprocess>=0.5 in ./Library/Python/3.6/lib/python/site-packages (from pexpect; sys_platform != "win32"->ipython)
Requirement already satisfied: wcwidth in ./Library/Python/3.6/lib/python/site-packages (from prompt-toolkit<2.0.0,>=1.0.15->ipython)
Requirement already satisfied: ipython-genutils in ./Library/Python/3.6/lib/python/site-packages (from traitlets>=4.2->ipython)
Requirement already satisfied: parso>=0.2.0 in ./Library/Python/3.6/lib/python/site-packages (from jedi>=0.10->ipython)
Requirement already satisfied: jinja2 in ./Library/Python/3.6/lib/python/site-packages (from notebook->jupyter)
Requirement already satisfied: pyzmq>=17 in ./Library/Python/3.6/lib/python/site-packages (from notebook->jupyter)
Requirement already satisfied: tornado>=4 in ./Library/Python/3.6/lib/python/site-packages (from notebook->jupyter)
Requirement already satisfied: jupyter-core>=4.4.0 in ./Library/Python/3.6/lib/python/site-packages (from notebook->jupyter)
Requirement already satisfied: nbformat in ./Library/Python/3.6/lib/python/site-packages (from notebook->jupyter)
Requirement already satisfied: terminado>=0.8.1 in ./Library/Python/3.6/lib/python/site-packages (from notebook->jupyter)
Requirement already satisfied: jupyter-client>=5.2.0 in ./Library/Python/3.6/lib/python/site-packages (from notebook->jupyter)
Requirement already satisfied: Send2Trash in ./Library/Python/3.6/lib/python/site-packages (from notebook->jupyter)
Requirement already satisfied: entrypoints>=0.2.2 in ./Library/Python/3.6/lib/python/site-packages (from nbconvert->jupyter)
Requirement already satisfied: pandocfilters>=1.4.1 in ./Library/Python/3.6/lib/python/site-packages (from nbconvert->jupyter)
Requirement already satisfied: bleach in ./Library/Python/3.6/lib/python/site-packages (from nbconvert->jupyter)
Requirement already satisfied: testpath in ./Library/Python/3.6/lib/python/site-packages (from nbconvert->jupyter)
Requirement already satisfied: mistune>=0.7.4 in ./Library/Python/3.6/lib/python/site-packages (from nbconvert->jupyter)
Requirement already satisfied: widgetsnbextension~=3.2.0 in ./Library/Python/3.6/lib/python/site-packages (from ipywidgets->jupyter)
Requirement already satisfied: MarkupSafe>=0.23 in ./Library/Python/3.6/lib/python/site-packages (from jinja2->notebook->jupyter)
Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in ./Library/Python/3.6/lib/python/site-packages (from nbformat->notebook->jupyter)
Requirement already satisfied: html5lib!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8,>=0.99999999pre in ./Library/Python/3.6/lib/python/site-packages (from bleach->nbconvert->jupyter)
Requirement already satisfied: webencodings in ./Library/Python/3.6/lib/python/site-packages (from html5lib!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8,>=0.99999999pre->bleach->nbconvert->jupyter)
Installing collected packages: numpy, scipy, matplotlib, ipython, jupyter, pandas, sympy, nose
  Running setup.py install for sympy ... done
Successfully installed ipython-6.4.0 jupyter-1.0.0 matplotlib-2.2.2 nose-1.3.7 numpy-1.14.3 pandas-0.23.0 scipy-1.1.0 sympy-1.1.1
You are using pip version 9.0.3, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
ART-C02PV4VHFVH8:~ meilleurmj$ python3 -m pip install networkx==2.1
Collecting networkx==2.1
  Using cached https://files.pythonhosted.org/packages/11/42/f951cc6838a4dff6ce57211c4d7f8444809ccbe2134179950301e5c4c83c/networkx-2.1.zip
Requirement already satisfied: decorator>=4.1.0 in ./Library/Python/3.6/lib/python/site-packages (from networkx==2.1)
Installing collected packages: networkx
  Running setup.py install for networkx ... done
Successfully installed networkx-2.1
You are using pip version 9.0.3, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
ART-C02PV4VHFVH8:~ meilleurmj$ 

New window in DBPy3, here's the code:

import numpy
import networkx

and here's the output: nothing wrong with just numpy (hooray!), but with the networkx call:

Traceback (most recent call last):
  File "<untitled>", line 2, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/networkx/__init__.py", line 98, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/networkx/utils/__init__.py", line 2, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/networkx/utils/decorators.py", line 8, in <module>
ModuleNotFoundError: No module named 'decorator'

[justvanrossum]

Maybe decorator is still installed as --user? Does import decorator work in the command line python3 interpreter? (I don't know the module.)

[MauriceMeilleur]

That did it! I think.

from networkx import *
from numpy import *
import networkx
import numpy

None of these throw any flags. Tomorrow I'll give them a try. Thanks, you two!

[MauriceMeilleur]

Close #209

[MauriceMeilleur]

Well, this is embarrassing: I'm trying to get a new laptop set up, and figured I'd refer to this issue thread: install Python 3.7 from the python.org package (and not HomeBrew), install numpy and networkx either with pip3 or python3 -m pip and without the --user flag … and guess what?

import networkx
import numpy

both throw ModuleNotFoundErrors. Clearly I'm in for remedial training, but I could use any advice I can get right now—sketches I'm working on use both modules, and I'm stuck without them …

[gferreira]

DrawBot uses Python 3.6, so you need to make sure that your modules are being installed for that version of Python (not 3.7). with pip: pip3.6 install networkx

[MauriceMeilleur]

Thanks, Gustavo—I didn't understand how modules are made available to which versions of Python, and now I (sort of) do. Fixed.