Graphviz installation issues for Windows

[jennirinker]

Looks like Windows users need to jump through a few more hoops in order to get graphviz to work, which is unfortunately needed for any .visualize commands. I'll make a pull request with some clarification in the instructions once I sort it out fully, if that's alright.

[rpanai]

Do you mean anything not already included here? In any case docs should be improved regarding this library usage. Even on Linux I end up reading some solution on SO.

[jennirinker]

The only thing I know is that installing the python-graphviz package did not actually allow me to do z.visualize(), I ended up having to install the graphviz software and add it to the path manually. But I haven't checked exactly what the conda package is supposed to do, to be honest.

[jrbourbeau]

installing the python-graphviz package did not actually allow me to do z.visualize()

@jennirinker would it possible to elaborate a little more on this? Could you provide a traceback?

[cjc77]

I also had a bug with this and haven't figured out how to get it working. I am running Windows 10. Here is the error printed in the notebook.
(Note: I followed installation guide in 00_overview.ipynb in the section titled Prepare)

ModuleNotFoundError                       Traceback (most recent call last)
~\Anaconda3\lib\site-packages\dask\utils.py in import_required(mod_name, error_msg)
     90     try:
---> 91         return import_module(mod_name)
     92     except ImportError:

~\Anaconda3\lib\importlib\__init__.py in import_module(name, package)
    126             level += 1
--> 127     return _bootstrap._gcd_import(name[level:], package, level)
    128 

~\Anaconda3\lib\importlib\_bootstrap.py in _gcd_import(name, package, level)

~\Anaconda3\lib\importlib\_bootstrap.py in _find_and_load(name, import_)

~\Anaconda3\lib\importlib\_bootstrap.py in _find_and_load_unlocked(name, import_)

ModuleNotFoundError: No module named 'graphviz'

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
<ipython-input-7-be67cab1446e> in <module>
      1 # Look at the task graph for `z`
----> 2 z.visualize()

~\Anaconda3\lib\site-packages\dask\base.py in visualize(self, filename, format, optimize_graph, **kwargs)
     86         """
     87         return visualize(self, filename=filename, format=format,
---> 88                          optimize_graph=optimize_graph, **kwargs)
     89 
     90     def persist(self, **kwargs):

~\Anaconda3\lib\site-packages\dask\base.py in visualize(*args, **kwargs)
    446     https://docs.dask.org/en/latest/optimize.html
    447     """
--> 448     from dask.dot import dot_graph
    449 
    450     filename = kwargs.pop('filename', 'mydask')

~\Anaconda3\lib\site-packages\dask\dot.py in <module>
     10 
     11 
---> 12 graphviz = import_required("graphviz", "Drawing dask graphs requires the "
     13                                        "`graphviz` python library and the "
     14                                        "`graphviz` system library to be "

~\Anaconda3\lib\site-packages\dask\utils.py in import_required(mod_name, error_msg)
     91         return import_module(mod_name)
     92     except ImportError:
---> 93         raise RuntimeError(error_msg)
     94 
     95 

RuntimeError: Drawing dask graphs requires the `graphviz` python library and the `graphviz` system library to be installed.

[rpanai]

@cjc77 You should add this

source activate dask-tutorial
conda install -c conda-forge python-graphviz

[martindurant]

If that succeeds, would like to see it added to the instructions.

[rpanai]

@martindurant let me check and I'll eventually make a PR.

[cjc77]

@rpanai - thanks for the recommendation. However, this did not solve my issue. I have the following versions installed in my conda environment:

graphviz                  2.38.0            h6538335_1011    conda-forge
python-graphviz           0.10.1                     py_0    conda-forge

[TomAugspurger]

What message do you get from just >>> import graphviz in that environment?

Is Dask Py3 only now? We could change import required to use raise ImportError(error_msg) from e, if we're OK with a longer traceback.

[martindurant]

Is Dask Py3 only now

I believe we are free now to make some py3-only code, but starting with distributed first, and that they should be limited and easily rolled back. py2 is still in the test matrix for dask.

[rpanai]

@cjc77 If you are trying to run 01_dask.delayed.ipynb do you mind to check if after the first cell with %%time you can call variable x? There is a bug with ipython=7.4.0 and you should downgrade to 7.3.0 or wait for 7.5.0 which is about to be released.

[cjc77]

• @TomAugspurger : If I run a python shell from my dask-tutorial environment and execute import graphviz, everything seems to work fine. It is only when I try to run some_delayed_object.visualize() that I receive an error.
• @rpanai: I have indeed been experiencing a bug where, whenever a variable is defined in a cell that uses the %%time magic function, it is not available outside of that cell. I downgraded to ipython=7.3.0 but am still experiencing this error. For reference, I am using JupyterLab=0.35.4 installed via Anaconda.

[rpanai]

@cjc77 Are you in Windows or in Linux? Cos maybe is better to split the issue given that @jennirinker issue is (?) a windows only one

[cjc77]

@rpanai I am on Windows 10. #86 (comment)

[rpanai]

@cjc77 my bad. Anyway this is happening in Linux too.

[cjc77]

I started from scratch and now am getting a new error.

What I have installed (via Anaconda):

python-graphviz           0.10.1                     py_0    conda-forge
graphviz                  2.38.0            h6538335_1011    conda-forge

New Error:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
~\Anaconda3\envs\dask-tutorial\lib\site-packages\graphviz\backend.py in run(cmd, input, capture_output, check, quiet, **kwargs)
    146     try:
--> 147         proc = subprocess.Popen(cmd, startupinfo=get_startupinfo(), **kwargs)
    148     except OSError as e:

~\Anaconda3\envs\dask-tutorial\lib\subprocess.py in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, encoding, errors)
    728                                 errread, errwrite,
--> 729                                 restore_signals, start_new_session)
    730         except:

~\Anaconda3\envs\dask-tutorial\lib\subprocess.py in _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_start_new_session)
   1016                                          os.fspath(cwd) if cwd is not None else None,
-> 1017                                          startupinfo)
   1018             finally:

FileNotFoundError: [WinError 2] The system cannot find the file specified

During handling of the above exception, another exception occurred:

ExecutableNotFound                        Traceback (most recent call last)
<ipython-input-22-d5fa05822eb0> in <module>
----> 1 total.visualize()

~\Anaconda3\envs\dask-tutorial\lib\site-packages\dask\base.py in visualize(self, filename, format, optimize_graph, **kwargs)
     86         """
     87         return visualize(self, filename=filename, format=format,
---> 88                          optimize_graph=optimize_graph, **kwargs)
     89 
     90     def persist(self, **kwargs):

~\Anaconda3\envs\dask-tutorial\lib\site-packages\dask\base.py in visualize(*args, **kwargs)
    487         raise NotImplementedError("Unknown value color=%s" % color)
    488 
--> 489     return dot_graph(dsk, filename=filename, **kwargs)
    490 
    491 

~\Anaconda3\envs\dask-tutorial\lib\site-packages\dask\dot.py in dot_graph(dsk, filename, format, **kwargs)
    245     """
    246     g = to_graphviz(dsk, **kwargs)
--> 247     return graphviz_to_file(g, filename, format)
    248 
    249 

~\Anaconda3\envs\dask-tutorial\lib\site-packages\dask\dot.py in graphviz_to_file(g, filename, format)
    257         format = 'png'
    258 
--> 259     data = g.pipe(format=format)
    260     if not data:
    261         raise RuntimeError("Graphviz failed to properly produce an image. "

~\Anaconda3\envs\dask-tutorial\lib\site-packages\graphviz\files.py in pipe(self, format, renderer, formatter)
    126         data = text_type(self.source).encode(self._encoding)
    127 
--> 128         out = backend.pipe(self._engine, format, data, renderer, formatter)
    129 
    130         return out

~\Anaconda3\envs\dask-tutorial\lib\site-packages\graphviz\backend.py in pipe(engine, format, data, renderer, formatter, quiet)
    204     """
    205     cmd, _ = command(engine, format, None, renderer, formatter)
--> 206     out, _ = run(cmd, input=data, capture_output=True, check=True, quiet=quiet)
    207     return out
    208 

~\Anaconda3\envs\dask-tutorial\lib\site-packages\graphviz\backend.py in run(cmd, input, capture_output, check, quiet, **kwargs)
    148     except OSError as e:
    149         if e.errno == errno.ENOENT:
--> 150             raise ExecutableNotFound(cmd)
    151         else:  # pragma: no cover
    152             raise

ExecutableNotFound: failed to execute ['dot', '-Tpng'], make sure the Graphviz executables are on your systems' PATH

[jennirinker]

Hi all,

Sorry for my delayed response -- been on vacation.

What I had to do to get graphviz working was to install it manually and add the executable to the path. Found the solution somewhere on stackexchange. It seems that the conda recipes (either on the default channel or conda forge) don't seem to be sufficient for graphviz to work right...a classic Windows problem.

More detailed instructions:

1. Install graphviz using the windows release here
2. Add the graphviz bin folder (mine was at C:\Program Files (x86)\Graphviz2.38\bin) to your Windows Path variable
3. (Possibly not necessary) Close and reopen/reenter any running dask-tutorial environments

After doing this, and with the graphviz and python-graphviz packages installed, I was able to visualize the graphs. I'm not sure how necessary the two packages are after this manual installation, but I'm too lazy/busy atm to uninstall them and see if it breaks.

[cjc77]

To follow up on @jennirinker 's excellent instructions, I would recommend a system restart between steps 2 and 3 - after performing the installation steps, I still received the same exception in my Jupyter notebook until I restarted my machine. Things seem to work now.

[TomAugspurger]

I don’t think a restart should be necessary. They problem is ensuring that graphviz’s binaries are on the path of the notebook kernel. We can link to that section of the Graphviz docs and recommend an easy way to check for the commands.

…

On May 20, 2019, at 13:01, cjc77 ***@***.***> wrote: To follow up on @jennirinker 's excellent instructions, I would recommend a system restart between steps 2 and 3 - after performing the installation steps, I still received the same exception in my Jupyter notebook until I restarted my machine. Things seem to work now. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[jrbourbeau]

Note there was a recent fix in the python-graphviz conda-forge feedstock (see conda-forge/python-graphviz-feedstock#27) that should fix the Windows installation issues reported in this issue. I think

conda install -c conda-forge python-graphviz

should work on Windows today