dot_graph writes empty png

[fonnesbeck]

Following Jake VDP's tutorial blog post, I am attempting to plot a dask graph. However, dot_graph plots an empty png. Here is the example. The resulting png file is zero bytes in size. As you can see, there is no exception raised.

Using dask 0.7.5 built from master on OS X 10.11, Anaconda Python 3.5. I've installed graphviz and pydot2 from pip.

[mrocklin]

@fonnesbeck can you verify that your graphviz system library works correctly? E.g. can you try writing out a .dot file and then using the dot command line utility directly?

>>> # dot_graph(b2_min.dask, filename='b2_min.dot')
>>> b2_min.visualize(filename='b2_min.dot')  # perhaps nicer syntax

$ dot -Tpng b2_min.dot -o b2_min.png

[mrocklin]

Or, an alternative test would be to install graphviz with a tool like brew.

[fonnesbeck]

I have GV installed via brew:

$ brew install graphviz
Warning: graphviz-2.38.0 already installed

[mrocklin]

I mention the tests above because, when others ran into problems like this (cc @Carreau) it was because the system graphviz library was not compiled with libpng, and so the dot application failed on png files. Perhaps you're running into a different problem though. My intention with the questions above was to determine if this was the same issue.

You could also try visualizing with svg, which, if it works, would corroborate this theory.

import dask.array as da
da.ones(5, chunks=2).visualize(filename='foo.svg')

[fonnesbeck]

When I did export to a .dot file, I had the following in the terminal:

dyld: Library not loaded: /usr/lib/libltdl.7.dylib
  Referenced from: /Users/fonnescj/anaconda3/bin/dot
  Reason: image not found

[fonnesbeck]

Your visualize example fails with the following:

---------------------------------------------------------------------------
ExpatError                                Traceback (most recent call last)
<ipython-input-9-01f98d3386e3> in <module>()
      1 import dask.array as da
----> 2 da.ones(5, chunks=2).visualize(filename='foo.svg')

/Users/fonnescj/anaconda3/lib/python3.5/site-packages/dask/base.py in visualize(self, filename, format, optimize_graph, **kwargs)
     25                   **kwargs):
     26         return visualize(self, filename=filename, format=format,
---> 27                          optimize_graph=optimize_graph, **kwargs)
     28 
     29     def _visualize(self, filename='mydask', format=None, optimize_graph=False):

/Users/fonnescj/anaconda3/lib/python3.5/site-packages/dask/base.py in visualize(*args, **kwargs)
    125     dsk = merge(dsks)
    126 
--> 127     return dot_graph(dsk, filename=filename, **kwargs)
    128 
    129 

/Users/fonnescj/anaconda3/lib/python3.5/site-packages/dask/dot.py in dot_graph(dsk, filename, format, **kwargs)
    218         f.write(data)
    219 
--> 220     return _get_display_cls(format)(filename=full_filename)

/Users/fonnescj/anaconda3/lib/python3.5/site-packages/IPython/core/display.py in __init__(self, data, url, filename)
    384         self.filename = None if filename is None else unicode_type(filename)
    385 
--> 386         self.reload()
    387         self._check_data()
    388 

/Users/fonnescj/anaconda3/lib/python3.5/site-packages/IPython/core/display.py in reload(self)
    403         if self.filename is not None:
    404             with open(self.filename, self._read_flags) as f:
--> 405                 self.data = f.read()
    406         elif self.url is not None:
    407             try:

/Users/fonnescj/anaconda3/lib/python3.5/site-packages/IPython/core/display.py in data(self, svg)
    488         from xml.dom import minidom
    489         svg = cast_bytes_py2(svg)
--> 490         x = minidom.parseString(svg)
    491         # get svg tag (should be 1)
    492         found_svg = x.getElementsByTagName('svg')

/Users/fonnescj/anaconda3/lib/python3.5/xml/dom/minidom.py in parseString(string, parser)
   1966     if parser is None:
   1967         from xml.dom import expatbuilder
-> 1968         return expatbuilder.parseString(string)
   1969     else:
   1970         from xml.dom import pulldom

/Users/fonnescj/anaconda3/lib/python3.5/xml/dom/expatbuilder.py in parseString(string, namespaces)
    923     else:
    924         builder = ExpatBuilder()
--> 925     return builder.parseString(string)
    926 
    927 

/Users/fonnescj/anaconda3/lib/python3.5/xml/dom/expatbuilder.py in parseString(self, string)
    221         parser = self.getParser()
    222         try:
--> 223             parser.Parse(string, True)
    224             self._setup_subset(string)
    225         except ParseEscape:

ExpatError: no element found: line 1, column 0

[mrocklin]

I suspect that the graphviz executables in anaconda are broken (note that the python graphviz library is probably fine.) I recommend uninstalling it and going with what brew would install. Will raise an issue with Anaconda folk.

conda remove graphviz

[fonnesbeck]

Yup, that is the issue. Removing conda's graphviz fixed all the examples.

[mrocklin]

OK, sorry for the runaround. If you have the time, a quick mention from you in the anaconda-issue linked above might help to get this issue resolved upstream.

[fonnesbeck]

Great, thanks for troubleshooting.

[mrocklin]

We should probably raise an informative error message if this occurs.

[d-diaz]

I am having this same issue. Installed on Windows 10 using conda-forge for dask (2.12.0) and python-graphviz (0.13.2). Installing python-graphviz also grabbed graphviz (2.38.0).

I was able to resolve this by uninstalling the conda python-graphviz package and using pip install graphviz instead.

Reporting here because I am not receiving a useful error message, as should be the case according to #1027.

import dask.array as da
da.ones(5, chunks=2).visualize(filename='foo.svg')

---------------------------------------------------------------------------
ExpatError                                Traceback (most recent call last)
<ipython-input-49-ceaf49eae531> in <module>
      1 import dask.array as da
----> 2 da.ones(5, chunks=2).visualize(filename='foo.svg')

C:\Miniconda\envs\pyFIRS\lib\site-packages\dask\base.py in visualize(self, filename, format, optimize_graph, **kwargs)
     90         https://docs.dask.org/en/latest/optimize.html
     91         """
---> 92         return visualize(
     93             self,
     94             filename=filename,

C:\Miniconda\envs\pyFIRS\lib\site-packages\dask\base.py in visualize(*args, **kwargs)
    529         raise NotImplementedError("Unknown value color=%s" % color)
    530 
--> 531     return dot_graph(dsk, filename=filename, **kwargs)
    532 
    533 

C:\Miniconda\envs\pyFIRS\lib\site-packages\dask\dot.py in dot_graph(dsk, filename, format, **kwargs)
    253     """
    254     g = to_graphviz(dsk, **kwargs)
--> 255     return graphviz_to_file(g, filename, format)
    256 
    257 

C:\Miniconda\envs\pyFIRS\lib\site-packages\dask\dot.py in graphviz_to_file(g, filename, format)
    284         f.write(data)
    285 
--> 286     return display_cls(filename=full_filename)

C:\Miniconda\envs\pyFIRS\lib\site-packages\IPython\core\display.py in __init__(self, data, url, filename, metadata)
    625             self.metadata = {}
    626 
--> 627         self.reload()
    628         self._check_data()
    629 

C:\Miniconda\envs\pyFIRS\lib\site-packages\IPython\core\display.py in reload(self)
    651         if self.filename is not None:
    652             with open(self.filename, self._read_flags) as f:
--> 653                 self.data = f.read()
    654         elif self.url is not None:
    655             try:

C:\Miniconda\envs\pyFIRS\lib\site-packages\IPython\core\display.py in data(self, svg)
    754         # parse into dom object
    755         from xml.dom import minidom
--> 756         x = minidom.parseString(svg)
    757         # get svg tag (should be 1)
    758         found_svg = x.getElementsByTagName('svg')

C:\Miniconda\envs\pyFIRS\lib\xml\dom\minidom.py in parseString(string, parser)
   1967     if parser is None:
   1968         from xml.dom import expatbuilder
-> 1969         return expatbuilder.parseString(string)
   1970     else:
   1971         from xml.dom import pulldom

C:\Miniconda\envs\pyFIRS\lib\xml\dom\expatbuilder.py in parseString(string, namespaces)
    923     else:
    924         builder = ExpatBuilder()
--> 925     return builder.parseString(string)
    926 
    927 

C:\Miniconda\envs\pyFIRS\lib\xml\dom\expatbuilder.py in parseString(self, string)
    221         parser = self.getParser()
    222         try:
--> 223             parser.Parse(string, True)
    224             self._setup_subset(string)
    225         except ParseEscape:

ExpatError: not well-formed (invalid token): line 1, column 0

[quasiben]

As this issue is several years old I would suggest opening a new issue