Make graphviz work in Jupyterlite

[MarcSkovMadsen]

Intro

Jupyterlite is Jupyter running in the browser only powered by Pyodide. It is a very easy way for user and frameworks to start up a Jupyter environment.

I'm a contributor to Panel which is a framework for easily creating interactive components for the notebooks and data apps to be deployed on a server. Panel also runs on Pyodide and Pyscript. We also have a custom version of Jupyterlite called Panelite where Panel and the rest of the HoloViz ecosystem is supposed to work.

I'm currently testing the Panelite notebooks and fixing issues. One of the notebooks displays how to integrate Panel and Graphviz. But this one is not working and raises OSError: [Errno 52] Function not implemented.

Reproduce

• Open Jupyterlite in your browser: https://jupyterlite.readthedocs.io/en/latest/_static/lab/index.html
• Run the code below in a new Pyodide notebook and observe the issue

import piplite
await piplite.install(['graphviz'])

from graphviz import Graph

graphviz_graph = Graph("Hello", format='svg', node_attr={'color': '#00aa41', 'style': 'filled', "fontcolor": 'white'})
graphviz_graph.attr(bgcolor='#A01346:pink', label='My Awesome Graph', fontcolor='white')
graphviz_graph.edge("Hello", "world")

graphviz_graph

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
File /lib/python3.11/site-packages/IPython/core/formatters.py:974, in MimeBundleFormatter.__call__(self, obj, include, exclude)
    971     method = get_real_method(obj, self.print_method)
    973     if method is not None:
--> 974         return method(include=include, exclude=exclude)
    975     return None
    976 else:

File /lib/python3.11/site-packages/graphviz/jupyter_integration.py:98, in JupyterIntegration._repr_mimebundle_(self, include, exclude, **_)
     96 include = set(include) if include is not None else {self._jupyter_mimetype}
     97 include -= set(exclude or [])
---> 98 return {mimetype: getattr(self, method_name)()
     99         for mimetype, method_name in MIME_TYPES.items()
    100         if mimetype in include}

File /lib/python3.11/site-packages/graphviz/jupyter_integration.py:98, in <dictcomp>(.0)
     96 include = set(include) if include is not None else {self._jupyter_mimetype}
     97 include -= set(exclude or [])
---> 98 return {mimetype: getattr(self, method_name)()
     99         for mimetype, method_name in MIME_TYPES.items()
    100         if mimetype in include}

File /lib/python3.11/site-packages/graphviz/jupyter_integration.py:112, in JupyterIntegration._repr_image_svg_xml(self)
    110 def _repr_image_svg_xml(self) -> str:
    111     """Return the rendered graph as SVG string."""
--> 112     return self.pipe(format='svg', encoding=SVG_ENCODING)

File /lib/python3.11/site-packages/graphviz/piping.py:104, in Pipe.pipe(self, format, renderer, formatter, neato_no_op, quiet, engine, encoding)
     55 def pipe(self,
     56          format: typing.Optional[str] = None,
     57          renderer: typing.Optional[str] = None,
   (...)
     61          engine: typing.Optional[str] = None,
     62          encoding: typing.Optional[str] = None) -> typing.Union[bytes, str]:
     63     """Return the source piped through the Graphviz layout command.
     64 
     65     Args:
   (...)
    102         '<?xml version='
    103     """
--> 104     return self._pipe_legacy(format,
    105                              renderer=renderer,
    106                              formatter=formatter,
    107                              neato_no_op=neato_no_op,
    108                              quiet=quiet,
    109                              engine=engine,
    110                              encoding=encoding)

File /lib/python3.11/site-packages/graphviz/_tools.py:171, in deprecate_positional_args.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
    162     wanted = ', '.join(f'{name}={value!r}'
    163                        for name, value in deprecated.items())
    164     warnings.warn(f'The signature of {func.__name__} will be reduced'
    165                   f' to {supported_number} positional args'
    166                   f' {list(supported)}: pass {wanted}'
    167                   ' as keyword arg(s)',
    168                   stacklevel=stacklevel,
    169                   category=category)
--> 171 return func(*args, **kwargs)

File /lib/python3.11/site-packages/graphviz/piping.py:121, in Pipe._pipe_legacy(self, format, renderer, formatter, neato_no_op, quiet, engine, encoding)
    112 @_tools.deprecate_positional_args(supported_number=2)
    113 def _pipe_legacy(self,
    114                  format: typing.Optional[str] = None,
   (...)
    119                  engine: typing.Optional[str] = None,
    120                  encoding: typing.Optional[str] = None) -> typing.Union[bytes, str]:
--> 121     return self._pipe_future(format,
    122                              renderer=renderer,
    123                              formatter=formatter,
    124                              neato_no_op=neato_no_op,
    125                              quiet=quiet,
    126                              engine=engine,
    127                              encoding=encoding)

File /lib/python3.11/site-packages/graphviz/piping.py:149, in Pipe._pipe_future(self, format, renderer, formatter, neato_no_op, quiet, engine, encoding)
    146 if encoding is not None:
    147     if codecs.lookup(encoding) is codecs.lookup(self.encoding):
    148         # common case: both stdin and stdout need the same encoding
--> 149         return self._pipe_lines_string(*args, encoding=encoding, **kwargs)
    150     try:
    151         raw = self._pipe_lines(*args, input_encoding=self.encoding, **kwargs)

File /lib/python3.11/site-packages/graphviz/backend/piping.py:212, in pipe_lines_string(engine, format, input_lines, encoding, renderer, formatter, neato_no_op, quiet)
    206 cmd = dot_command.command(engine, format,
    207                           renderer=renderer,
    208                           formatter=formatter,
    209                           neato_no_op=neato_no_op)
    210 kwargs = {'input_lines': input_lines, 'encoding': encoding}
--> 212 proc = execute.run_check(cmd, capture_output=True, quiet=quiet, **kwargs)
    213 return proc.stdout

File /lib/python3.11/site-packages/graphviz/backend/execute.py:79, in run_check(cmd, input_lines, encoding, quiet, **kwargs)
     77     if kwargs.pop('capture_output'):
     78         kwargs['stdout'] = kwargs['stderr'] = subprocess.PIPE
---> 79     proc = _run_input_lines(cmd, input_lines, kwargs=kwargs)
     80 else:
     81     proc = subprocess.run(cmd, **kwargs)

File /lib/python3.11/site-packages/graphviz/backend/execute.py:99, in _run_input_lines(cmd, input_lines, kwargs)
     98 def _run_input_lines(cmd, input_lines, *, kwargs):
---> 99     popen = subprocess.Popen(cmd, stdin=subprocess.PIPE, **kwargs)
    101     stdin_write = popen.stdin.write
    102     for line in input_lines:

File /lib/python311.zip/subprocess.py:818, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group)
    816 """Create new Popen instance."""
    817 if not _can_fork_exec:
--> 818     raise OSError(
    819         errno.ENOTSUP, f"{sys.platform} does not support processes."
    820     )
    822 _cleanup()
    823 # Held while anything is calling waitpid before returncode has been
    824 # updated to prevent clobbering returncode if wait() or poll() are
    825 # called from multiple threads at once.  After acquiring the lock,
    826 # code must re-check self.returncode to see if another thread just
    827 # finished a waitpid() call.

OSError: [Errno 138] emscripten does not support processes.

Additional Context

I see Jupyterlite as the future for learning about python and its ecosystem of python packages. Its also a place where a lot of lighter data and analysis work will be done. That is why I believe its important for graphviz to at least seriously consider if this platform can be supported.

[xflr6]

Thanks for the detailed feature request and sorry for the delay.

Of course I agree that it would be great to make Graphviz rendering available on more platforms like the in-browser one you propose. However, I am afraid this Python wrapper might not be the best addresse for your request. This is a pretty simple wrapper library that helps to glue together DOT code from Python and send it off to the dot binary from upstream https://graphviz.org via stdlib's subprocess.run() (omitting some details). So the error you are getting AFAIU just spells out the fundamental restriction of the in-browser platform you are running on that it does not support creating subprocesses, which make sense, right?

popen = subprocess.Popen(cmd, stdin=subprocess.PIPE, **kwargs)
...
OSError: [Errno 138] emscripten does not support processes.

So IIUC, this will require a different way to call upstream https://graphviz.org to make it work on that platform. From https://graphviz.readthedocs.io/en/stable/#see-also, AFAICT, only https://pygraphviz.github.io/ uses a more sophisticated method to call upstream Graphviz that at some point might make it possible to distribute the library parts it depends on with it, so I think this might be the safest bet (another option might be to discuss your request also with upstream Graphviz developers: https://gitlab.com/graphviz/graphviz/).

I see that you have already opened pygraphviz/pygraphviz#453 in the sister-library so I am going to close this for now.