matplotlib dependency cycle matplotlib <- ipython <- matplotlib - how to resolve?

[daym]

I'm trying to update the package matplotlib from 1.4.3 to 1.5.3 in the Guix distribution (I'm writing the package definition).

In the course of that I got into a reference cycle because matplotlib needs ipython. ipython needs matplotlib. How best to proceed?

[anntzer]

There's no dependency in either direction.

Let's ask pip:

$ pip show ipython matplotlib
---
Metadata-Version: 2.0
Name: ipython
Version: 5.1.0
Summary: IPython: Productive Interactive Computing
Home-page: http://ipython.org
Author: The IPython Development Team
Author-email: ipython-dev@scipy.org
Installer: pip
License: BSD
Location: /usr/lib/python3.5/site-packages
Requires: pickleshare, pygments, prompt-toolkit, decorator, traitlets, simplegeneric, setuptools, pexpect
Classifiers:
  Framework :: IPython
  Intended Audience :: Developers
  Intended Audience :: Science/Research
  License :: OSI Approved :: BSD License
  Programming Language :: Python
  Programming Language :: Python :: 2
  Programming Language :: Python :: 2.7
  Programming Language :: Python :: 3
  Topic :: System :: Shells
Entry-points:
  [console_scripts]
  iptest = IPython.testing.iptestcontroller:main
  iptest3 = IPython.testing.iptestcontroller:main
  ipython = IPython:start_ipython
  ipython3 = IPython:start_ipython
  [pygments.lexers]
  ipython = IPython.lib.lexers:IPythonLexer
  ipython3 = IPython.lib.lexers:IPython3Lexer
  ipythonconsole = IPython.lib.lexers:IPythonConsoleLexer
---
Metadata-Version: 2.0
Name: matplotlib
Version: 1.5.3
Summary: Python plotting package
Home-page: http://matplotlib.org
Author: John D. Hunter, Michael Droettboom
Author-email: matplotlib-users@python.org
Installer: pip
License: BSD
Location: /home/antony/.local/lib/python3.5/site-packages
Requires: cycler, pytz, pyparsing, numpy, python-dateutil
Classifiers:
  Development Status :: 5 - Production/Stable
  Intended Audience :: Science/Research
  License :: OSI Approved :: Python Software Foundation License
  Programming Language :: Python
  Programming Language :: Python :: 2
  Programming Language :: Python :: 3
  Topic :: Scientific/Engineering :: Visualization

Let's ask conda:

$ conda info matplotlib
<elided>
matplotlib 1.5.3 np111py35_0
----------------------------
file name   : matplotlib-1.5.3-np111py35_0.tar.bz2
name        : matplotlib
version     : 1.5.3
build number: 0
build string: np111py35_0
channel     : defaults
size        : 8.4 MB
date        : 2016-09-20
fn          : matplotlib-1.5.3-np111py35_0.tar.bz2
license     : PSF-based
license_family: PSF
md5         : 8158052c96728a8018496dd3105c8e99
priority    : 0
schannel    : defaults
url         : https://repo.continuum.io/pkgs/free/linux-64/matplotlib-1.5.3-np111py35_0.tar.bz2
dependencies:
    cycler
    freetype 2.5.*
    libpng 1.6.*
    numpy 1.11*
    pyparsing
    pyqt 5.*
    python 3.5*
    python-dateutil
    pytz

$ conda info ipython
<elided>
ipython 5.1.0 py35_0
--------------------
file name   : ipython-5.1.0-py35_0.tar.bz2
name        : ipython
version     : 5.1.0
build number: 0
build string: py35_0
channel     : defaults
size        : 991 KB
date        : 2016-08-14
fn          : ipython-5.1.0-py35_0.tar.bz2
license     : BSD
md5         : 292a238183d416b70fc41dfe1187d1b4
priority    : 0
schannel    : defaults
url         : https://repo.continuum.io/pkgs/free/linux-64/ipython-5.1.0-py35_0.tar.bz2
dependencies:
    decorator
    pexpect
    pickleshare
    prompt_toolkit >=1.0.3,<2.0.0
    pygments
    python 3.5*
    simplegeneric >0.8
    traitlets >=4.2

Closing, but feel free to continue the discussion.

[WeatherGod]

A possible source of confusion is that I think ipython is needed for building the docs?

[tacaswell]

I strongly suspect that this is an error in the guix packaging

[daym]

ipython is indeed needed for building the matplotlib docs - thus it is a (build-time) dependency. The build fails if I remove it.

@anntzer: Do I understand you correctly that you suggest we should remove the dependency "ipython requires matplotlib" ? Doesn't that break anything?

Edit: yes, it does:

Could not import extension matplotlib.sphinxext.mathmpl (exception: No module named matplotlib.sphinxext.mathmpl)
builder for `/gnu/store/hxdxicbqjp4cis7m1zgvcvz5vvvnvzaf-python2-ipython-3.2.1.drv' failed with exit code 1

Also, there's an extra cycle: python-ipython requires python-numpy requires python-matplotlib requires python-ipython

[daym]

ipython issue: ipython/ipython#10003

[anntzer]

I think the normal way to solve this is to have separate matplotlib-doc and ipython-doc packages, which both depend on the (docless) matplotlib and ipython packages.

Clearly it hasn't created any problems for conda (or for Arch Linux packaging, FWIW) to declare the (doc-less) packages as independent from each other.

[tacaswell]

Both ipython and mpl have conditional imports of the other in the main code base, but both can be used without the other.

Both seem to depend on the other for doc building. I suggest either making a seperate doc package as @anntzer suggests or to run the builds once without the docs and then once you have built binary packages, restore the docs and then bootstrap your self.

[daym]

Yeah, we'll do one of those. I'll talk to the others about it.

But do you agree that this is not an ideal situation? It would be better if they didn't depend on each other (and in the past, they didn't).

For example, what does the matplotlib doc builder do with ipython?

[anntzer]

It highlights IPython sessions, as in http://matplotlib.org/users/shell.html?highlight=ipython#ipython-to-the-rescue (the pygments syntax is provided by ipython).

Honestly I don't see why you'd want bundling the docs with the default build anyways...

[tacaswell]

Packaging the docs is reasonable, means that you get nice off-line docs, but the mpl docs are pretty big (size wise).

[takluyver]

I'm not sure that IPython actually uses mpl much for the docs build. Extensions may have been introduced before the big split and never cleaned up. I'd want to check more carefully before removing the extensions, but the only things I could find with a bit of grepping were a few uses of the htmlonly directive, which should be pretty trivial.

[takluyver]

As I suspected, the IPython doc build doesn't really need matplotlib; ipython/ipython#10022 removes the matplotlib extensions.