Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Work around missing subprocess members on Google App Engine #1825

Merged
merged 8 commits into from

6 participants

@mgiuca-google

On Google App Engine, the subprocess module exists, but is empty. This patch introduces a replacement module, subprocess_fixed, which provides a stub for subprocess.Popen which is compatible with existing usage.

I noticed that cbook already has a bit of a hack to work around the fact that subprocess.check_output is missing in Python 2.6, so I've moved that into my subprocess_fixed module as well, so now it has two reasons to exist.

Calls to subprocess_fixed.Popen on Google App Engine always raise OSError, which most of the codebase is already prepared for (since that's what happens when the application is missing). I've added a few missing catches of OSError.

Partial fix for Issue #1823.

@pelson pelson commented on the diff
lib/matplotlib/cbook.py
((6 lines not shown))
pid = os.getpid()
if sys.platform == 'sunos5':
- a2 = Popen('ps -p %d -o osz' % pid, shell=True,
- stdout=PIPE).stdout.readlines()
+ try:
+ a2 = Popen('ps -p %d -o osz' % pid, shell=True,
+ stdout=PIPE).stdout.readlines()
+ except OSError:
+ raise NotImplementedError(
@pelson Collaborator
pelson added a note

I'm tempted to suggest a RuntimeError here. None-the-less, this is an API change which should be documented in docs/api/api_changes.rst

Done. Note that I am having trouble building the docs locally, so I'm hoping my ReST syntax is correct!

Also, I agree RuntimeError is more appropriate (since NotImplementedError implies that it is planning to be fixed one day), but I changed this to be compatible with the Windows implementation. Happy to change it if you want. For what it's worth, NotImplementedError is a subclass of RuntimeError.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@pelson pelson commented on the diff
lib/matplotlib/cbook.py
((28 lines not shown))
- if 'stdout' in kwargs:
- raise ValueError('stdout argument not allowed, it will be overridden.')
- process = subprocess.Popen(stdout=subprocess.PIPE, *popenargs, **kwargs)
- output, unused_err = process.communicate()
- retcode = process.poll()
- if retcode:
- cmd = kwargs.get("args")
- if cmd is None:
- cmd = popenargs[0]
- raise subprocess.CalledProcessError(retcode, cmd, output=output)
- return output
-
-
-# python2.7's subprocess provides a check_output method
-if hasattr(subprocess, 'check_output'):
- check_output = subprocess.check_output
@pelson Collaborator
pelson added a note

This will also need to be documented in api_changes.

The fact that it has been removed from cbook? It was never documented in the first place.

If you think that people are depending on cbook.check_output, perhaps I should just put it back there as an alias (check_output = subprocess_fixed.check_output)?

@pelson Collaborator
pelson added a note

No, I'm happy for us to move it out - the subprocess compat module makes more sense than cbook. We just need to make sure any public (even if undocumented) API changes are put in the api_changes.rst file.

Done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
lib/matplotlib/subprocess_fixed.py
@@ -0,0 +1,76 @@
+"""
+A replacement wrapper around the subprocess module, with a number of
+work-arounds:
+- Provides the check_output function (which subprocess only provides from Python
+ 2.7 onwards).
+- Provides a stub implementation of subprocess members on Google App Engine
+ (which are missing in subprocess).
@pelson Collaborator
pelson added a note

I'd suggest an enhancement to core cpython might be desirable here too. Have you looked into doing that?

@pelson Collaborator
pelson added a note

(that would probably only apply to python3.4 onwards though)

@mdboom Owner
mdboom added a note

Is there a reason AppEngine has an empty subprocess module rather than one that provides exception-raising stubs?

@pelson: What enhancement are you suggesting for core cpython? The missing subprocess members is not a cpython problem, it's an App Engine problem. I don't know the rationale (why we don't supply stubs that raise OSError instead of them simply being missing), but this is fairly consistent across App Engine for modules that are unsupported. I've had a discussion with the Python team about changing this and it would be unlikely, because it could break existing users expectations (for example, they may be catching AttributeError). I think for now it would be easier to bring this into matplotlib, if it's not too much trouble. If App Engine ends up changing subprocess to include stubs, then it can be removed from Matplotlib.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@pelson pelson commented on the diff
lib/matplotlib/texmanager.py
@@ -63,8 +63,12 @@
def dvipng_hack_alpha():
- p = Popen('dvipng -version', shell=True, stdin=PIPE, stdout=PIPE,
- stderr=STDOUT, close_fds=(sys.platform != 'win32'))
+ try:
+ p = Popen('dvipng -version', stdin=PIPE, stdout=PIPE, stderr=STDOUT,
+ close_fds=(sys.platform != 'win32'))
+ except OSError:
+ mpl.verbose.report('No dvipng was found', 'helpful')
+ return False
@pelson Collaborator
pelson added a note

Is this a good idea? Previously an exception would have been raised, now False is returned. What impact does that have?

I'm happy to back off on this since it doesn't block texmanager from being imported, and it won't work anyway if dvipng is missing.

I don't think dvipng_hack_alpha is a command that should be used externally. It's used within the module to decide whether to change the output from dvipng. It looks like matplotlib detects the presence of dvipng on startup and if it isn't present, it disables the use of the tex module anyway. So this shouldn't really have an effect.

Did you want me to revert this, or document it, or just leave as-is?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
lib/matplotlib/dviread.py
@@ -23,9 +23,9 @@
import errno
import matplotlib
import matplotlib.cbook as mpl_cbook
+import matplotlib.subprocess_fixed as subprocess
@pelson Collaborator
pelson added a note

I'm not a huge fan of the name subprocess_fixed. I wonder if it would be worth having matplotlib.compatibility.subprocess.

@mdboom - what are your thoughts on this?

@mdboom Owner
mdboom added a note

Yes -- I like the idea of having a compat submodule containing stuff like this... we already have compatibility code scattered about, but I've seen the compat module approach in other projects and it's really nice. It makes it easy to clean it out later when the minimum requirements change etc.

@WeatherGod Collaborator

Moved matplotlib.subprocess_fixed to matplotlib.compat.subprocess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@pelson
Collaborator

Again, this is looking good @mgiuca-google - thanks for your hard work.

@pelson pelson was assigned
@mgiuca-google

Hmm ... looks like Travis failed on Python 3 with this error:

$ python ../matplotlib/tests.py
Traceback (most recent call last):
  File "../matplotlib/tests.py", line 8, in <module>
    import matplotlib
  File "/home/travis/virtualenv/python3.2/lib/python3.2/site-packages/matplotlib-1.3.x-py3.2-linux-x86_64.egg/matplotlib/__init__.py", line 136, in <module>
    from matplotlib.compat import subprocess
  File "/home/travis/virtualenv/python3.2/lib/python3.2/site-packages/matplotlib-1.3.x-py3.2-linux-x86_64.egg/matplotlib/compat/subprocess.py", line 18, in <module>
    from . import subprocess
ImportError: cannot import name subprocess

The command "python ../matplotlib/tests.py" exited with 1.

I'm not sure why my code import subprocess was magically changed to from . import subprocess -- can you think of what is doing this and a way around it? In any case, I think the problem is that the name subprocess is now overloaded. From within the compat package, it could refer to either my overriding version, or the official subprocess library. This doesn't sound like a very good idea, but then if we can't call it "subprocess" then we're back to requiring all imports of this module to use "as".

I think this can be worked around using __import__ magic. Does that sound like a good idea, or would you rather just rename the module?

@pelson
Collaborator

I'm not sure why my code import subprocess was magically changed to from . import subprocess -- can you think of what is doing this and a way around it

Python 2's default is to use relative imports before absolute ones, so import subprocess in python 2 is the same as from . import subprocess in python 3 (assuming you have a submodule called subprocess).

In order to get the behaviour you desire, I think you can just add the line from __future__ import absolute_import into your subprocess compatibility module.

It does look a little like 2to3 is being a little over zealous here...

mgiuca-google added some commits
@mgiuca-google mgiuca-google Added new module, matplotlib.subprocess_fixed, as a replacement for s…
…ubprocess.

cbook: Moved check_output to subprocess_fixed.
backend_pgf: Import check_output from subprocess_fixed instead of cbook.
0755aa6
@mgiuca-google mgiuca-google All modules now use matplotlib.subprocess_fixed instead of subprocess. 52c075f
@mgiuca-google mgiuca-google subprocess_fixed: Gracefully handle platforms without subprocess.Popen.
If subprocess.Popen is missing (for example, on Google App Engine), replaces it
with a dummy version that raises OSError. In these environments, calls to
subprocess will be handled properly as if the called app could not be found,
instead of raising AttributeError.
b29c93c
@mgiuca-google mgiuca-google texmanager: Ignore dvipng if it cannot be found, instead of raising O…
…SError.
a117e19
@mgiuca-google mgiuca-google cbook.report_memory: If ps not found, raises NotImplementedError, not…
… OSError.
ea342b6
@mgiuca-google mgiuca-google Moved matplotlib.subprocess_fixed to matplotlib.compat.subprocess. f41000d
@mgiuca-google mgiuca-google api_changes: Document the API changes.
- The subtle change to cbook.report_memory's exception types.
- Moving subprocess_fixed to compat.subprocess.
888fa20
@mgiuca-google mgiuca-google compat.subprocess: Turn on absolute importing.
This prevents the module from importing itself (as opposed to the global
subprocess module) on Python 3 after 2to3 conversion.
03cddc8
@mgiuca-google

OK thanks for the explanation. What I don't get is if Python 2 has relative imports by default, why does it work? (It only stopped working in Python 3.)

Oh well, I've added the absolute import so we'll see what Travis says. Also rebased.

@pelson pelson merged commit 8ec9555 into matplotlib:master

1 check passed

Details default The Travis build passed
@mgiuca-google mgiuca-google deleted the mgiuca-google:subprocess-google-app-engine branch
@dhomeier

This Popen call is broken with matplotlib.compat.subprocess, as its Popen (just like subprocess.Popen) does not accept a full command line but rather a list of [command, flags, args...].
So this needs to be changed to Popen('dvipng', '-version'], ...)

What do you mean "broken"? It worked for me last time I checked (from memory?), and as far as I know, it always has. From the Python docs (emphasis mine):

args should be a sequence of program arguments or else a single string. By default, the program to execute is the first item in args if args is a sequence. If args is a string, the interpretation is platform-dependent and described below.

It goes on to say that "it is recommended to pass args as a sequence", which I agree is good advice. But this shouldn't be broken.

It goes further on just into the next paragraph:

On Unix, if args is a string, the string is interpreted as the name or path of the program to execute. However, this can only be done if not passing arguments to the program.

The string is interpreted as a command line only with shell=True set, which is not the case here and also discouraged. Otherwise the Unix interpreter takes "dvipng -version" as the literal name of the executable, which of course does not exist:

matplotlib version 1.3.x
verbose.level helpful
interactive is True
platform is darwin
>>> p = matplotlib.compat.subprocess.Popen('ls')
>>> CHANGELOG       LICENSE         README.rst      agg24           distribute_setup.py lib         setup.cfg.template  src         ttconv
CONTRIBUTING.md     MANIFEST.in     TODO            boilerplate.py      distribute_setup.pyc    license.py      setup.py        tests.py        unit
CXX         Makefile        TODO_TESTS      build           doc         matplotlibrc.template   setupext.py     tools
INSTALL         README.osx      __pycache__     dist            examples        release         setupext.pyc        tox.ini

>>> p = matplotlib.compat.subprocess.Popen('ls -F')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/sw/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/sw/lib/python2.7/subprocess.py", line 1308, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
>>> p = matplotlib.compat.subprocess.Popen('ls -F'.split())
>>> CHANGELOG       LICENSE/        README.rst      agg24/          distribute_setup.py*    lib/            setup.cfg.template  src/            ttconv/
CONTRIBUTING.md     MANIFEST.in     TODO            boilerplate.py      distribute_setup.pyc    license.py      setup.py        tests.py*       unit/
CXX/            Makefile        TODO_TESTS      build/          doc/            matplotlibrc.template   setupext.py     tools/
INSTALL         README.osx      __pycache__/        dist/           examples/       release/        setupext.pyc        tox.ini

This is on MacOS X, but subprocess.Popen shows exactly the same behaviour on Debian 7, and according to the docs on every Unix system.

Cheers,

Derek

Owner

Confirmed. I've opened #1911 to track this.

Oh, I see, I removed shell=True in my patch. No idea why I did that or if it was an accident. Oops. I'll fix it (properly). Edit: Thanks for finding this, @dhomeier.

Collaborator

All fixed in master. Thanks everyone.

@daradib

Looks like 70f2296 added the line CalledProcessError = subprocess.CalledProcessError to compat/subprocess.py which causes an exception to raised since the attribute doesn't exist.

@mdboom
Owner

@daradib: If you're a Google App Engine user, would you mind writing up a PR that works around the issue?

@daradib daradib referenced this pull request from a commit
@daradib daradib Work around subprocess.CalledProcessError on Google App Engine
Set subprocess.CalledProcessError to None on restrictive environments
like Google App Engine because the exception cannot be raised and there
is no need to catch it. Builds upon #1825 which broke after #1857.
e99acd6
@daradib daradib referenced this pull request from a commit
@daradib daradib Work around subprocess.CalledProcessError on Google App Engine
Set subprocess.CalledProcessError to None on restrictive environments
like Google App Engine because the exception cannot be raised and there
is no need to catch it. Fixes #1825 which broke after #1857.
15d09a5
@daradib

@mdboom Ok, sure. I submitted #2860 to the maintenance branch because it's a tiny bugfix. If I should set it to master, let me know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 19, 2013
  1. @mgiuca-google

    Added new module, matplotlib.subprocess_fixed, as a replacement for s…

    mgiuca-google authored
    …ubprocess.
    
    cbook: Moved check_output to subprocess_fixed.
    backend_pgf: Import check_output from subprocess_fixed instead of cbook.
  2. @mgiuca-google
  3. @mgiuca-google

    subprocess_fixed: Gracefully handle platforms without subprocess.Popen.

    mgiuca-google authored
    If subprocess.Popen is missing (for example, on Google App Engine), replaces it
    with a dummy version that raises OSError. In these environments, calls to
    subprocess will be handled properly as if the called app could not be found,
    instead of raising AttributeError.
  4. @mgiuca-google
  5. @mgiuca-google
  6. @mgiuca-google
  7. @mgiuca-google

    api_changes: Document the API changes.

    mgiuca-google authored
    - The subtle change to cbook.report_memory's exception types.
    - Moving subprocess_fixed to compat.subprocess.
  8. @mgiuca-google

    compat.subprocess: Turn on absolute importing.

    mgiuca-google authored
    This prevents the module from importing itself (as opposed to the global
    subprocess module) on Python 3 after 2to3 conversion.
This page is out of date. Refresh to see the latest.
View
7 doc/api/api_changes.rst
@@ -32,6 +32,13 @@ Changes in 1.3.x
by ``self.vline`` for vertical cursors lines and ``self.hline`` is added
for the horizontal cursors lines.
+* On POSIX platforms, the :func:`~matplotlib.cbook.report_memory` function
+ raises :class:`NotImplementedError` instead of :class:`OSError` if the
+ :command:`ps` command cannot be run.
+
+* The :func:`~matplotlib.cbook.check_output` function has been moved to
+ `~matplotlib.compat.subprocess`.
+
Changes in 1.2.x
================
View
3  lib/matplotlib/__init__.py
@@ -124,7 +124,7 @@
'.'.join(str(x) for x in _required),
sys.version_info[0]))
-import os, re, shutil, subprocess, warnings
+import os, re, shutil, warnings
import distutils.sysconfig
import distutils.version
@@ -132,6 +132,7 @@
# definitions, so it is safe to import from it here.
from matplotlib.cbook import MatplotlibDeprecationWarning
from matplotlib.cbook import is_string_like
+from matplotlib.compat import subprocess
try:
reload
View
2  lib/matplotlib/animation.py
@@ -20,8 +20,8 @@
import sys
import itertools
import contextlib
-import subprocess
from matplotlib.cbook import iterable, is_string_like
+from matplotlib.compat import subprocess
from matplotlib import verbose
from matplotlib import rcParams
View
4 lib/matplotlib/backends/backend_pgf.py
@@ -7,7 +7,6 @@
import shutil
import tempfile
import codecs
-import subprocess
import atexit
import weakref
@@ -21,7 +20,8 @@
from matplotlib import font_manager
from matplotlib.ft2font import FT2Font
from matplotlib.cbook import is_string_like, is_writable_file_like
-from matplotlib.cbook import check_output
+from matplotlib.compat import subprocess
+from matplotlib.compat.subprocess import check_output
###############################################################################
View
2  lib/matplotlib/backends/backend_ps.py
@@ -88,7 +88,7 @@ def gs_version(self):
except KeyError:
pass
- from subprocess import Popen, PIPE
+ from matplotlib.compat.subprocess import Popen, PIPE
pipe = Popen(self.gs_exe + " --version",
shell=True, stdout=PIPE).stdout
if sys.version_info[0] >= 3:
View
72 lib/matplotlib/cbook.py
@@ -17,7 +17,6 @@
import locale
import os
import re
-import subprocess
import sys
import threading
import time
@@ -1212,19 +1211,34 @@ def restrict_dict(d, keys):
def report_memory(i=0): # argument may go away
'return the memory consumed by process'
- from subprocess import Popen, PIPE
+ from matplotlib.compat.subprocess import Popen, PIPE
pid = os.getpid()
if sys.platform == 'sunos5':
- a2 = Popen('ps -p %d -o osz' % pid, shell=True,
- stdout=PIPE).stdout.readlines()
+ try:
+ a2 = Popen('ps -p %d -o osz' % pid, shell=True,
+ stdout=PIPE).stdout.readlines()
+ except OSError:
+ raise NotImplementedError(
@pelson Collaborator
pelson added a note

I'm tempted to suggest a RuntimeError here. None-the-less, this is an API change which should be documented in docs/api/api_changes.rst

Done. Note that I am having trouble building the docs locally, so I'm hoping my ReST syntax is correct!

Also, I agree RuntimeError is more appropriate (since NotImplementedError implies that it is planning to be fixed one day), but I changed this to be compatible with the Windows implementation. Happy to change it if you want. For what it's worth, NotImplementedError is a subclass of RuntimeError.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ "report_memory works on Sun OS only if "
+ "the 'ps' program is found")
mem = int(a2[-1].strip())
elif sys.platform.startswith('linux'):
- a2 = Popen('ps -p %d -o rss,sz' % pid, shell=True,
- stdout=PIPE).stdout.readlines()
+ try:
+ a2 = Popen('ps -p %d -o rss,sz' % pid, shell=True,
+ stdout=PIPE).stdout.readlines()
+ except OSError:
+ raise NotImplementedError(
+ "report_memory works on Linux only if "
+ "the 'ps' program is found")
mem = int(a2[1].split()[1])
elif sys.platform.startswith('darwin'):
- a2 = Popen('ps -p %d -o rss,vsz' % pid, shell=True,
- stdout=PIPE).stdout.readlines()
+ try:
+ a2 = Popen('ps -p %d -o rss,vsz' % pid, shell=True,
+ stdout=PIPE).stdout.readlines()
+ except OSError:
+ raise NotImplementedError(
+ "report_memory works on Mac OS only if "
+ "the 'ps' program is found")
mem = int(a2[1].split()[0])
elif sys.platform.startswith('win'):
try:
@@ -1795,45 +1809,3 @@ def get_instancemethod(self):
else:
def _putmask(a, mask, values):
return np.copyto(a, values, where=mask)
-
-
-def _check_output(*popenargs, **kwargs):
- r"""Run command with arguments and return its output as a byte
- string.
-
- If the exit code was non-zero it raises a CalledProcessError. The
- CalledProcessError object will have the return code in the
- returncode
- attribute and output in the output attribute.
-
- The arguments are the same as for the Popen constructor. Example::
-
- >>> check_output(["ls", "-l", "/dev/null"])
- 'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
-
- The stdout argument is not allowed as it is used internally.
- To capture standard error in the result, use stderr=STDOUT.::
-
- >>> check_output(["/bin/sh", "-c",
- ... "ls -l non_existent_file ; exit 0"],
- ... stderr=STDOUT)
- 'ls: non_existent_file: No such file or directory\n'
- """
- if 'stdout' in kwargs:
- raise ValueError('stdout argument not allowed, it will be overridden.')
- process = subprocess.Popen(stdout=subprocess.PIPE, *popenargs, **kwargs)
- output, unused_err = process.communicate()
- retcode = process.poll()
- if retcode:
- cmd = kwargs.get("args")
- if cmd is None:
- cmd = popenargs[0]
- raise subprocess.CalledProcessError(retcode, cmd, output=output)
- return output
-
-
-# python2.7's subprocess provides a check_output method
-if hasattr(subprocess, 'check_output'):
- check_output = subprocess.check_output
@pelson Collaborator
pelson added a note

This will also need to be documented in api_changes.

The fact that it has been removed from cbook? It was never documented in the first place.

If you think that people are depending on cbook.check_output, perhaps I should just put it back there as an alias (check_output = subprocess_fixed.check_output)?

@pelson Collaborator
pelson added a note

No, I'm happy for us to move it out - the subprocess compat module makes more sense than cbook. We just need to make sure any public (even if undocumented) API changes are put in the api_changes.rst file.

Done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
-else:
- check_output = _check_output
View
0  lib/matplotlib/compat/__init__.py
No changes.
View
77 lib/matplotlib/compat/subprocess.py
@@ -0,0 +1,77 @@
+"""
+A replacement wrapper around the subprocess module, with a number of
+work-arounds:
+- Provides the check_output function (which subprocess only provides from Python
+ 2.7 onwards).
+- Provides a stub implementation of subprocess members on Google App Engine
+ (which are missing in subprocess).
+
+Instead of importing subprocess, other modules should use this as follows:
+
+from matplotlib.compat import subprocess
+
+This module is safe to import from anywhere within matplotlib.
+"""
+
+from __future__ import absolute_import # Required to import subprocess
+from __future__ import print_function
+
+import subprocess
+
+__all__ = ['Popen', 'PIPE', 'STDOUT', 'check_output']
+
+
+if hasattr(subprocess, 'Popen'):
+ Popen = subprocess.Popen
+ # Assume that it also has the other constants.
+ PIPE = subprocess.PIPE
+ STDOUT = subprocess.STDOUT
+else:
+ # In restricted environments (such as Google App Engine), these are
+ # non-existent. Replace them with dummy versions that always raise OSError.
+ def Popen(*args, **kwargs):
+ raise OSError("subprocess.Popen is not supported")
+ PIPE = -1
+ STDOUT = -2
+
+
+def _check_output(*popenargs, **kwargs):
+ r"""Run command with arguments and return its output as a byte
+ string.
+
+ If the exit code was non-zero it raises a CalledProcessError. The
+ CalledProcessError object will have the return code in the
+ returncode
+ attribute and output in the output attribute.
+
+ The arguments are the same as for the Popen constructor. Example::
+
+ >>> check_output(["ls", "-l", "/dev/null"])
+ 'crw-rw-rw- 1 root root 1, 3 Oct 18 2007 /dev/null\n'
+
+ The stdout argument is not allowed as it is used internally.
+ To capture standard error in the result, use stderr=STDOUT.::
+
+ >>> check_output(["/bin/sh", "-c",
+ ... "ls -l non_existent_file ; exit 0"],
+ ... stderr=STDOUT)
+ 'ls: non_existent_file: No such file or directory\n'
+ """
+ if 'stdout' in kwargs:
+ raise ValueError('stdout argument not allowed, it will be overridden.')
+ process = Popen(stdout=PIPE, *popenargs, **kwargs)
+ output, unused_err = process.communicate()
+ retcode = process.poll()
+ if retcode:
+ cmd = kwargs.get("args")
+ if cmd is None:
+ cmd = popenargs[0]
+ raise subprocess.CalledProcessError(retcode, cmd, output=output)
+ return output
+
+
+# python2.7's subprocess provides a check_output method
+if hasattr(subprocess, 'check_output'):
+ check_output = subprocess.check_output
+else:
+ check_output = _check_output
View
2  lib/matplotlib/dviread.py
@@ -23,9 +23,9 @@
import errno
import matplotlib
import matplotlib.cbook as mpl_cbook
+from matplotlib.compat import subprocess
import numpy as np
import struct
-import subprocess
import sys
if sys.version_info[0] >= 3:
View
3  lib/matplotlib/font_manager.py
@@ -43,7 +43,7 @@
see license/LICENSE_TTFQUERY.
"""
-import os, sys, subprocess, warnings
+import os, sys, warnings
try:
set
except NameError:
@@ -54,6 +54,7 @@
from matplotlib import rcParams, get_configdir
from matplotlib.cbook import is_string_like
import matplotlib.cbook as cbook
+from matplotlib.compat import subprocess
from matplotlib.fontconfig_pattern import \
parse_fontconfig_pattern, generate_fontconfig_pattern
View
2  lib/matplotlib/testing/compare.py
@@ -7,6 +7,7 @@
from __future__ import division
import matplotlib
+from matplotlib.compat import subprocess
from matplotlib.testing.noseclasses import ImageComparisonFailure
from matplotlib.testing import image_util, util
from matplotlib import _png
@@ -17,7 +18,6 @@
import os
import numpy as np
import shutil
-import subprocess
import sys
from functools import reduce
View
2  lib/matplotlib/testing/util.py
@@ -1,5 +1,5 @@
-import subprocess
import sys
+from matplotlib.compat import subprocess
class MiniExpect:
View
2  lib/matplotlib/tests/test_backend_pgf.py
@@ -2,12 +2,12 @@
import os
import shutil
-import subprocess
import numpy as np
import nose
from nose.plugins.skip import SkipTest
import matplotlib as mpl
import matplotlib.pyplot as plt
+from matplotlib.compat import subprocess
from matplotlib.testing.compare import compare_images, ImageComparisonFailure
from matplotlib.testing.decorators import _image_directories
View
12 lib/matplotlib/texmanager.py
@@ -41,7 +41,6 @@
import os
import shutil
import sys
-from subprocess import Popen, PIPE, STDOUT
from hashlib import md5
@@ -51,6 +50,7 @@
from matplotlib import rcParams
from matplotlib._png import read_png
from matplotlib.cbook import mkdirs
+from matplotlib.compat.subprocess import Popen, PIPE, STDOUT
import matplotlib.dviread as dviread
import re
@@ -63,8 +63,12 @@
def dvipng_hack_alpha():
- p = Popen('dvipng -version', shell=True, stdin=PIPE, stdout=PIPE,
- stderr=STDOUT, close_fds=(sys.platform != 'win32'))
+ try:
+ p = Popen('dvipng -version', stdin=PIPE, stdout=PIPE, stderr=STDOUT,
+ close_fds=(sys.platform != 'win32'))
+ except OSError:
+ mpl.verbose.report('No dvipng was found', 'helpful')
+ return False
@pelson Collaborator
pelson added a note

Is this a good idea? Previously an exception would have been raised, now False is returned. What impact does that have?

I'm happy to back off on this since it doesn't block texmanager from being imported, and it won't work anyway if dvipng is missing.

I don't think dvipng_hack_alpha is a command that should be used externally. It's used within the module to decide whether to change the output from dvipng. It looks like matplotlib detects the presence of dvipng on startup and if it isn't present, it disables the use of the tex module anyway. So this shouldn't really have an effect.

Did you want me to revert this, or document it, or just leave as-is?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
stdin, stdout = p.stdin, p.stdout
for line in stdout:
if line.startswith(b'dvipng '):
@@ -74,7 +78,7 @@ def dvipng_hack_alpha():
version = version.decode('ascii')
version = distutils.version.LooseVersion(version)
return version < distutils.version.LooseVersion('1.6')
- mpl.verbose.report('No dvipng was found', 'helpful')
+ mpl.verbose.report('Unexpected response from dvipng -version', 'helpful')
return False
View
1  setupext.py
@@ -494,6 +494,7 @@ def get_packages(self):
'matplotlib',
'matplotlib.backends',
'matplotlib.backends.qt4_editor',
+ 'matplotlib.compat',
'matplotlib.projections',
'matplotlib.sphinxext',
'matplotlib.testing',
Something went wrong with that request. Please try again.