Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Formatting Exception in io/formats/terminal.py #25080

Closed
MTKnife opened this issue Feb 1, 2019 · 4 comments

Comments

@MTKnife
Copy link

commented Feb 1, 2019

The Code

from pandas import DataFrame
df = DataFrame([[1,2,3], [4,5,6]])
print df

What I get when I try to print "df":

Traceback (most recent call last):

  File "C:\Anaconda2\lib\site-packages\IPython\core\formatters.py", line 699, in __call__
    printer.pretty(obj)

  File "C:\Anaconda2\lib\site-packages\IPython\lib\pretty.py", line 403, in pretty
    return _repr_pprint(obj, self, cycle)

  File "C:\Anaconda2\lib\site-packages\IPython\lib\pretty.py", line 703, in _repr_pprint
    output = repr(obj)

  File "C:\Anaconda2\lib\site-packages\pandas\core\base.py", line 78, in __repr__
    return str(self)

  File "C:\Anaconda2\lib\site-packages\pandas\core\base.py", line 58, in __str__
    return self.__bytes__()

  File "C:\Anaconda2\lib\site-packages\pandas\core\base.py", line 70, in __bytes__
    return self.__unicode__().encode(encoding, 'replace')

  File "C:\Anaconda2\lib\site-packages\pandas\core\frame.py", line 633, in __unicode__
    line_width=width, show_dimensions=show_dimensions)

  File "C:\Anaconda2\lib\site-packages\pandas\core\frame.py", line 713, in to_string
    line_width=line_width)

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\format.py", line 428, in __init__
    self._chk_truncate()

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\format.py", line 444, in _chk_truncate
    (w, h) = get_terminal_size()

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\terminal.py", line 41, in get_terminal_size
    tuple_xy = _get_terminal_size_tput()

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\terminal.py", line 103, in _get_terminal_size_tput
    cols = int(output[0])

ValueError: invalid literal for int() with base 10: ''

Traceback (most recent call last):

  File "C:\Anaconda2\lib\site-packages\IPython\core\formatters.py", line 338, in __call__
    return method()

  File "C:\Anaconda2\lib\site-packages\pandas\core\frame.py", line 667, in _repr_html_
    show_dimensions=show_dimensions, notebook=True)

  File "C:\Anaconda2\lib\site-packages\pandas\core\frame.py", line 2253, in to_html
    render_links=render_links)

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\format.py", line 428, in __init__
    self._chk_truncate()

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\format.py", line 444, in _chk_truncate
    (w, h) = get_terminal_size()

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\terminal.py", line 41, in get_terminal_size
    tuple_xy = _get_terminal_size_tput()

  File "C:\Anaconda2\lib\site-packages\pandas\io\formats\terminal.py", line 103, in _get_terminal_size_tput
    cols = int(output[0])

ValueError: invalid literal for int() with base 10: ''

Problem description

I get this exception when running the code quoted above in the iPython console in the latest version of Anaconda Spyder (3.3.2); the Python version is Anaconda 2.7.15.

Apparently, the ValueError has existed all along, but it was previously masked by the try/except block that wraps line 103 in terminal.py.

The function in 0.23 looked like this:

def _get_terminal_size_tput():
    # get terminal width
    # src: http://stackoverflow.com/questions/263890/how-do-i-find-the-width
    # -height-of-a-terminal-window
    try:
        import subprocess
        proc = subprocess.Popen(["tput", "cols"],
                                stdin=subprocess.PIPE,
                                stdout=subprocess.PIPE)
        output = proc.communicate(input=None)
        cols = int(output[0])
        proc = subprocess.Popen(["tput", "lines"],
                                stdin=subprocess.PIPE,
                                stdout=subprocess.PIPE)
        output = proc.communicate(input=None)
        rows = int(output[0])
        return (cols, rows)
    except:
return None

In 0.24, it looks like this:

def _get_terminal_size_tput():
    # get terminal width
    # src: http://stackoverflow.com/questions/263890/how-do-i-find-the-width
    # -height-of-a-terminal-window
    try:
        import subprocess
        proc = subprocess.Popen(["tput", "cols"],
                                stdin=subprocess.PIPE,
                                stdout=subprocess.PIPE)
        output = proc.communicate(input=None)
        cols = int(output[0])
        proc = subprocess.Popen(["tput", "lines"],
                                stdin=subprocess.PIPE,
                                stdout=subprocess.PIPE)
        output = proc.communicate(input=None)
        rows = int(output[0])
        return (cols, rows)
    except OSError:
        return None

The change here is in the "except" line: it's gone from a blanket "except" (which is, admittedly, bad practice), to an "OSError". I believe the "except" needs to look like this:

    except (OSError, ValueError):

Yes, the underlying issue will still be there (it always has been), but nonetheless it displays just fine in my Spyder console window. I'd submit a PR myself, but I'm not confident I know enough about the pandas codebase that my fix is what's needed.

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.15.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: en
LOCALE: None.None

pandas: 0.24.0
pytest: 4.1.1
pip: 18.1
setuptools: 40.6.3
Cython: 0.29.2
numpy: 1.15.4
scipy: 1.2.0
pyarrow: None
xarray: None
IPython: 5.8.0
sphinx: 1.8.2
patsy: 0.5.1
dateutil: 2.7.5
pytz: 2018.9
blosc: None
bottleneck: 1.2.1
tables: 3.4.4
numexpr: 2.6.9
feather: None
matplotlib: 2.2.3
openpyxl: 2.5.12
xlrd: 1.2.0
xlwt: 1.3.0
xlsxwriter: 1.1.2
lxml.etree: 4.3.0
bs4: 4.7.1
html5lib: 1.0.1
sqlalchemy: 1.2.16
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

commented Feb 1, 2019

Rather than catching OSError, ValueError I think we should

  1. Do the subprocess stuff in a try / except OSError
  2. Do the parsing (cols = int(output[0]), etc.) in a try / except ValueError
@EternalLearner42

This comment has been minimized.

Copy link
Contributor

commented Feb 2, 2019

Seems interesting. I'm a bit of a newbie to open source, but I wouldn't mind taking a look at it. Do I have your go-ahead?

@EternalLearner42 EternalLearner42 referenced this issue Feb 2, 2019
2 of 4 tasks complete

@jreback jreback modified the milestones: Contributions Welcome, 0.24.2 Feb 2, 2019

EternalLearner42 added a commit to EternalLearner42/pandas that referenced this issue Feb 2, 2019

EternalLearner42 added a commit to EternalLearner42/pandas that referenced this issue Feb 3, 2019

@paplio

This comment has been minimized.

Copy link

commented Feb 5, 2019

Hi! I'm new to open-source as well but I think I could handle this well. Saw the issue was still open, and was wondering if you would accept a PR?

@gfyoung

This comment has been minimized.

Copy link
Member

commented Feb 5, 2019

@paplio : Thanks for your interest! I think we already have a PR in the works to address this (see #25088). That being said, you are more than welcome to look at any of our other open issues (those tagged with "good first issue" are definitely good ones to start with)!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.