pip list |head raises BrokenPipeError

[bmaggard]

• Pip version: pip 9.0.1
• Python version: python 3.5
• Operating System: OS X 10.11.6 (15G1108)

Description:

pip list |tail
works like it should

pip list |head
... not so much

What I've run:

pip list |head

Output, for each package:

--- Logging error ---
Traceback (most recent call last):
File "/Users/.../anaconda/lib/python3.5/logging/init.py", line 984, in emit
self.flush()
File "/Users/.../anaconda/lib/python3.5/logging/init.py", line 964, in flush
self.stream.flush()
BrokenPipeError: [Errno 32] Broken pipe
Call stack:
File "/Users/.../anaconda/bin/pip", line 6, in
sys.exit(pip.main())
File "/Users/.../anaconda/lib/python3.5/site-packages/pip/init.py", line 249, in main
return command.main(cmd_args)
File "/Users/.../anaconda/lib/python3.5/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/Users/.../anaconda/lib/python3.5/site-packages/pip/commands/list.py", line 164, in run
self.output_package_listing(packages, options)
File "/Users/.../anaconda/lib/python3.5/site-packages/pip/commands/list.py", line 258, in output_package_listing
logger.info(self.output_legacy(dist))
Message: 'xlwt (1.1.2)'
Arguments: ()
Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'>
BrokenPipeError: [Errno 32] Broken pipe

[IanLee1521]

FWIW, I do not see this in pip 9.0.1, python 2.7.12, OS X 10.11.6 --

$ pip list |head
Package    Version
---------- --------
pip        9.0.1
setuptools 30.4.0
wheel      0.30.0a0

Note, I'm also not using anaconda.

[xavfernandez]

This behavior only appears with enough output (in my case, at least 11 lines).
I'm getting the same issue but it seems unrelated to pip.

import logging.config


logging.config.dictConfig({
    "version": 1,
    "handlers": {
        "console": {
            "level": logging.INFO,
            "class": "logging.StreamHandler",
            "stream": "ext://sys.stdout",
        },
    },
    "root": {
        "level": logging.INFO,
        "handlers": ["console"],
    },
})

logger = logging.getLogger(__name__)
for i in range(11):
    logger.info('Test %s', i)

yields the same logging error:

Test 0
Test 1
Test 2
Test 3
Test 4
Test 5
Test 6
Test 7
Test 8
Test 9
--- Logging error ---
Traceback (most recent call last):
  File "/usr/lib/python3.5/logging/__init__.py", line 984, in emit
    self.flush()
  File "/usr/lib/python3.5/logging/__init__.py", line 964, in flush
    self.stream.flush()
BrokenPipeError: [Errno 32] Broken pipe
Call stack:
  File "test_log_pip.py", line 21, in <module>
    logger.info('Test %s', i)
Message: 'Test %s'
Arguments: (10,)
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
BrokenPipeError: [Errno 32] Broken pipe

Note how it successfully work for the 10 first lines.
I'm getting the same error with python 3.5 and 3.4

[pradyunsg]

To quote from http://www.unixguide.net/unix/bash/E2.shtml

If a sequence of commands appears in a pipeline, and one of the
reading commands finishes before the writer has finished, the
writer receives a SIGPIPE signal.

As far as I can see, this also closes the pipe's stdout, so I'm not sure how to handle this.

[piotr-dobrogost]

For reference: How can I fix a Broken Pipe error? at SU.

[pradyunsg]

From the link @piotr-dobrogost posted, here's a workaround:

pip list | tail -n +1 | head -1

This "bug" is essentially how unix pipes work. pip can't do anything about this.

[pradyunsg]

Can we please close this? This is not a pip bug.

[bmaggard]

I'm not very educated on this topic, and I'm not clear on who you're asking to close it. My goal was to report it as a usability issue I was experiencing when using pip. Someone else marked it a 'bug'.

Isn't it pip's responsibility to handle the EPIPE/SIGPIPE gracefully? Or perhaps it's some underlying library's responsibility?

Do we at least agree that I should be seeing the first 10 lines of output, rather than a stack trace?

[ankostis]

Cause

As can bee seen by the stack-trace, this annoyance is caused by standard python's logging-handlers that are not guarding against BrokenPipeError exceptions (from the OP):

...
File "/Users/.../anaconda/lib/python3.5/logging/init.py", line 964, in flush
self.stream.flush()
BrokenPipeError: [Errno 32] Broken pipe

Prevention

It maybe worthy for pip to mitigate it in the code that sets up logging:

pip/src/pip/_internal/utils/logging.py

Line 67 in fc4ce3d

class ColorizedStreamHandler(logging.StreamHandler):

• either by overriding flush() in all XyzHandler classes declared there,
• or by monkey-patching the base logging.Streamhandler.flush() with:

   # Monkeypatch logging to behave when piped.
    old_flush = logging.StreamHandler.flush

    def flush(self):
        try:
            old_flush(self)
        except BrokenPipeError:
            pass  # is our stderr piped in `head` or `tail`?

    logging.StreamHandler.flush = flush

I can provide a patch if you agree (and decide which method to apply).