TypeError in test collection when sys.stdin isn't an iterator

[jaraco]

In this build, I found that because of the construction of a csv.DictReader(sys.stdin) during import, the file can't be collected.

Looking a --pdb trace on it, I see:

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
jaraco/financial/lifo.py:82: in <module>
    input: compose(csv.DictReader, open)=csv.DictReader(sys.stdin),
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/csv.py:87: in __init__
    self.reader = reader(f, dialect, *args, **kwds)
E   TypeError: argument 1 must be an iterator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/csv.py(87)__init__()
-> self.reader = reader(f, dialect, *args, **kwds)
(Pdb) f
<_pytest.capture.DontReadFromInput object at 0x1035952e8>

Obviously, I could refactor the code to bypass the error, but since the code runs, I feel it should be supported for pytest collection.

Perhaps DontReadFromInput could be adapted to be an iterator to satisfy csv.DictReader.

The issue can be minimally replicated with

$ cat > test.py
import sys
import csv
csv.DictReader(sys.stdin)
$ rwt -q pytest -- -m pytest test.py
======================================================== test session starts =========================================================
platform darwin -- Python 3.7.0b2, pytest-3.4.2, py-1.5.2, pluggy-0.6.0
rootdir: /Users/jaraco/clean, inifile:
collected 0 items / 1 errors

=============================================================== ERRORS ===============================================================
______________________________________________________ ERROR collecting test.py ______________________________________________________
test.py:3: in <module>
    csv.DictReader(sys.stdin)
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/csv.py:87: in __init__
    self.reader = reader(f, dialect, *args, **kwds)
E   TypeError: argument 1 must be an iterator
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
====================================================== 1 error in 0.09 seconds =======================================================

[pytestbot]

GitMate.io thinks possibly related issues are #3251 (Warnings not captured during test collection), #2532 (TypeError on conftest.py during collection), #33 (TypeError when module under test or function under test is named 'test'), #1976 (please don't suppress tracebacks on ImportError during test collection), and #149 (Distributed testing silently ignores collection errors).

[jaraco]

I thought maybe adding __next__ to DontReadFromInput would make it an iterator:

$ git diff
diff --git a/_pytest/capture.py b/_pytest/capture.py
index 36658acc..8d7ba6e7 100644
--- a/_pytest/capture.py
+++ b/_pytest/capture.py
@@ -566,6 +566,7 @@ class DontReadFromInput(object):
     readline = read
     readlines = read
     __iter__ = read
+    __next__ = read

     def fileno(self):
         raise UnsupportedOperation("redirected stdin is pseudofile, "

but it didn't:

jaraco/financial/lifo.py:82: in <module>
    input: compose(csv.DictReader, open)=csv.DictReader(sys.stdin),
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/csv.py:87: in __init__
    self.reader = reader(f, dialect, *args, **kwds)
E   TypeError: argument 1 must be an iterator
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/csv.py(87)__init__()
-> self.reader = reader(f, dialect, *args, **kwds)
(Pdb) f
<_pytest.capture.DontReadFromInput object at 0x11013e390>
(Pdb) f.__next__
<bound method DontReadFromInput.read of <_pytest.capture.DontReadFromInput object at 0x11013e390>>

[jaraco]

Here's the code where csv.reader detects an iterator. It uses PyObject_GetIter, which according to the docs is equivalent to __iter__.

Aha! and f.__iter__() raises OSError. I think I know the fix.

[jaraco]

This patch fixes the failure:

$ git diff
diff --git a/_pytest/capture.py b/_pytest/capture.py
index 6e213944..6545e6fa 100644
--- a/_pytest/capture.py
+++ b/_pytest/capture.py
@@ -574,7 +574,10 @@ class DontReadFromInput(object):
         raise IOError("reading from stdin while output is captured")
     readline = read
     readlines = read
-    __iter__ = read
+    __next__ = read
+
+    def __iter__(self):
+        return self

     def fileno(self):
         raise UnsupportedOperation("redirected stdin is pseudofile, "

It does it by making DontReadFromInput an iterable and only raising the OSError when something attempts to consume the iterator.