ARROW-10617: [Python] Fix RecordBatchStreamReader iteration with Python 3.8

[sighingnow]

No description provided.

[github-actions]

https://issues.apache.org/jira/browse/ARROW-10617

[pitrou]

Thanks for reporting this. Could you add a test?

[sighingnow]

I do think a test is needed, but the case has already been tested in test_ipc.py (e.g., test_stream_simple_roundtrip) but seems it was skiped in the Python 3.8 case (in Github Action). I'm not sure how could I enable the test case, and the code I pasted in the Jira issue is taken from pyarrow's example, it should be easy to reproduce.

BTW, the appveyor failure is not caused by this PR.

[jorisvandenbossche]

Indeed, it seems the full test_ipc.py file is being skipped on the "Python 3.8 Without Pandas" build (https://github.com/apache/arrow/pull/8677/checks?check_run_id=1406477146). And this is because we don't have a 3.8 build with pandas, see this comment at the top of the file:

# TODO(wesm): The IPC tests depend a lot on pandas currently, so all excluded
# when it is not installed
pytestmark = pytest.mark.pandas

(so certainly a TODO we should resolve)

[jorisvandenbossche]

But don't we have a ursabot build with python 3.8? (they don't seem active / be triggered on this PR, though)

[sighingnow]

Hi @jorgecarleitao Then could we get this one merged first ? Or should I enable pandas with python 3.8 tests on github workflow?

[jorisvandenbossche]

Then could we get this one merged first ?

Ideally we would first ensure we have such a build, so we can know it is failing before applying this patch (so we know that the issue would actually be caught by our tests).

(now, I just mentioned on the issue that I can't reproduce the failure)

[pitrou]

I can't reproduce the failure either. @sighingnow, I suggest you investigate a bit more?

[pitrou]

Hmm, perhaps you're using a development version of Cython?

[pitrou]

In any case, the change looks sane, so I'm gonna merge if CI is ok.

[sighingnow]

Thank you, Pitrou!

[sighingnow]

Hmm, perhaps you're using a development version of Cython?

Thanks for you insight! I just realized that I'm working with Cython 0.3a6.