CsvItemExporter fails on py3

[elacuesta]

from scrapy.exporters import CsvItemExporter
with open('temp.csv', 'w') as f:
    exporter = CsvItemExporter(f)
    exporter.start_exporting()
    exporter.export_item({'a': 'b'})
    exporter.finish_exporting()

The previous snippet works fine on python 2, however the following error appears when running on python 3:

Traceback (most recent call last):
  File "exporter.py", line 5, in <module>
    exporter.export_item({'a': 'b'})
  File "/Users/eugenio/.../venv/lib/python3.5/site-packages/scrapy/exporters.py", line 206, in export_item
    self._write_headers_and_set_fields_to_export(item)
  File "/Users/eugenio/.../venv/lib/python3.5/site-packages/scrapy/exporters.py", line 230, in _write_headers_and_set_fields_to_export
    self.csv_writer.writerow(row)
TypeError: write() argument must be str, not bytes

Am I missing something? Thanks!

[TDominiqueWilliams]

It appears to be a problem with your "with open('temp.csv', 'w') as f:" statement. The mode you are using does not properly write to the file in the way you need it. When I tested the line myself, I added 'b' to the mode for the open statement. Try the following:

om scrapy.exporters import CsvItemExporter
with open('temp.csv', 'wb') as f:
exporter = CsvItemExporter(f)
exporter.start_exporting()
exporter.export_item({'a': 'b'})
exporter.finish_exporting()

I have pasted a helpful link below:
http://stackoverflow.com/questions/23051062/open-files-in-rt-and-wt-modes

[kmike]

I think this is working as intended; as @TDominiqueWilliams said, exporters expect files opened in binary mode.

[kmike]

Example at https://doc.scrapy.org/en/latest/topics/exporters.html shows that the file should be opened in binary mode. But we don't document explicitly that file arguments should be opened in binary mode; this is what can be improved.

[elacuesta]

Thanks, makes sense now 😄
I'll make a PR to mention this in the docs, if that's OK.

[kmike]

That'd be awesome!