Skip to content

Commit

Permalink
in2csv accepts utf-8 arguments to the --sheet option in Python 2
Browse files Browse the repository at this point in the history
  • Loading branch information
James McKinney committed Jan 12, 2017
1 parent 943d7cb commit ad146cd
Show file tree
Hide file tree
Showing 5 changed files with 31 additions and 3 deletions.
10 changes: 8 additions & 2 deletions CHANGELOG.rst
@@ -1,10 +1,16 @@
1.0.2
-----

Improvements:

* :doc:`/scripts/csvjoin` supports :code:`--snifflimit` and :code:`--no-inference`.
* :doc:`/scripts/csvsql` restores support for :code:`--no-constraints` and :code:`--db-schema`.
* :doc:`/scripts/in2csv` now supports a :code:`--names` flag to print Excel sheet names.

Fixes:

* :doc:`/scripts/csvsql` restores support for :code:`--no-constraints` and :code:`--db-schema`.
* :doc:`/scripts/in2csv` accepts utf-8 arguments to the :code:`--sheet` option in Python 2.

1.0.1 - December 29, 2016
-------------------------

Expand All @@ -14,7 +20,7 @@ This is a minor release which fixes several bugs reported in the :code:`1.0.0` r
* :doc:`/scripts/csvlook` docs now note that output tables are Markdown-compatible. (#734)
* :doc:`/scripts/csvstat` now supports a :code:`--csv` flag for tabular output. (#584)
* :doc:`/scripts/csvstat` output is now easier to read. (#714)
* :doc:`/scripts/csvpy` now has a better description when using the :code:`--agate` flag (#729)
* :doc:`/scripts/csvpy` now has a better description when using the :code:`--agate` flag. (#729)
* Fix a Python 2.6 bug preventing :doc:`/scripts/csvjson` from parsing utf-8 files. (#732)
* Update required version of unittest to latest. (#727)

Expand Down
11 changes: 10 additions & 1 deletion csvkit/utilities/in2csv.py
@@ -1,9 +1,12 @@
#!/usr/bin/env python

import sys

import agate
import agatedbf # noqa
import agateexcel # noqa
import openpyxl
import six
import xlrd

from csvkit import convert
Expand All @@ -20,6 +23,12 @@ class In2CSV(CSVKitUtility):
override_flags = ['f']

def add_arguments(self):
def option_parser(bytestring):
if six.PY2:
return bytestring.decode(sys.getfilesystemencoding())
else:
return bytestring

self.argparser.add_argument(metavar="FILE", nargs='?', dest='input_path',
help='The CSV file to operate on. If omitted, will accept input on STDIN.')
self.argparser.add_argument('-f', '--format', dest='filetype',
Expand All @@ -30,7 +39,7 @@ def add_arguments(self):
help='Specifies a top-level key to use look within for a list of objects to be converted when processing JSON.')
self.argparser.add_argument('-n', '--names', dest='names_only', action='store_true',
help='Display sheet names from the input Excel file.')
self.argparser.add_argument('--sheet', dest='sheet',
self.argparser.add_argument('--sheet', dest='sheet', type=option_parser,
help='The name of the Excel sheet to operate on.')
self.argparser.add_argument('-y', '--snifflimit', dest='sniff_limit', type=int,
help='Limit CSV dialect sniffing to the specified number of bytes. Specify "0" to disable sniffing entirely.')
Expand Down
Binary file modified examples/sheets.xls
Binary file not shown.
Binary file modified examples/sheets.xlsx
Binary file not shown.
13 changes: 13 additions & 0 deletions tests/test_utilities/test_in2csv.py
@@ -1,4 +1,5 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-

import sys

Expand Down Expand Up @@ -51,12 +52,24 @@ def test_convert_xls(self):
def test_convert_xls_with_sheet(self):
self.assertConverted('xls', 'examples/sheets.xls', 'examples/testxls_converted.csv', ['--sheet', 'data'])

def test_convert_xls_with_unicode_sheet(self):
self.assertLines(['--sheet', 'ʤ', 'examples/sheets.xls'], [
'a,b,c',
'1.0,2.0,3.0',
])

def test_convert_xlsx(self):
self.assertConverted('xlsx', 'examples/test.xlsx', 'examples/testxlsx_converted.csv')

def test_convert_xlsx_with_sheet(self):
self.assertConverted('xlsx', 'examples/sheets.xlsx', 'examples/testxlsx_converted.csv', ['--sheet', 'data'])

def test_convert_xlsx_with_unicode_sheet(self):
self.assertLines(['--sheet', 'ʤ', 'examples/sheets.xlsx'], [
'a,b,c',
'True,2,3',
])

def test_csv_no_headers(self):
self.assertLines(['--no-header-row', 'examples/no_header_row.csv'], [
'a,b,c',
Expand Down

0 comments on commit ad146cd

Please sign in to comment.