Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

csv module: add header row to DictWriter #43803

Closed
edabraham mannequin opened this issue Aug 9, 2006 · 17 comments
Closed

csv module: add header row to DictWriter #43803

edabraham mannequin opened this issue Aug 9, 2006 · 17 comments
Assignees
Labels
type-feature A feature request or enhancement

Comments

@edabraham
Copy link
Mannequin

edabraham mannequin commented Aug 9, 2006

BPO 1537721
Nosy @smontanaro, @pitrou, @djc, @florentx, @kynan

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = 'https://github.com/djc'
closed_at = <Date 2010-03-05.09:13:34.511>
created_at = <Date 2006-08-09.22:20:01.000>
labels = ['type-feature']
title = 'csv module: add header row to DictWriter'
updated_at = <Date 2010-08-29.14:14:47.795>
user = 'https://bugs.python.org/edabraham'

bugs.python.org fields:

activity = <Date 2010-08-29.14:14:47.795>
actor = 'kynan'
assignee = 'djc'
closed = True
closed_date = <Date 2010-03-05.09:13:34.511>
closer = 'djc'
components = ['None']
creation = <Date 2006-08-09.22:20:01.000>
creator = 'ed_abraham'
dependencies = []
files = []
hgrepos = []
issue_num = 1537721
keywords = ['patch', 'buildbot']
message_count = 17.0
messages = ['61251', '84796', '86157', '96063', '96069', '96074', '96078', '96089', '96091', '96102', '96103', '96104', '99952', '100404', '100407', '100409', '100455']
nosy_count = 7.0
nosy_names = ['skip.montanaro', 'pitrou', 'ed_abraham', 'djc', 'ivo', 'flox', 'kynan']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'patch review'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue1537721'
versions = ['Python 2.7', 'Python 3.2']

@edabraham
Copy link
Mannequin Author

edabraham mannequin commented Aug 9, 2006

I use the DictWriter class from the csv module, and
have to manually write the header row. A mindless chore
which I would like to see eliminated. Can we have a
writeheader method added to the class? Something like
the following:

def writeheader(self, headernames = {}):
    """Write a header row"""
    if not headernames:
        headernames = dict(zip(self.fieldnames,
self.fieldnames))
        self.writerow(headernames)

This would let you either use the fieldnames directly,
or supply your own pretty header names.

Would be nice to have another keyword argument to
DictWriter, 'header = False'. If header was true, then
the __init__ method could call writeheader().

At the moment I have to write things like

fields = ['a','b','c']
w = csv.DictWriter(fid, fields)
w.writerow(dict(zip(fields, fields)))
for row in rows:
    w.writerow(row)

The proposed changes would let me write the simpler

w = csv.DictWriter(fid, ['a','b','c'], header = True)
for row in rows:
    w.writerow(row)

A problem is that including a new keyword argument
would break code which used position to fill the
keyword arguments and to supply arguments through *args
to the writer class.

@edabraham edabraham mannequin added type-feature A feature request or enhancement labels Aug 9, 2006
@smontanaro
Copy link
Contributor

smontanaro commented Mar 31, 2009

I don't see a patch. Is there some reason that if you need this
you can't simply subclass DictWriter?

@ivo
Copy link
Mannequin

ivo mannequin commented Apr 19, 2009

Skip, you were arguing in another csv issue on a NamedTupleReader that
the Reader and Writer should work in concert together.

Certainly, making this default functionality for DictWriter would
definitely make it work more in concert with DictReader.

A sample process of reading and writing might be:

  1. DictReader reads in header names, reads in data
  2. Pass in header names to DictWriter on init
  3. DictWriter writes out modified data back to csv file (headers get
    written automatically or with a method call)
  4. DictReader can now reader in the csv file automatically again with
    header names

My feeling is, if DictReader can read in head names, why can't
DictWriter write them back out again? Shouldn't there be a good amount
of symmatry to their function/abilities?

My feeling of how to implement this functionality would be to include a
new init argument, say 'writeheader'. It's default would be True if you
wanted to implement this new feature by default, or False if you wanted
to keep it an option so that older scripts relying on the old
functionality won't break immmediately.
DictWriter would then write the header on the first call to writerow(s),
or perhaps also with an explicit call to writeheader() say.

I certainly am miffed by the fact that DictWriter cannot produce
(normally) a csv file that DictReader can read in automatically.

@djc
Copy link
Member

djc commented Dec 7, 2009

I'd like to commit this, but it would be nice to get a review first:

Index: Lib/csv.py
===================================================================

--- Lib/csv.py	(revision 76697)
+++ Lib/csv.py	(working copy)
@@ -132,6 +132,10 @@
         self.extrasaction = extrasaction
         self.writer = writer(f, dialect, *args, **kwds)
 
+    def writeheader(self):
+        header = dict(zip(self.fieldnames, self.fieldnames))
+        self.writerow(header)
+
     def _dict_to_list(self, rowdict):
         if self.extrasaction == "raise":
             wrong_fields = [k for k in rowdict if k not in self.fieldnames]
Index: Lib/test/test_csv.py
===================================================================
--- Lib/test/test_csv.py	(revision 76697)
+++ Lib/test/test_csv.py	(working copy)
@@ -598,8 +598,10 @@
         fileobj = os.fdopen(fd, "w+b")
         try:
             writer = csv.DictWriter(fileobj, fieldnames = ["f1", "f2",
"f3"])
+            writer.writeheader()
             writer.writerow({"f1": 10, "f3": "abc"})
             fileobj.seek(0)
+            self.assertEqual(fileobj.readline(), "f1,f2,f3\r\n")
             self.assertEqual(fileobj.read(), "10,,abc\r\n")
         finally:
             fileobj.close()

(I think I have commit privileges already.)

@smontanaro
Copy link
Contributor

smontanaro commented Dec 7, 2009

I'm sorry, but I don't have time to look at this right now. On the one
hand, one person asks for more symmetry. Someone else wants to add a
writeheader method. If you want symmetry shouldn't the DictWriter
simply write the header without being asked? I'm confused by the
various options and feel that someone is going to be disappointed by any
solution. For the time being at least I would prefer that the status
quo remain in place.

S

@djc
Copy link
Member

djc commented Dec 7, 2009

Skip, I agree that it's hard to decide if we should have the class write
the header on __init__(). I figured starting off with a method to make
doing it "manually" is a good start; people can start using that, and if
it's deemed useful we can always add the auto-write later.

@pitrou
Copy link
Member

pitrou commented Dec 7, 2009

We can't change default behaviour because it will break compatibility,
so an additional method looks ok to me.

@smontanaro
Copy link
Contributor

smontanaro commented Dec 7, 2009

Antoine> We can't change default behaviour because it will break
Antoine> compatibility, so an additional method looks ok to me.

Why can't default behavior be changed?

S

@pitrou
Copy link
Member

pitrou commented Dec 7, 2009

Le lundi 07 décembre 2009 à 23:42 +0000, Skip Montanaro a écrit :

Skip Montanaro <skip@pobox.com> added the comment:

Antoine> We can't change default behaviour because it will break
Antoine> compatibility, so an additional method looks ok to me.

Why can't default behavior be changed?

Well, because it will break assumptions about the generated documents?

@smontanaro
Copy link
Contributor

smontanaro commented Dec 8, 2009

Antoine> We can't change default behaviour because it will break
Antoine> compatibility, so an additional method looks ok to me.

>> Why can't default behavior be changed?

Antoine> Well, because it will break assumptions about the generated documents?

Isn't the alpha period (2.7 and 3.2 in this case) precisely when an API can
change?

Skip

@pitrou
Copy link
Member

pitrou commented Dec 8, 2009

Isn't the alpha period (2.7 and 3.2 in this case) precisely when an API can
change?

Well, it can, but only if there are compelling reasons to do so. It
should be the exception rather than the rule.
The reasons here seem far from compelling, and moreover we can't detect
whether the user is expecting the new or the old behaviour.

@smontanaro
Copy link
Contributor

smontanaro commented Dec 8, 2009

> Isn't the alpha period (2.7 and 3.2 in this case) precisely when an
>> API can change?

Antoine> Well, it can, but only if there are compelling reasons to do
Antoine> so. It should be the exception rather than the rule.  The
Antoine> reasons here seem far from compelling, and moreover we can't
Antoine> detect whether the user is expecting the new or the old
Antoine> behaviour.

Fine. Dirkjan, assuming there are the necessary test cases and
documentation changes, feel free to check in your patch and close the
ticket.

Skip

@djc
Copy link
Member

djc commented Feb 23, 2010

Fixed in SVN, r78384.

@djc djc closed this as completed Feb 23, 2010
@djc djc self-assigned this Feb 23, 2010
@djc djc closed this as completed Feb 23, 2010
@djc djc self-assigned this Feb 23, 2010
@florentx
Copy link
Mannequin

florentx mannequin commented Mar 4, 2010

according to the buidbots, it hurts some platforms: Windows XP, Windows 7 and sparc Solaris10

  • sparc solaris10

test_writerows (test.test_csv.Test_Csv) ... ok

======================================================================
FAIL: test_write_simple_dict (test.test_csv.TestDictFields)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_csv.py", line 607, in test_write_simple_dict
    self.assertEqual(fileobj.read(), "10,,abc\r\n")
AssertionError: 'f1,f2,f3\r\n' != '10,,abc\r\n'

Ran 84 tests in 17.335s

FAILED (failures=1)
test test_csv failed -- Traceback (most recent call last):
  File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_csv.py", line 607, in test_write_simple_dict
    self.assertEqual(fileobj.read(), "10,,abc\r\n")
AssertionError: 'f1,f2,f3\r\n' != '10,,abc\r\n'
  • x86 XP and x86 Windows7
test_csv
test test_csv failed -- Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\test\test_csv.py", line 604, in test_write_simple_dict
    writer.writerow({"f1": 10, "f3": "abc"})
  File "D:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\csv.py", line 148, in writerow
    return self.writer.writerow(self._dict_to_list(rowdict))
IOError: [Errno 0] Error

@florentx florentx mannequin reopened this Mar 4, 2010
@florentx florentx mannequin reopened this Mar 4, 2010
@djc
Copy link
Member

djc commented Mar 4, 2010

Testing on Windows with this:

Index: Lib/test/test_csv.py
===================================================================

--- Lib/test/test_csv.py	(revision 78430)
+++ Lib/test/test_csv.py	(working copy)
@@ -9,6 +9,7 @@
 import tempfile
 import csv
 import gc
+import io
 from test import test_support
 
 class Test_Csv(unittest.TestCase):
@@ -595,7 +596,7 @@
     ### "short" means there are fewer elements in the row than fieldnames
     def test_write_simple_dict(self):
         fd, name = tempfile.mkstemp()
-        fileobj = os.fdopen(fd, "w+b")
+        fileobj = io.open(fd, 'w+b')
         try:
             writer = csv.DictWriter(fileobj, fieldnames = ["f1", "f2", "f3"])
             writer.writeheader()

@djc
Copy link
Member

djc commented Mar 4, 2010

Committed in r78660 after positive comment from briancurtin re Windows. Hopefully this fixes Solaris, as well.

@djc
Copy link
Member

djc commented Mar 5, 2010

Both the solaris and windows slaves seem to have succeeded this time.

@djc djc closed this as completed Mar 5, 2010
@djc djc closed this as completed Mar 5, 2010
@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-feature A feature request or enhancement
Projects
None yet
Development

No branches or pull requests

3 participants