keyword.py main does not preserve line endings when rewriting keyword file

[zware]

BPO	17830
Nosy	@benjaminp, @merwok, @bitdancer, @zware
Files	test_keyword_cleanup.diff keyword_preserve_nl.patch test_keyword_patch.diff: David's patch plus a bit

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-04-25.16:07:39.096>
created_at = <Date 2013-04-24.15:29:10.229>
labels = ['type-bug', 'tests']
title = 'keyword.py main does not preserve line endings when rewriting keyword file'
updated_at = <Date 2013-04-25.16:07:39.094>
user = 'https://github.com/zware'

bugs.python.org fields:

activity = <Date 2013-04-25.16:07:39.094>
actor = 'r.david.murray'
assignee = 'none'
closed = True
closed_date = <Date 2013-04-25.16:07:39.096>
closer = 'r.david.murray'
components = ['Tests']
creation = <Date 2013-04-24.15:29:10.229>
creator = 'zach.ware'
dependencies = []
files = ['30002', '30014', '30015']
hgrepos = []
issue_num = 17830
keywords = ['patch']
message_count = 5.0
messages = ['187710', '187780', '187787', '187789', '187790']
nosy_count = 5.0
nosy_names = ['benjamin.peterson', 'eric.araujo', 'r.david.murray', 'gregmalcolm', 'zach.ware']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue17830'
versions = ['Python 3.4']

[zware]

(Copying the nosy list from bpo-9607)

test_keyword has a couple of failures on Windows, all due to newline issues--see for example http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/1845/steps/test/logs/stdio.

test_keywords_py_without_markers_produces_error looks for a line ending with '\n', but Windows ends it with '\r\n'.

test_real_grammar_and_keyword_file, on the other hand, doesn't fail on my machine, due to the hg eol extension being enabled, but the cause of failure is filecmp.cmp working only in binary mode and paying no attention to line endings.

The attached patch fixes both failures, with and without the eol extension, by using a private _compare_files function instead of filecmp.cmp. The private function makes use of universal newlines to avoid issue. Also, all instances of self.addCleanup(lambda ...) have had the lambda removed as suggested by Éric Araujo in msg187567.

[bitdancer]

The filecmp test failure is not because of using filecmp. We want a binary comparison: the line endings of the generated file should match the line endings of the input file. So either the _copy_file_without_generated_keywords code is buggy, or this is a real bug in keyword.py. Probably the former.

As for addCleanup, I prefer the lambda version. But I'm fine with using the non-lambda version if that is more common in the test suite.

[bitdancer]

Or both.

Zach, can you try this patch on Windows? I tested it by setting my local keywords.py file to have DOS line endings, and it seems to work correctly.

Although this turns out to be a bug, it has never bothered anyone, so I don't have any intent to backport it.

[zware]

Your patch works for me. And for what it's worth, I agree about not backporting; keywords are frozen and never going to change in 2.7 or 3.3 anyway.

I did find one other small bug in test_keyword; running the test via "-m test.test_keyword", test_real_grammar_and_keyword_file was skipped. So I've patched your patch to include a fix; GRAMMAR_FILE is now an absolute path with relative elements derived from test_keyword.__file__ rather than a relative path.

[bitdancer]

I've committed this (in 58b0e301b78a), but it looks like the email to the tracker hasn't come through yet. I'm going to blithely assume this will fix the buildbot failures, and will reopen it if I'm wrong.