New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixed a bug that caused newlines on Windows to be \r\r\n in ASCII tables #8659
Conversation
Codecov Report
@@ Coverage Diff @@
## master #8659 +/- ##
=======================================
Coverage 86.95% 86.95%
=======================================
Files 399 399
Lines 59329 59329
Branches 1100 1100
=======================================
Hits 51587 51587
Misses 7101 7101
Partials 641 641
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept looks fine, just some nits.
astropy/io/ascii/tests/test_write.py
Outdated
with open(filename, 'r') as f: | ||
content = f.read() | ||
|
||
assert content.splitlines() == ['col', 'a', 'b', 'c'] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It takes looking at the docs of splitlines
to be sure this test is meaningful. I.e. I think that a\r\r\nb
would split into ['a', '', 'b']
but I would not bet on it without trying. What about being more explicit and reading in binary mode and testing against 'col{0}a{0}b{0}c'.format(os.linesep).encode('ascii')
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
splitlines does split into ['a', '', 'b', '', 'c']
(the test fails before the fix) but I'll go with the more explicit test you mention.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, it turns out the open
in the test also needs a newline=''
as it was automatically changing newlines to just \n
!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
Fixed a bug that caused newlines on Windows to be \r\r\n in ASCII tables
This is what happens when I decide to develop on Windows for fun!
On Windows:
returns:
Fun! This is because by default Python will 'helpfully' translate \n to \r\n on Windows, so if one already uses
os.linesep
(\r\n), \r\n gets translated to \r\r\n. Don't ask. This causes #5126.There are two possible solutions to this - either we always only use \n internally for newlines, or we continue to use
os.linesep
but we need to then open files withnewline=''
which disables any auto-translation.This PR implements the latter. Note that this will still produce the incorrect output if users open files themselves without specifying
newline=
in theopen()
call, but if we use the first approach above, we will produce the incorrect output if users pass in other kinds of file-like objects which don't auto-translate newlines (e.g. StringIO).Fixes #5126
Fixes #5299