fileinput.hook_encoded has no way to pass arguments to codecs

[lac]

BPO	25788
Nosy	@terryjreedy, @bitdancer, @berkerpeksag, @serhiy-storchaka, @jhasapp
Files	issue25788.patch: Patch File issue25788-2.patch issue25788-3.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-04-27.20:16:43.993>
created_at = <Date 2015-12-03.16:41:30.713>
labels = ['easy', 'type-feature', 'library', 'expert-IO']
title = 'fileinput.hook_encoded has no way to pass arguments to codecs'
updated_at = <Date 2016-04-27.20:16:43.991>
user = 'https://bugs.python.org/lac'

bugs.python.org fields:

activity = <Date 2016-04-27.20:16:43.991>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-04-27.20:16:43.993>
closer = 'serhiy.storchaka'
components = ['Library (Lib)', 'IO']
creation = <Date 2015-12-03.16:41:30.713>
creator = 'lac'
dependencies = []
files = ['41777', '42620', '42631']
hgrepos = []
issue_num = 25788
keywords = ['patch', 'easy']
message_count = 15.0
messages = ['255818', '255819', '255824', '255826', '255827', '256164', '258072', '259347', '264179', '264193', '264334', '264351', '264379', '264400', '264401']
nosy_count = 8.0
nosy_names = ['terry.reedy', 'r.david.murray', 'python-dev', 'berker.peksag', 'serhiy.storchaka', 'lac', 'curioswati', 'joseph.hackman']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue25788'
versions = ['Python 3.6']

[lac]

Right now there is no way, aside from writing your own openhook, to
get around the limitation that openhook=fileinput.hook_encoded("utf")
will open things with the default option for codecs.open()
of errors=strict. Adding a way to pass the errors argument seems
both easy to do and useful.

[serhiy-storchaka]

It would be easy to add the errors parameter to fileinput.hook_encoded().

Do you want to provide a patch Laura?

[lac]

I haven't made a patch to Python for over 10 years. Before mercurial. :) Where do you start in terms of 'how to submit a patch'?

[bitdancer]

Mercurial didn't change the patch submission process (we're talking about making such changes on python-workflow now...or rather enhancements as the current process should continue to work). The only thing that mercurial changed is how you get a copy of the current repo. After that (which is explained in the devguide), just make your changes, do 'hg diff', and attach the diff file to the issue as usual.

[serhiy-storchaka]

https://docs.python.org/devguide/#contributing

[curioswati]

I want to work on this issue. @lac, can you please help as I searched but couldn't find the related files. Where can I find the code for this?

[terryjreedy]

fileinput is in Python, hence the source is in Lib. Online version is
https://hg.python.org/cpython/file/tip/Lib/fileinput.py
https://hg.python.org/cpython/file/tip/Lib/test/test_fileinput.py

Patching openhook itself looks trivial. A test is the hard part. Laura, can you provide a test case? -- a short (one line?) file that fails with errors=strict and passes with something else? A new test_errors method should be added to class Test_hook_encoded (the last testCase in the file). I think the 'test' method would need revision.

[jhasapp]

I haven't seen OP in over 30 days, so am posting my own patch.

I've added an optional argument that defaults to strict and gets passed along.

I've updated the primary test to verify the argument passing, as well as that things get handled as specified in the documentation at https://docs.python.org/3.5/library/codecs.html

This is off-topic, but is there any way I can submit a patch that allows a similar fix for stdin? Presently there is no way at all to pass malformed unicode through fileinput using stdin that I can find.

[jhasapp]

Ping.

Just wondering if anyone on the nosy list would be willing to help review my patch. :)

[serhiy-storchaka]

Added comments on Rietveld (follow the "review" link beside the patch link).

[jhasapp]

Uploading a new patch to address the issues in previous patch.

[serhiy-storchaka]

Oh, I forget. Needed updates of the documentation (including the "versionchanged" directive). And would be nice if you add corresponding entities in Doc/whatsnew/3.6.rst, Misc/NEWS and Misc/ACKS. The rest of the patch LGTM.

[jhasapp]

Updated documentation in fileinput.rst, Doc/whatsnew/3.6.rst, Misc/NEWS and Misc/ACKS.

Thank you so much Serhiy for taking the time to review!

[python-dev]

New changeset 8ab8f5259f09 by Serhiy Storchaka in branch 'default':
Issue bpo-25788: fileinput.hook_encoded() now supports an "errors" argument
https://hg.python.org/cpython/rev/8ab8f5259f09

[serhiy-storchaka]

Committed with some changes. Thank you for your contribution Joseph.