AST literal_eval exceptions provide no information about line number

[stevemer]

BPO	28964
Nosy	@berkerpeksag, @serhiy-storchaka, @stevemer, @mlouielu, @mbdevpl, @pablogsal, @isidentical, @iritkatriel
PRs	bpo-28964: add line number of node (if available) to ast.literal_eval… #23677
Files	mywork.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-12-25.17:05:11.980>
created_at = <Date 2016-12-13.21:47:44.235>
labels = ['type-feature', 'library', '3.10']
title = 'AST literal_eval exceptions provide no information about line number'
updated_at = <Date 2020-12-27.17:27:50.879>
user = 'https://github.com/stevemer'

bugs.python.org fields:

activity = <Date 2020-12-27.17:27:50.879>
actor = 'stevemerritt'
assignee = 'none'
closed = True
closed_date = <Date 2020-12-25.17:05:11.980>
closer = 'BTaskaya'
components = ['Library (Lib)']
creation = <Date 2016-12-13.21:47:44.235>
creator = 'stevemerritt'
dependencies = []
files = ['45879']
hgrepos = []
issue_num = 28964
keywords = ['patch']
message_count = 13.0
messages = ['283142', '284419', '284422', '284430', '292330', '292381', '377110', '382630', '382631', '383587', '383589', '383754', '383854']
nosy_count = 8.0
nosy_names = ['berker.peksag', 'serhiy.storchaka', 'stevemerritt', 'louielu', 'mbdevpl', 'pablogsal', 'BTaskaya', 'iritkatriel']
pr_nums = ['23677']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue28964'
versions = ['Python 3.10']

[stevemer]

Without line numbers, debugging syntax errors in large documents is a tedious and painful process.

[berkerpeksag]

Thanks for the patch, Steve!.

This looks like a reasonable request to me, but 1) we can only make this change in 3.7 2) we need a test case for the new message 3) you can use f-strings instead of str.format()

[serhiy-storchaka]

Usually literal_eval() is used with short one-line input. For what large documents do you use it?

[stevemer]

We're using Python dictionaries to express sizeable (read: 100-600 lines)
chunks of configuration data. We previously used JSON, but we now use
Jinja2 to template chunks of this configuration data, and JSON's inability
to handle trailing commas creates a problem here. So we've been using
ast.literal_eval, but the lack of line numbers makes debugging problems
with the configurations a painful process.

On Sun, Jan 1, 2017 at 2:41 AM Serhiy Storchaka <report@bugs.python.org>
wrote:

Serhiy Storchaka added the comment:

Usually literal_eval() is used with short one-line input. For what large
documents do you use it?

----------
nosy: +serhiy.storchaka

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue28964\>

---

[mlouielu]

Steve, will you work on this patch to GitHub?

[stevemer]

I'll try to get to it this weekend. A bit swamped on another project right now. Thanks for the interest and the reminder!

[iritkatriel]

Steve, are you planning to work on porting this patch to github?

Note: it would need a unit test, and also node is not guaranteed to have a lineno field.

[isidentical]

Usually literal_eval() is used with short one-line input. For what large documents do you use it?

I kind of agree with Serhiy on this, nearly every use case I've encountered was simple, single-line expression. Though, by seeing Irit's patch I think the implementation is reasonable enough to do.

[iritkatriel]

This just came up again in bpo-42561 so I went ahead and created the PR.

[isidentical]

I'm +0 on this (even though we only had 2 users wanting this, the implementation seems very trivial). @serhiy.storchaka, @pablogsal any opinions?

[pablogsal]

This is a recurring feature request (I have heard people asking for this a couple of times) and is simple enough to do, so I am fine with this. Unless Serhiy has some concern, I would go forward :)

[isidentical]

New changeset 586f3db by Irit Katriel in branch 'master':
bpo-28964: add line number of node (if available) to ast.literal_eval error messages (GH-23677)
586f3db

[stevemer]

Thank you!

On Fri, Dec 25, 2020 at 11:05 AM Batuhan Taskaya <report@bugs.python.org>
wrote:

Change by Batuhan Taskaya <isidentical@gmail.com>:

----------
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue28964\>

---