Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
refactor to preserve reporting of original line numbers in requirements files #3125
This was referenced
Sep 21, 2015
I should have mentioned that I also fixed two others bugs in #3030:
If the file contains
Then the three lines are merged as 'line1 # comment line2' and 'line2' is dropped.
And more anecdotally, when there is no space before :
Is merged into 'line1#line2' and the comment is not dropped.
Merging join_lines() and ignore_comments() seemed the cleanest way to handle that.
There is also another cornercase: if the last line in the file ends with \ it is discarded because the yield in join_lines is never executed.
I would recommend at least porting some of my tests to this PR to make sure the above cases are handled properly.
@pgervais I made some updates for the cases you mentioned.
for readability sake, I'd like to keep the function structure the way I have it, if possible, but I also want to respect the time you put into your PR and thinking through all the test cases
So... if you're interested, I'd like your help in porting over missing tests. I haven't looked at all of them yet.
You'd basically cut a PR to my
Does that work for you?
ok @pgervais , I migrated over what tests I thought were important, except for one case which won't work.
you had it ending up like this
whereas, I have it like this:
prior to this change, comments were processed first, then line continuations, so we should stay with that for compatibility.
added a commit
this pull request
Oct 5, 2015
@qwcode Thanks a lot for following-up on that, sorry for not being very responsive.
I think processing comments before joining lines will confuse people. It's not the way parsing works for Python and as far as I can tell not in any other languages. In that case keeping backward compatibility seems to me less important than being consistent.
But you did all the work so up to you :-)
@pfmoore @dstufft @Xafer @Ivoz
In terms of the examples in the discussion, IMO "right" is not to do stuff like that...
So I more or less don't care what you do, and am happy with whatever you choose.
For reference, though, Python treats a comment as ending at the end of the physical line, so that
So if I understand the PR, you're consistent with Python's behaviour, which is probably the most intuitive thing to do.
A) requirements.txt never had the most formal definition... (slightly helps us in this instance)
B) If you define "Everything after a
C) You can choose to interpret the other order (parse
But that really looks inferior, in readability, to
In my eyes. So I see no reason not to parse comments first.
if we did process
processing comments first gives:
@pgervais that's because the comment is removed, but the line still ends - so you get
@qwcode I would expect
(i.e. process \ first). But my logic for thinking so is not that you process \ first. Rather it's that you process comments first, but a full-line comment is still a line. So
when the backslash joins the empty line to the req1 line.
This feels more in line with
But again, the real point here is that people shouldn't be mixing line continuation and comments anyway, because the results are confusing :-)
I may be wrong, but I don't think that's it @pfmoore
if you have "test.py" like so:
python will error with
it appears to error as if you had
which fails like this
Well, it removes the comment, leaving a blank line, joins it with the "a =" line, then errors on the newline. When reporting the error, it reports the original content of the line where the unexpected newline character was found.
But honestly, who cares about why? What concerns me is behaviour. I'll add notes to the tests to say what I prefer, and then leave it at that.
can you reference something in the python docs that supports this version?
I see this, which seems to not support this: https://docs.python.org/3.4/reference/lexical_analysis.html#blank-lines
I care about both. : ) seems easier to end up with the right behavior, if you know the why