Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Line number discrepancy in TestRunner Report/ScriptRunner outp #983

Closed
aaronatball opened this issue Apr 22, 2019 · 1 comment

Comments

Projects
None yet
3 participants
@aaronatball
Copy link
Contributor

commented Apr 22, 2019

There is a discrepancy between the Line Number that an Exception occurs on, and the Line Number reported in the test_results.txt. The issue seems to stem from a discrepancy between the error produced in the instrumented_text (which generates the backtrace) and the actual line that error is on in the Script.

Right now, this discrepancy occurs because there are blank lines that are not instrumented, so for instance if an error occurs on Script line 14, but there are 4 blank lines in the script, the test_results.txt reports the error on line 10 because the 4 lines aren't instrumented.

This doesn't effect line highlighting, because the line number is explicitly passed to the pre_instrumented method, and while line_number is passed to the handle_exception method, the backtrace isn't updated to reflect the issue.

Here is a TestSuite that can reproduce this issue (ScriptRunnerFrame correctly reports the issue as being line 14, test_results incorrectly reports it as line 10):

load 'cosmos/tools/test_runner/test.rb'

class LineErrorTestSuite < Cosmos::TestSuite
  def initialize()
    super()
    add_test("LineErrorOne")
  end
end



class LineErrorOne < Cosmos::Test
  def test_case_one
    check_expression("1==2")    
  end
end

I think there are 2 basic ways to solve this issue.

The first, is that we simple instrument a blank line in the real instrumented code, whenever there is one in the Script. For instance, this spot in the code: https://github.com/BallAerospace/COSMOS/blob/master/lib/cosmos/tools/script_runner/script_runner_frame.rb#L578, would now look something like this:

          if segment.strip.empty?
            instrumented_text << "\n"
            next
          end

This has the drawback of forcing the instrumented text to have the exact same line numbers as the Script itself. This isn't too crazy, and the normal version of COSMOS is like this, but sometimes it is nice when editing the instrumented text to not have to have everything happen on one line (just makes development test easier, Ruby doesn't care).

The second way to solve it would be to have handle_exception update the line numbers of the backtrace based on the line number it was passed. This would allow instrumented code to not have to have a 1-to-1 relationship with line number.

@jasonatball

This comment has been minimized.

Copy link
Collaborator

commented Apr 24, 2019

I like the newline approach. I think it actually makes viewing the instrumented text easier to read.

@jasonatball jasonatball self-assigned this Apr 24, 2019

@jasonatball jasonatball added the bug label Apr 24, 2019

@ryanatball ryanatball added this to the v4.4.0 milestone Jul 2, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.