Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: using "go tool test2json dlv test-binary" marks the last test as failed on Mac OS #34286

Open
nd opened this issue Sep 13, 2019 · 4 comments

Comments

@nd
Copy link
Contributor

commented Sep 13, 2019

This is continuation of #23033 which is still actual in go1.13.

To reproduce the issue create <GOPATH>/src/debugTest/foo_test.go:

package foo

import "testing"

func Test1(t *testing.T) {
}

compile it:

go test -c -o testBinary -gcflags "all=-N -l" debugTest

run:

go tool test2json dlv --listen=:2345 --headless exec testBinary -- -test.v

and connect debugger from another console: dlv connect :2345.

In the Delve console type continue and once the testBinary terminates, exit the delve client via the exit command.

Test2json produces output:

{"Action":"run","Test":"Test1"}
{"Action":"output","Test":"Test1","Output":"=== RUN   Test1\r\n"}
{"Action":"output","Test":"Test1","Output":"--- PASS: Test1 (0.00s)\r\n"}
{"Action":"output","Test":"Test1","Output":"PASS\r\n"}
{"Action":"fail","Test":"Test1"}

This happens because on Mac OS Delve runs programs via debugserver and debugserver uses \r\n line-endings. For most of test output it is not a problem because test2json check prefixes and is not affected by line-ending differences. But for final PASS or FAIL the exact Equal check is used and it fails due to added \r.

This behavior was accidentally fixed in d47526e, but judging by comments in #31969 the fix will be reverted later.

I understand that this is not a bug in test2json. Test2json assumes the output is produced by the test framework and the test framework always uses \n. It is possible to wrap Delve and replace PASS\r\n in its output to PASS\n, but every test2json client will have to do that. Maybe it is possible to support this case in test2json itself?

In #23033 there was a question: why one would want to debug a test and also see the its final status. The most relevant use-case I can think of is a modification of a program state during debug session. Let's say I have a complex algorithm test for which started to fail. During a debug session I've found a potential problem. I cannot easily change the sources and rerun a test because the fix is complicated, but I can change the program state at runtime. I want to verify that changing some value in a program produces a desirable effect and test starts to pass. At the moment it is not possible as the test status is always "fail".

@ianlancetaylor ianlancetaylor changed the title Test2json marks the last test as failed on Mac OS testing: using "go tool test2json dlv test-binary" marks the last test as failed on Mac OS Sep 13, 2019
@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Sep 13, 2019

Why do you run

go tool test2json dlv --listen=:2345 --headless exec testBinary -- -test.v

instead of

dlv --listen=:2345 --headless exec testBinary -- -test.v

? How does running under go tool test2json make it easier to debug the program?

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Sep 13, 2019

Why does delve output \r\n rather than \n when its standard output is not a terminal? Where is that coming from?

@nd

This comment has been minimized.

Copy link
Contributor Author

commented Sep 13, 2019

Test2json is used to simplify test output parsing, json is easier to parse than the raw output produced by test executable. Test2json output is consumed by an editor and is shown in a dedicated UI where each test has an icon according to its status. It is easier to grasp tests status from such a view than from reading output of all tests. Test2json doesn't change the debugging experience per se, but it is useful to be able both debug a test and see its outcome once debug session is over.

If I understand correctly Delve redirects debugserver output to its stdout without changing anything. @aarzilli could you please confirm?

@aarzilli

This comment has been minimized.

Copy link
Contributor

commented Sep 13, 2019

This thing with tests getting \r\n on macOS came out before and I couldn't figure out where it happens. The way it works is that delve runs the test executable under debugserver and then debugserver sends us the output in a stop packet (it's the 'O' kind). It already contains the \r there so either debugserver is adding it (werid) or possibly the tty layer in the kernel (weirder).

@toothrot toothrot added this to the Unplanned milestone Sep 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.