Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tap reporter not returning valuable info #2118

Closed
yovasx2 opened this issue May 10, 2019 · 2 comments · Fixed by #2119
Closed

Tap reporter not returning valuable info #2118

yovasx2 opened this issue May 10, 2019 · 2 comments · Fixed by #2119

Comments

@yovasx2
Copy link
Contributor

yovasx2 commented May 10, 2019

While integrating with puppetter the stack trace is cool:

  ✖ suite › browser › signin › signin.ad › [monitoring, xx] Active Directory User - Rejected with valid user and invalid password Rejected promise returned by test

  1 test failed

  suite › browser › signin › signin.ad › [monitoring, xx] Active Directory User - Rejected with valid user and invalid password

  /Users/giovanni/onelogin/platform/e2e-tests/node_modules/puppeteer/lib/helper.js:231

  Rejected promise returned by test. Reason:

  Error {
    message: 'No node found for selector: button[type=submitss]',
  }

  assert (node_modules/puppeteer/lib/helper.js:231:11)
  DOMWorld.click (node_modules/puppeteer/lib/DOMWorld.js:366:5)
  Frame.<anonymous> (node_modules/puppeteer/lib/helper.js:110:27)
  Page.click (node_modules/puppeteer/lib/Page.js:988:29)
  click (test/suite/browser/signin/signin.ad.spec.js:42:16)

make: *** [test-xx] Error 1

But if we use the tap formatter, it returns the first element that is not so valuable:
'new WaitTask (node_modules/puppeteer/lib/DOMWorld.js:561:28)' extracted from:

TAP version 13
# suite › browser › signin › signin.ad › [monitoring, xx] Active Directory User - Rejected with valid user and invalid password
not ok 1 - suite › browser › signin › signin.ad › [monitoring, xx] Active Directory User - Rejected with valid user and invalid password
  ---
    name: AssertionError
    message: Rejected promise returned by test
    values:
      'Rejected promise returned by test. Reason:': |-
        TimeoutError {
          message: 'waiting for selector "[data-testid="auth-username-screen"]" failed: timeout 30000ms exceeded',
        }
    at: 'new WaitTask (node_modules/puppeteer/lib/DOMWorld.js:561:28)'
  ...


1..1
# tests 1
# pass 0
# fail 1

instead click (test/suite/browser/signin/signin.ad.spec.js:42:16)

@novemberborn
Copy link
Member

What does the TAP spec say about representing stack traces?

@yovasx2
Copy link
Contributor Author

yovasx2 commented Jun 6, 2019

From https://testanything.org/tap-version-13-specification.html:

If the test line is immediately followed by an indented block beginning with /^\s+---/ and ending with /^\s+.../ that block will be interpreted as an inline YAML document. The YAML encodes a data structure that provides more detailed information about the preceding test.
Currently (2007/03/17) the format of the data structure represented by a YAML block has not been standardized. It is likely that whatever schema emerges will be able to capture the kind of forensic information about a test’s execution seen in the example above.
The YAML block following each failure gives additional information about the failure that may be displayed by the harness:

TAP version 13
ok 1 - retrieving servers from the database
# need to ping 6 servers
ok 2 - pinged diamond
ok 3 - pinged ruby
not ok 4 - pinged saphire
  ---
  message: 'hostname "saphire" unknown'
  severity: fail
  ...
ok 5 - pinged onyx
not ok 6 - pinged quartz
  ---
  message: 'timeout'
  severity: fail
  ...
ok 7 - pinged gold
1..7

Since spec mentioned it's a yaml block if you need to show extra info for stacktrace (at key), then we should use standard yaml multiline: key: >-
According to https://yaml-multiline.info/

so the PR #2119 is valid

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants