Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Tape tests from tap.createConsumer to concat-stream #2

Closed
nelsonic opened this issue Aug 27, 2016 · 11 comments
Closed

Update Tape tests from tap.createConsumer to concat-stream #2

nelsonic opened this issue Aug 27, 2016 · 11 comments
Assignees

Comments

@nelsonic
Copy link
Member

nelsonic commented Aug 27, 2016

While reading Tape source code I decided to check the devDependendices and spotted that they are way out-of-date: tape-testing/tape#312

When we update devDependencies 25 tests fail.
tape-test-fail

This is because tap no longer has a createConsumer method.

After a bit of googling we found a "guide" for how to update the tests...
See: https://github.com/substack/tape/pull/171/files

@nelsonic nelsonic changed the title In-depth Study of Tape source How is Tape tested? Aug 27, 2016
@nelsonic nelsonic self-assigned this Aug 27, 2016
@nelsonic
Copy link
Member Author

Run an individual tap test with:

node_modules/.bin/tap test/array.js

tap-run-one-test

@nelsonic nelsonic changed the title How is Tape tested? Update Tape tests from tap.createConsumer to concat-stream Aug 27, 2016
nelsonic added a commit to nelsonic/tape that referenced this issue Aug 28, 2016
@nelsonic
Copy link
Member Author

Once we update the test/array.js to use concat-stream the test passes:
tape-array-test-passes-after-update

@nelsonic
Copy link
Member Author

(Un)fortunately the latest version of tap has more detailed _stack trace_ when a test fails:

tape-end-as-callaback-error-stacktrace

So in order to make the tests pass we need to hard code the stack trace ...? 😕

@ghost
Copy link

ghost commented Aug 28, 2016

@nelsonic What is the difference? The path?
I guess you are testing actual error message wich will be different from computer to computer and even in a CI environment.

I think I had similar issue couple of days ago when I made my own long stack trace solution. One solution I used was cwd() and then I also used indexOf to do a deep comparison with some of the error message that I would know was identical and didn't include the paths.

I further "hacked" into the reporter itself and checked if "failed" was triggered.

Maybe not the way to go... How is the TAP stack trace code?

@nelsonic
Copy link
Member Author

I initially considered writing a function that attempted to return a semi-hard-coded version of the stack trace:

var basepath = __filename.replace('test/end-as-callback.js', '');
// stack trace for new tap error 
var s = '  ---\n';
s += '    operator: error\n';
s += '    expected: |-\n';
s += '      undefined\n';
s += '    actual: |-\n';
s += '      [Error: fail]\n';
s += '    stack: |-\n';
s += '      Error: fail\n';
s += '          at fakeAsyncWriteFail (' + basepath + 'test/end-as-callback.js:61:8)\n';
s += '          at ' + basepath + 'test/end-as-callback.js:47:13\n';
s += '          at fakeAsyncTask (' + basepath + 'test/end-as-callback.js:52:5)\n';
s += '          at Test.<anonymous> (' + basepath + 'test/end-as-callback.js:42:9)\n';
s += '          at Test.bound [as _cb] (' + basepath + 'lib/test.js:63:32)\n';
s += '          at Test.run (' + basepath + 'lib/test.js:82:10)\n';
s += '          at Test.bound [as run] (' + basepath + 'lib/test.js:63:32)\n';
s += '          at Immediate.next [as _onImmediate] (' + basepath + 'lib/results.js:71:15)\n';
s += '          at processImmediate [as _immediateCallback] (timers.js:383:17)\n';
s += '  ...\n';

But I don't want everyone to hate me each time one of the files gets updated so instead decided to read the stacktrace lines from the stdout

@nelsonic
Copy link
Member Author

nelsonic commented Aug 28, 2016

Finished the little getStackTrace function. It should be working ... but instead we get:
tape-end-as-callback-should-pass

What does this mean?!?

+++ found
--- wanted

There's no difference!

nelsonic added a commit to nelsonic/tape that referenced this issue Aug 28, 2016
@nelsonic
Copy link
Member Author

Tape skip tests are not cooperating ...
tape-skip-test-failing

@nelsonic
Copy link
Member Author

nelsonic commented Aug 28, 2016

When I attempt to re-organise the file I get:
tape-skip-useless

Which is worse than useless! :-(

@nelsonic
Copy link
Member Author

nelsonic commented Aug 28, 2016

tape test/throws.js tests fail inexplicably:
tape-throws-tests-fail-inexplicably

@nelsonic
Copy link
Member Author

double-blank-lines in test/exit.js
tape-test-exit-double-blank-lines

@nelsonic
Copy link
Member Author

closing. tape-testing/tape#318 was merged. ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant