Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] emit skipped tests as objects #473

Merged
merged 1 commit into from Jun 25, 2019

Conversation

r0mflip
Copy link
Contributor

@r0mflip r0mflip commented Jun 22, 2019

When using objectMode skipped tests are emitted as comments, they are actual tests that need to be emitted as objects.

Test code:

const test = require('tape');
const printer = new require('stream').Writable({objectMode: true});
printer.write = printer.end = obj => {
  console.log(obj);
};

test.createStream({objectMode: true})
.pipe(printer);

test('parent', t => {
  t.equal(true, true);
  t.test('child1', {skip: 'not installed'}, t2 => {
    t2.equal(true, true);
    t2.equal(true, false);
    t2.end();
  });
  t.test('child2', {todo: 'will get done soon'}, t3 => {
    t3.equal(true, false);
    t3.equal(true, true);
    t3.end();
  });
  t.test('child3', {todo: 'will do'});
  t.equal(true, true);
  t.equal(true, true);
  t.end();
});

When used normally:

{ type: 'test', name: 'parent', id: 0 }
{
  id: 0,
  ok: true,
  skip: undefined,
  todo: false,
  name: 'should be equal',
  operator: 'equal',
  objectPrintDepth: 5,
  actual: true,
  expected: true,
  test: 0,
  type: 'assert'
}
{
  id: 1,
  ok: true,
  skip: undefined,
  todo: false,
  name: 'should be equal',
  operator: 'equal',
  objectPrintDepth: 5,
  actual: true,
  expected: true,
  test: 0,
  type: 'assert'
}
{
  id: 2,
  ok: true,
  skip: undefined,
  todo: false,
  name: 'should be equal',
  operator: 'equal',
  objectPrintDepth: 5,
  actual: true,
  expected: true,
  test: 0,
  type: 'assert'
}
SKIP child1                                                 <- plain text
{ type: 'end', test: 1 }                                    <- no object with testId 1
{ type: 'test', name: 'child2', id: 2, parent: 0 }
{
  id: 0,
  ok: false,
  skip: undefined,
  todo: 'will get done soon',
  name: 'should be equal',
  operator: 'equal',
  objectPrintDepth: 5,
  actual: true,
  expected: false,
  functionName: 'Test.<anonymous>',
  file: 'D:\\dev\\testing\\tape-test\\test.js:19:8',
  line: 19,
  column: 8,
  at: 'Test.<anonymous> (D:\\dev\\testing\\tape-test\\test.js:19:8)',
  test: 2,
  type: 'assert'
}
{
  id: 1,
  ok: true,
  skip: undefined,
  todo: 'will get done soon',
  name: 'should be equal',
  operator: 'equal',
  objectPrintDepth: 5,
  actual: true,
  expected: true,
  test: 2,
  type: 'assert'
}
{ type: 'end', test: 2 }
{ type: 'end', test: 3 }                                       <- no object with testId 3
{ type: 'end', test: 0 }

After patch:

{ type: 'test', name: 'parent', id: 0, skip: false, todo: false }
{
  id: 0,
  ok: true,
  skip: undefined,
  todo: false,
  name: 'should be equal',
  operator: 'equal',
  objectPrintDepth: 5,
  actual: true,
  expected: true,
  test: 0,
  type: 'assert'
}
{
  id: 1,
  ok: true,
  skip: undefined,
  todo: false,
  name: 'should be equal',
  operator: 'equal',
  objectPrintDepth: 5,
  actual: true,
  expected: true,
  test: 0,
  type: 'assert'
}
{
  id: 2,
  ok: true,
  skip: undefined,
  todo: false,
  name: 'should be equal',
  operator: 'equal',
  objectPrintDepth: 5,
  actual: true,
  expected: true,
  test: 0,
  type: 'assert'
}
{
  type: 'test',
  name: 'child1',
  id: 1,                                                         <- object with testId 1
  skip: 'not installed',
  todo: false,
  parent: 0
}
{ type: 'end', test: 1 }                                         <- reference exists
{
  type: 'test',
  name: 'child2',
  id: 2,
  skip: false,
  todo: 'will get done soon',
  parent: 0
}
{
  id: 0,
  ok: false,
  skip: undefined,
  todo: 'will get done soon',
  name: 'should be equal',
  operator: 'equal',
  objectPrintDepth: 5,
  actual: true,
  expected: false,
  functionName: 'Test.<anonymous>',
  file: 'D:\\dev\\testing\\tape-test\\test.js:19:8',
  line: 19,
  column: 8,
  at: 'Test.<anonymous> (D:\\dev\\testing\\tape-test\\test.js:19:8)',
  test: 2,
  type: 'assert'
}
{
  id: 1,
  ok: true,
  skip: undefined,
  todo: 'will get done soon',
  name: 'should be equal',
  operator: 'equal',
  objectPrintDepth: 5,
  actual: true,
  expected: true,
  test: 2,
  type: 'assert'
}
{ type: 'end', test: 2 }
{
  type: 'test',
  name: 'child3',
  id: 3,                                                         <- object with testId 3
  skip: false,
  todo: 'will do',
  parent: 0
}
{ type: 'end', test: 3 }                                         <- reference exists
{ type: 'end', test: 0 }

lib/results.js Outdated Show resolved Hide resolved
lib/results.js Outdated Show resolved Hide resolved
Copy link
Collaborator

@ljharb ljharb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add tests covering your new changes.

lib/results.js Show resolved Hide resolved
lib/results.js Outdated Show resolved Hide resolved
lib/results.js Outdated Show resolved Hide resolved
test/todo.js Show resolved Hide resolved
@r0mflip
Copy link
Contributor Author

r0mflip commented Jun 22, 2019

Please add tests covering your new changes.

Will add tests for objectMode stream and stuff, if all regarding changes in this PR are ok

@ljharb
Copy link
Collaborator

ljharb commented Jun 22, 2019

Please do; tests help review of code.

@r0mflip
Copy link
Contributor Author

r0mflip commented Jun 23, 2019

Node 0.8 has a weird issue that I don't understand why only it fails.

lib/results.js Outdated Show resolved Hide resolved
+ 'not ok 2 needs insight # TODO\n'
+ ' ---\n'
+ ' operator: fail\n'
+ ' at: Test.<anonymous> ($TEST/todo_explanation.js:$LINE:$COL)\n'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the test on 0.8 seems to want Test.tap.test.test.todo here instead of Test.<anonymous> :-/

I'm confused because we have many other tests using Test.<anonymous> that don't have special handling for node 0.8.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get that, but only why node 0.8 😭 I've gone through the source and debugged it a thousand times but only node 0.8 failes for only todo_explanation.js. It's just a copy of todo.js

Copy link
Collaborator

@ljharb ljharb Jun 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if the change to `emit('prerun') somehow changed things for the v8 version 0.8 is using.

We could update test/common, perhaps, for node 0.8 only, to swap out Test.tap.test.test.todo for Test.<anonymous>?

Copy link
Contributor Author

@r0mflip r0mflip Jun 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if the change to `emit('prerun') somehow changed things for the v8 version 0.8 is using.

I tried by using the latest commit from master, It still has the same problem. It works normally, this PR effects it somehow.

My [naive] understanding is that in V8 version of 0.8, the stack part assumes the {todo: 'something'} object as one of the stack layer. The same happens for my local skip_explanation.js test. The test in #473 (comment) comment has the function in stack pointing to Test.tap.test.platform.

Update: IDK but when I pass strings as arguments to todo or skip this happens, when they are booleans it totally works.

test/todo_explanation.js Outdated Show resolved Hide resolved
@r0mflip
Copy link
Contributor Author

r0mflip commented Jun 24, 2019

Removed loose patches. Current changes include:

  1. Emit all objects (tests added)
  2. Add prefix (# TODO ..., # SKIP ...) to comments (tests altered accordingly)

Will try to add descriptive messages in another PR

Copy link
Collaborator

@ljharb ljharb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great as a bugfix.

@ljharb ljharb merged commit 8d5dc2f into tape-testing:master Jun 25, 2019
ljharb added a commit that referenced this pull request Jun 29, 2019
 - [New] Add descriptive messages for skipped asserts (#476)
 - [Fix] emit skipped tests as objects (#473)
 - [Refactor] use `!!` over `Boolean()`
 - [meta] clean up license so github can detect it
 - [Deps] update `inherits`, `resolve`
 - [Tests] add tests for 'todo' exit codes (#471)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants