New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

events details types #30

Closed
leobalter opened this Issue Oct 25, 2015 · 10 comments

Comments

Projects
None yet
3 participants
@leobalter

leobalter commented Oct 25, 2015

On the README doc I miss the types for the event details, so I'm assuming:

  • Test: object - A test holds basic information on a single test/spec.
    • testName: string - name of the test
    • suiteName: string - name of the suite the test belongs to
    • status: string - result of the test. Can "passed", "failed" or "skipped". A skipped test is disabled, i.e. it will not be executed.
    • runtime: number - execution time in milliseconds
    • errors: array of string only | string - Depending on the framework, this is a single exception or a list of failed assertions. Will be an empty array for statuses other than failed.
  • Suite: A suite is a collection of tests and potentially other suites.
    • name: string - name of the suite
    • childSuites: array - array with all direct subsuites (objects or string names?)
    • tests: array - array containing all tests that directly belong to the suite (but not to a child suite)
    • runtime: number - execution time of the whole suite in milliseconds (including child suites)
    • status: object - summarized status of the suite (should it appear on suiteStart?)
      • failed, number - if at least one test failed how many tests failed? from 0 to n?
      • skipped: number: if all tests in the suite are skipped (and there is at least one skipped test) how many skipped tests? from 0 to n?
      • passed: number if there is at least one passed test and all other tests are skipped or if there are no tests in the suite. how many passed tests? from 0 to n?

Should we have an status total as well?

@jzaefferer

This comment has been minimized.

Show comment
Hide comment
@jzaefferer

jzaefferer Oct 29, 2015

Contributor

errors is an array of Error objects with (optional?) additional properties, see later part of #12 (comment) - need to add that to the spec.

Contributor

jzaefferer commented Oct 29, 2015

errors is an array of Error objects with (optional?) additional properties, see later part of #12 (comment) - need to add that to the spec.

@jzaefferer

This comment has been minimized.

Show comment
Hide comment
@jzaefferer

jzaefferer Oct 29, 2015

Contributor

For Suite:

  • childSuites: array of Suite
  • tests: array of Test
  • status: 0 to n sounds right, for all three
  • adding status.total as failed + skipped + passed makes sense, since that seems like a pretty common usecase
Contributor

jzaefferer commented Oct 29, 2015

For Suite:

  • childSuites: array of Suite
  • tests: array of Test
  • status: 0 to n sounds right, for all three
  • adding status.total as failed + skipped + passed makes sense, since that seems like a pretty common usecase
@jzaefferer

This comment has been minimized.

Show comment
Hide comment
@jzaefferer

jzaefferer Oct 29, 2015

Contributor

@fcarstens your help getting this into the spec would be much appreciated!

Contributor

jzaefferer commented Oct 29, 2015

@fcarstens your help getting this into the spec would be much appreciated!

@leobalter leobalter referenced this issue Jan 18, 2016

Closed

Core: Introduce before/after hooks for modules #919

11 of 11 tasks complete
@jzaefferer

This comment has been minimized.

Show comment
Hide comment
@jzaefferer

jzaefferer Mar 3, 2016

Contributor

Btw. the current draft was partially developed in this Google Doc, might still be useful: https://docs.google.com/document/d/1E9X90ODgnMW8bTssIAkfRIQiGX0n0FaGI1vk3geviIs/edit?usp=sharing

Contributor

jzaefferer commented Mar 3, 2016

Btw. the current draft was partially developed in this Google Doc, might still be useful: https://docs.google.com/document/d/1E9X90ODgnMW8bTssIAkfRIQiGX0n0FaGI1vk3geviIs/edit?usp=sharing

@jzaefferer

This comment has been minimized.

Show comment
Hide comment
@jzaefferer

jzaefferer Jul 25, 2016

Contributor

Also need to add fullName

Contributor

jzaefferer commented Jul 25, 2016

Also need to add fullName

@flore77 flore77 referenced this issue Aug 1, 2016

Merged

Update spec #84

@flore77

This comment has been minimized.

Show comment
Hide comment
@flore77

flore77 Aug 2, 2016

Contributor

I have noticed now this Suite property:

status: object:
failed, number
skipped, number
passed number

Currently status is only a string with values: failed, passed and skipped.

We could make an object with the following properties:

var details = {
status: String,
passed: Number,
failed: Number,
skipped: Number
total: Number
};

The suite property could be called details, status will be a string representing the summarized status, all tests passed, or at least one failed etc, and the other properties will give a proper report of how many passed, how many failed etc.

I think it is valuable information, what do you think @jzaefferer

Contributor

flore77 commented Aug 2, 2016

I have noticed now this Suite property:

status: object:
failed, number
skipped, number
passed number

Currently status is only a string with values: failed, passed and skipped.

We could make an object with the following properties:

var details = {
status: String,
passed: Number,
failed: Number,
skipped: Number
total: Number
};

The suite property could be called details, status will be a string representing the summarized status, all tests passed, or at least one failed etc, and the other properties will give a proper report of how many passed, how many failed etc.

I think it is valuable information, what do you think @jzaefferer

@jzaefferer

This comment has been minimized.

Show comment
Hide comment
@jzaefferer

jzaefferer Aug 3, 2016

Contributor

Making status a string, consistent with Test.status makes sense.

As for the other properties, nesting them seems fine as well. Though details isn't very useful. Since we're counting tests, maybe something like testCounts?

Contributor

jzaefferer commented Aug 3, 2016

Making status a string, consistent with Test.status makes sense.

As for the other properties, nesting them seems fine as well. Though details isn't very useful. Since we're counting tests, maybe something like testCounts?

@flore77

This comment has been minimized.

Show comment
Hide comment
@flore77

flore77 Aug 3, 2016

Contributor

Sounds cool. So we could strap out of the nesting, the status prop, so we will have:

var Suite = {
  name: String,
  ...
  status: String,
  testCounts: {
    passed: Number,
    failed: Number,
    skipped: Number,
    total: Number
  }
};
Contributor

flore77 commented Aug 3, 2016

Sounds cool. So we could strap out of the nesting, the status prop, so we will have:

var Suite = {
  name: String,
  ...
  status: String,
  testCounts: {
    passed: Number,
    failed: Number,
    skipped: Number,
    total: Number
  }
};
@flore77

This comment has been minimized.

Show comment
Hide comment
@flore77

flore77 Aug 3, 2016

Contributor

The testCounts property will count also the tests of the childSuites.

Contributor

flore77 commented Aug 3, 2016

The testCounts property will count also the tests of the childSuites.

@flore77

This comment has been minimized.

Show comment
Hide comment
@flore77

flore77 Aug 10, 2016

Contributor

Fixed by #84 and #85.

Contributor

flore77 commented Aug 10, 2016

Fixed by #84 and #85.

@flore77 flore77 closed this Aug 10, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment