cy.task() calls do not fire in 'test:after:run' event block

[proCrichard]

Current behavior:

Attempting to fire cy.task() commands after a test completes appears to outright fail.

Desired behavior:

At the end of each test, I would expect the cy.task command to fire, running a block of code within the task registered in cypress/plugins/index.js

Steps to reproduce: (app code and test code)

Simple Test

it('Can navigate to Account page', () => {
    cy.get('nav').within(() => {
      cy.getByText('Account').should('have.attr', 'href', '/parent/settings');
    });
  });

In cypress/support/index.js I have the following event listener enabled:

Cypress.on('test:after:run', (testAttr, runnable) => {
    cy.task('logTestResults', {
      title: `${runnable.parent.title}${runnable.title}`,
      state: runnable.state,
      err: runnable.err,
    });
  });

And that should fire this task in cypress/plugins/index.js:

on('task', {
      logTestResults({ title, state, err }) {
        console.log('made it to the task');
        const tags = [];
        tags.push(`outcome:${state}`);
        tags.push(`description:${changeCase.dot(title)}`)
        if (err) {
          tags.push(`error:${changeCase.dot(err.name)}`);
          tags.push(`error.message:${changeCase.dot(err.message)}`);
        }
        dogapi.metric.send(`e2e`, 1, {tags:tags}, function(err, results){
          console.dir(results);
        });

        return null
      }
    });

Through console logging I can tell that the test:after:run event listener is working, as console logs appear in the cypress runner's JS console. However console logs within the plugin task do not get written to my terminal.

However, if I add a cy.task command DIRECTLY to my test like so:

it('Can navigate to Account page', () => {
    cy.task('logTestResults', {title: 'test', state: 'failed', err:{name:'testerr', message:'test err message'}});
    cy.get('nav').within(() => {
      cy.getByText('Account').should('have.attr', 'href', '/parent/settings');
    });
  });

The task registered in the plugins file DOES catch the command and logs the faked details I pass it. Why is that same cy.task command not working in my test:after:run event listener?

Versions

Cypress: 3.4.0

[proCrichard]

I thought about using an afterEach hook to accomplish the same goal, however in an afterEach block I do not have access to details of the test that has just finished. cy.state('runnable') in an afterEach block lacks the context I need.

[proCrichard]

So, I figured out how to get this to work in an after hook by learning more about the context available in an after hook. Here's how I got it working:

cypress/support/index.js

after(() => {
  const findTests = suite => {
    suite.tests.forEach(test => {
      cy.task('logTestResults', {
        title: `${test.parent.title}${test.title}`,
        state: test.state,
      });
    });

    if (suite.suites.length < 1) {
      return;
    }
    suite.suites.forEach(nestedSuite => {
      findTests(nestedSuite);
    });
  };

  cy.state('runnable').parent.suites.forEach(suite => {
    findTests(suite);
  });
});

However I still feel that the test:after:run event should be able to fire cy.task() commands so I will leave this issue open.

[gustawx]

I'm having same issue but with cy.writeFile(), it seems to be ignored. Nothing happens, no error...

[gustawx]

I observed that all cy commands are being ignored in test:after:run, i tried cy.writeFile, cy.exec and cy.task. They all seems to be ignored. Can anyone tell me if i need to do some 'magic' or this hsould just work?

[JRoggekamp]

I'm having the same issue with running node commands (path.basename) in a task fired on "test:after:run". I can't find a way to get it to work so hopefully, the Cypress team picks this up soon.

[drewbrend]

I also ran into this issue today.

Here's a little more info that may or may not help.

This does work if you have an empty after() or afterEach() block in either your spec or the support/index file. before() or beforeEach() won't make it work.

Weird I know, but if you just add after(() => {}) in your spec your example will work.

[drewbrend]

Here's a reproducible example:

Steps:

1. Check out https://github.com/drewbrend/use-typescript-with-cypress/tree/Issue-4823 (fork of https://github.com/bahmutov/use-typescript-with-cypress)
2. run npm i
3. run npm run cy:open
4. run spec.ts with the console open

Notice there is no output about test:after:run

5. edit spec.ts and add after(() => {})
6. run spec.ts with the console open

Notice there is a line that mentions test:after:run

7. cry with confusion

[brian-mann]

The test:after:run event is synchronous and will not respect async cypress commands - and likely never will. Hooks before, beforeEach, afterEach, and after are the only places you'll be able to put cypress commands because of how Cypress must tie hooks + tests together.

I understand what you all are trying to do - getting access to the currentTest properties from inside of a hook. This data should already be available to you as cy.state('runnable').currentTest.

[bahmutov]

We got to add cy.state method to our typescript definitions even if just a few properties, then our docs would be enough for people to build whatever they want I think

…

Sent from my iPhone

On Dec 12, 2019, at 19:11, Brian Mann ***@***.***> wrote:  The test:after:run event is synchronous and will not respect async cypress commands - and likely never will. Hooks before, beforeEach, afterEach, and after are the only places you'll be able to put cypress commands because of how Cypress must tie hooks + tests together. I understand what you all are trying to do - getting access to the currentTest properties from inside of a hook. This data should already be available to you as cy.state('runnable').currentTest. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[drewbrend]

That gets pretty much what I'd need, except I'm sending that test data to a service, which I have a node client for... I was making the request from within a cy.task because I ran into a CORS error when I tried to make that request from within the test:after:run hook.

Do you have an idea how I could accomplish this without using cy.task?

[brian-mann]

No. Use hooks and cy.task or cy.request. That's what they're there for.

[drewbrend]

I'm confused by that answer - when I call a cy.task from a hook it leads me to this issue 🙃

[bahmutov]

@drewbrend Brian means use afterEach or after hook and execute cy.task or any other Cypress command there :) Not Cypress.on hook

[proCrichard]

We got to add cy.state method to our typescript definitions even if just a few properties, then our docs would be enough for people to build whatever they want I think

This would be very helpful :)

[drewbrend]

I understand what you all are trying to do - getting access to the currentTest properties from inside of a hook. This data should already be available to you as cy.state('runnable').currentTest.

Unfortunately currentTest is not available on the runnable in an afterEach hook - I'm thinking this is a bug? I can understand it not being available in an after hook since there could be multiple tests and currentTest doesn't make as much sense.

All tests and suites in the run do exist on the object but it's pretty ugly to navigate. I could seriously hack this by doing something like

const afterHook = cy.state('runnable')
afterHook.parent.suites[0].tests

But there can be multiple parent.suites and there can be multiple suites nested inside each Suite in parent.suites - this would be the case if you have nested describe/context blocks in your spec. So to fully navigate it there would be a whole bunch of iterating.

There's also the fact that you can't actually tell which test this afterEach was triggered from as far as I can see. I could work around this since I'm reporting the results for all tests - I could check the pending value for each Test, keep track of what I've reported already and report any new Test with pending = false

All this to say:

1. Is runnable.currentTest supposed to be defined in an afterEach()?
2. If not, using test:after:run is waaaayyyyyyyyy cleaner
3. Please don't let me to hacky things 🙏

[drewbrend]

For anyone that stumbles on this I ended up finding a decent way to handle parsing the tests from the afterEach() block. It isn't as ideal as getting the single test representing the test that just ran, but it works.

suite.eachTest() solves the dilemma I was having above - it iterates through all levels of suites to collect all the tests

This is in /cypress/support/index/ts

const reportedTests: string[] = [];

function sendTestMetrics(test: Mocha.Test) {
  if (test.pending || !test.state) {
    // Test is either skipped or hasn't ran yet.
    // We need to check this because all tests will show up in the hook every time
    return;
  }
  
  reportedTests.push(test.fullTitle());

  // parse some data from test and send it to an internal service
}

function sendSuite(suite: Mocha.Suite) {
  suite.eachTest(test => {
    if (!reportedTests.includes(test.fullTitle())) {
      sendTestMetrics(test);
    }
  });
}

afterEach(() => {
  const afterHook = cy.state("runnable");

  if (afterHook.parent) {
    sendSuite(afterHook.parent);
  }
});

[jennifer-shehane]

Duplicate of #6316

Cypress commands are not supported to be run within events. The issue above requests that they be supported. Please comment in that issue if you want this feature.

[Hazzard17h]

@drewbrend maybe a bit late, but as I can see you can reference the current test in afterEach hook with:

afterEach(() => {
  // @ts-ignore
  const test = cy.state('runnable')?.ctx?.currentTest;
  if (test) {
    // stuffs here
  }
});

[kizylirio]

@jennifer-shehane I want to implement the full capture of page when some failure happens.
(The documentation explains in this case, the capture is always coerced to runner: https://docs.cypress.io/api/cypress-api/screenshot-api#Arguments)

So I thought something like that:

Cypress.Screenshot.defaults({
    capture: 'fullPage'
})

Cypress.on('test:after:run', (test, runnable) => {
    if (test.state === 'failed') {
        cy.screenshot()
    }
})

But I cannot call any cypress command from events, as you said.

How could I resolve it?