Code coverage in Component Tests

[dvkuenssberg]

What would you like?

We currently use code coverage for E2E tests in our project, but would also like to use it with Cypress component tests.

Why is this needed?

Setting up the component section in cypress.config.ts similarly to the e2e section does not seem to successfully recognize code coverage: e.g. one file has 0.5 % coverage despite being thoroughly tested by Component tests.

We use this plugin to gather the coverage reports and have them analyzed by SonarQube.

  component: {
    setupNodeEvents(on, config) {
      on('after:run', results => {
        return require('cypress-sonarqube-reporter/mergeReports')(results);
      });
      require('@cypress/code-coverage/task')(on, config);
      return config;
    },
    devServer: {
      framework: 'angular',
      bundler: 'webpack',
    },
    specPattern: 'src/**/*.cy.ts',
  },

Of course, this would allow to ensure that every component is sufficiently tested.

When running a component test, there is the following warning: Could not find any coverage information in your application by looking at the window coverage object. Did you forget to instrument your application?

How can we instrument the application so that coverage reports can be generated for component tests?

Other

No response

[mike-plummer]

Hi @dvkuenssberg , thanks for reaching out. Typically questions about plugins are best answered by the plugin author & community, so you may have better luck reaching out to them directly. From what I can see that plugin hasn't been updated in a while and doesn't have any particular support for component testing, but there might be some work in progress or a plan forward.

Code coverage in CT is a bit trickier than E2E since it has to hook into the dev server to set up instrumentation. Angular is particularly tricky since it requires customizing the default build process. We're still working on updating our docs & recipes with good examples since each framework handles these concerns differently (and with different levels of ease). Here are a couple example apps I can point you to that might give you a starting point if your chosen plugin doesn't yet have support:
Create React App
Another Create React App
Older version of Cypress with Angular, but may give some hints

[dvkuenssberg]

Hi @mike-plummer, thank you for your answer. Our current E2E testing setup actually uses an approach very similar to the one in the third link you included. We include the extraWebpackConfig option in our angular.json for the ng serve command so that the application is instrumented that way.

This leads me to my next question: do you know how Cypress starts the Angular application in order to run the component tests? Does it call ng serve with the default configuration? Since this would mean it uses the same configuration as the E2E tests, it should have the coverage information available; however that is not the case.

[mike-plummer]

@dvkuenssberg I'll try to give a simple and less-simple answer to your question - hopefully one of them helps 😁

Simple Answer
We provide a webpackConfig field when defining your devServer in cypress.config.js - this can be used to override/customize the generated Webpack config used to launch the component testing dev server. There's a little more context in this issue/comment.

More Technical Answer
Angular component testing has to build a Webpack server that's under Cypress' control so that we can hook in and customize it without sacrificing performance (relying on the default build process like ng serve would compile & scaffold large portions of your app rather than just the parts we need for testing). Most of the "magic" is in the @cypress/webpack-dev-server package, specifically in the helpers/angularHandler.ts file. We generate a config that can be passed to a @angular-devkit/build-angular instance, using base options out of your angular.json file, but custom builders and configs beyond the base browser builder direct from that file aren't yet supported.

Hopefully that gives you some context. Happy to answer any other questions!

[dvkuenssberg]

@mike-plummer Thank you for the explanation! I did some digging yesterday and also found the webpackConfig option.
Using the coverage.webpack.js that we use to instrument E2E tests, I am now also getting coverage reports for the component tests :)

[martin-mueller-solutions]

@mike-plummer how would you override certain parts of the webpack config? as I understand, the custom webpack config just gets merged with the default config. so it is not possible to update something in the default configuration, is it?

my use case is to add configuration to an existing loader which comes from angular.

[mike-plummer]

@dvkuenssberg Awesome! Glad to hear you got it working! 🚀

@magile-de You are correct - the webpackConfig values are merged on top of the base config, so to alter anything from Angular using this mechanism you would need to either change its config in source or completely override it via webpackConfig. Passing in and allowing the base configuration to be mutated was causing a lot of problems since it was very easy to remove/break the Cypress customizations we add - the merge strategy seemed like the best compromise to allow customization without unnecessary complication/risk. If you feel this is a necessary capability and have an idea for how it could be done safely & reliably we're always open to new feature requests and/or PRs!

Since this GH issue has been answered I will be closing it - if there are additional questions about how to use/configure Cypress I would recommend checking out our Discord server, it can be helpful for debugging or answering questions.

[martin-mueller-solutions]

@mike-plummer thanks for your response and the clarification.

I know the possibility to overwrite the config-object was removed to simplify it. And yeah, the developer might break something, if important stuff gets overwritten or removed from that configuration.

But, by removing this completely, you are closing the door for a lot of people who already have a custom webpack config, which is a little more complicated than just adding a loader. e.g. if you want to tinker with the scss-loader (like we do) or make changes to postcss, you have to adapt the config which comes from angular.

I would reason, that if somebody overwrites plugins or whatever in the webpack-config he/she knows what's happening.