RHCLOUD-27198: Improve Cypress testing of sample plugin host application

[vojtechszocs]

Summary of changes:

• bump Cypress to version 12 and scope all Cypress tests to sample app package
• improve existing Cypress E2E test for sample app (load sample plugin and assert plugin runtime information)
• add Cypress component tests using TestPluginStore for React components that consume plugin information
• it is now possible to pass custom PluginLoader implementation to PluginStore via the PluginLoaderInterface
• add "Extensions" column to PluginInfoTable showing the number of extensions brought in by the given plugin

Notes on Cypress component testing

We recommend using a single TestPluginStore instance for all your tests. This can be done via the corresponding Cypress support file which is typically used for one-time preparation of your testing environment.

let pluginStore: TestPluginStore;

beforeEach(() => {
  pluginStore = new TestPluginStore();
});

Cypress.Commands.add('mount', (component, options = {}) => {
  return mount(<PluginStoreProvider store={pluginStore}>{component}</PluginStoreProvider>, options);
});

Cypress.Commands.add('getPluginStore', () => {
  return cy.wrap(pluginStore);
});

In your component test, you can then just cy.mount the given component and use cy.getPluginStore command to access the TestPluginStore instance.

describe('FooComponent', () => {

  it('should work', () => {
    // renders inside <PluginStoreProvider> automatically
    cy.mount(<FooComponent />);

    // add plugins, enable them, etc.
    cy.getPluginStore().then((pluginStore) => {
      pluginStore.addLoadedPlugin(mockPluginManifest({ name: 'test' }), mockPluginEntryModule());
      pluginStore.enablePlugins(['test']);
    });

    // assertions go here
    cy.get('[data-test-id="foo"]').should('contain.text', 'Foo Text');

    // you can also use "cy.getPluginStore" to assert the current state of TestPluginStore
  });

});

Check out this article to understand the sync / async duality of Cypress and how it transforms and runs your tests.

[codecov-commenter]

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (0eab266) 41.16% compared to head (066a043) 41.16%.
Report is 8 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #232   +/-   ##
=======================================
  Coverage   41.16%   41.16%           
=======================================
  Files          69       69           
  Lines        1693     1693           
  Branches      365      365           
=======================================
  Hits          697      697           
  Misses        956      956           
  Partials       40       40           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[openshift-ci-robot]

@vojtechszocs: This pull request references RHCLOUD-27198 which is a valid jira issue.

In response to this:

Summary of changes:

• bump Cypress to version 12 and scope all Cypress tests to sample app package
• improve existing Cypress End-To-End test for sample app (load sample plugin and assert plugin runtime information)
• add Cypress component tests using TestPluginStore for React components that consume extensions or plugin info
• it is now possible to pass custom PluginLoader implementation to PluginStore via the PluginLoaderInterface
• add "Extensions" column to PluginInfoTable showing the number of extensions brought in by the given plugin

Notes on Cypress component testing

We recommend using a single TestPluginStore instance for all your tests. This can be done via the corresponding Cypress support file which is typically used for one-time preparation of your testing environment.

let pluginStore: TestPluginStore;

beforeEach(() => {
 pluginStore = new TestPluginStore();
});

Cypress.Commands.add('mount', (component, options = {}) => {
 return mount(<PluginStoreProvider store={pluginStore}>{component}</PluginStoreProvider>, options);
});

Cypress.Commands.add('getPluginStore', () => {
 return cy.wrap(pluginStore);
});

In your component test, you can then just cy.mount the given component and use cy.getPluginStore command to access the TestPluginStore instance.

describe('FooComponent', () => {

 it('should work', () => {
   // renders inside <PluginStoreProvider> automatically
   cy.mount(<FooComponent />);

   // add plugins, enable them, etc.
   cy.getPluginStore().then((pluginStore) => {
     pluginStore.addLoadedPlugin(mockPluginManifest({ name: 'test' }), mockPluginEntryModule());
     pluginStore.enablePlugins(['test']);
   });

   // assertions go here
   cy.get('[data-test-id="foo"]').should('contain.text', 'Foo Text');

   // you can also use "cy.getPluginStore" to assert the current state of TestPluginStore
 });

});

Check out this article to understand the sync / async duality of Cypress and how it transforms and runs your tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[florkbr]

NIT - maybe we could rename this directory test-fixtures?

[vojtechszocs]

I'd prefer mocks since it's more specific to test-fixtures 😃

I always thought that in programming, test fixtures refer to means for providing a fixed (stable, repeatable) testing env. like specific test input data, specific mock objects etc.

But the above functions mockPluginManifest and mockPluginEntryModule create new mock objects from provided data with reasonable defaults, I don't think they are technically fixtures since they are not fully fixed, if that makes sense.

[florkbr]

Are these 2 utilities something we really want to expose/maintain? I'd imagine most apps/plugins consuming the SDK and writing tests will be providing their own test manifest/entry module.

[vojtechszocs]

That's a good question.

Mocking a plugin manifest (like in mockPluginManifest) is very simple so this could be offloaded to consumers.

Mocking an entry module (like in mockPluginEntryModule) is a bit more complex, but still most typically a simple transformation from static pluginModules: { [moduleRequest: string]: AnyObject } declaration to SDK expected get function (moduleRequest: string) => Promise<() => AnyObject> - I don't think it makes sense to test asynchronicity within that function, so the resulting Promise would most typically be Promise.resolve(() => pluginModules[moduleRequest]) or similar.

Overall, I think you're right here. We don't have to expose these functions, creating such mock objects should be relatively easy from consumer perspective.

[florkbr]

NIT: Wondering if it makes sense to put the cypress and e2e directories in the src folder or if we should move them out to top level directories.

[vojtechszocs]

We currently put all the source code in src directory per each package.

I don't see much value in creating sibling source directories scoped to specific tools or purposes - having all the sources under one directory seems simpler to me. (Test code is source code too, but oftentimes get treated as second class citizen in the codebase.)

AFAK there is no special treatment necessary for src/cypress directory - this code is used in the Cypress component testing webpack dev build, so the usual context applies (React code to be part of the resulting web application).

I've placed component-index.html into the cypress directory to co-locate the HTML template with associated setup code. This is similar to src/app-index.html.ejs and src/app-styles.css sitting next to src/app.tsx.

Note to self: app.css could be inlined to app-styles.css and the latter renamed to app.css.

[florkbr]

Out of curiosity - any reason we shouldn't bump to 13.x.x?

[vojtechszocs]

When I posted this PR, Cypress 13 wasn't released yet 😃

yarn npm info cypress -f dist-tags shows latest: '13.2.0' which is better than 13.0.x.

I will check https://docs.cypress.io/guides/references/changelog but generally I'd prefer staying at stable v12 for now.

[vojtechszocs]

/retest

[openshift-ci-robot]

@vojtechszocs: This pull request references RHCLOUD-27198 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.15.0" version, but no target version was set.

In response to this:

Summary of changes:

• bump Cypress to version 12 and scope all Cypress tests to sample app package
• improve existing Cypress E2E test for sample app (load sample plugin and assert plugin runtime information)
• add Cypress component tests using TestPluginStore for React components that consume plugin information
• it is now possible to pass custom PluginLoader implementation to PluginStore via the PluginLoaderInterface
• add "Extensions" column to PluginInfoTable showing the number of extensions brought in by the given plugin

Notes on Cypress component testing

We recommend using a single TestPluginStore instance for all your tests. This can be done via the corresponding Cypress support file which is typically used for one-time preparation of your testing environment.

let pluginStore: TestPluginStore;

beforeEach(() => {
 pluginStore = new TestPluginStore();
});

Cypress.Commands.add('mount', (component, options = {}) => {
 return mount(<PluginStoreProvider store={pluginStore}>{component}</PluginStoreProvider>, options);
});

Cypress.Commands.add('getPluginStore', () => {
 return cy.wrap(pluginStore);
});

In your component test, you can then just cy.mount the given component and use cy.getPluginStore command to access the TestPluginStore instance.

describe('FooComponent', () => {

 it('should work', () => {
   // renders inside <PluginStoreProvider> automatically
   cy.mount(<FooComponent />);

   // add plugins, enable them, etc.
   cy.getPluginStore().then((pluginStore) => {
     pluginStore.addLoadedPlugin(mockPluginManifest({ name: 'test' }), mockPluginEntryModule());
     pluginStore.enablePlugins(['test']);
   });

   // assertions go here
   cy.get('[data-test-id="foo"]').should('contain.text', 'Foo Text');

   // you can also use "cy.getPluginStore" to assert the current state of TestPluginStore
 });

});

Check out this article to understand the sync / async duality of Cypress and how it transforms and runs your tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci]

@vojtechszocs: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[florkbr]

I've checked these changes out locally and ran the sample app/plugin, builds, and tests. Things appear to be functioning as expected.

[openshift-ci-robot]

@vojtechszocs: This pull request references RHCLOUD-27198 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.15.0" version, but no target version was set.

In response to this:

Summary of changes:

• bump Cypress to version 12 and scope all Cypress tests to sample app package
• improve existing Cypress E2E test for sample app (load sample plugin and assert plugin runtime information)
• add Cypress component tests using TestPluginStore for React components that consume plugin information
• it is now possible to pass custom PluginLoader implementation to PluginStore via the PluginLoaderInterface
• add "Extensions" column to PluginInfoTable showing the number of extensions brought in by the given plugin

Notes on Cypress component testing

We recommend using a single TestPluginStore instance for all your tests. This can be done via the corresponding Cypress support file which is typically used for one-time preparation of your testing environment.

let pluginStore: TestPluginStore;

beforeEach(() => {
 pluginStore = new TestPluginStore();
});

Cypress.Commands.add('mount', (component, options = {}) => {
 return mount(<PluginStoreProvider store={pluginStore}>{component}</PluginStoreProvider>, options);
});

Cypress.Commands.add('getPluginStore', () => {
 return cy.wrap(pluginStore);
});

In your component test, you can then just cy.mount the given component and use cy.getPluginStore command to access the TestPluginStore instance.

describe('FooComponent', () => {

 it('should work', () => {
   // renders inside <PluginStoreProvider> automatically
   cy.mount(<FooComponent />);

   // add plugins, enable them, etc.
   cy.getPluginStore().then((pluginStore) => {
     pluginStore.addLoadedPlugin(mockPluginManifest({ name: 'test' }), mockPluginEntryModule());
     pluginStore.enablePlugins(['test']);
   });

   // assertions go here
   cy.get('[data-test-id="foo"]').should('contain.text', 'Foo Text');

   // you can also use "cy.getPluginStore" to assert the current state of TestPluginStore
 });

});

Check out this article to understand the sync / async duality of Cypress and how it transforms and runs your tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: florkbr, vojtechszocs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [florkbr,vojtechszocs]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment