Move e2e to api tests and unify test command

[alexandrebodin]

What does it do?

• Renames e2e tests to api tests
• Refactor the commands to run the api tests to just be yarn test:api and make default tests to be CE and enable EE when detecting a STRAPI_LICENSE env var only.

Why is it needed?

Describe the issue you are solving.

How to test it?

Provide information about the environment and the path to verify the behaviour.

Related issue(s)/PR(s)

Let us know if this is related to any issue/pull request

[codecov]

Codecov Report

Base: 58.76% // Head: 58.78% // Increases project coverage by +0.02% 🎉

Coverage data is based on head (15696c2) compared to base (c1de0e1).
Patch has no changes to coverable lines.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #14733      +/-   ##
==========================================
+ Coverage   58.76%   58.78%   +0.02%     
==========================================
  Files        1340     1341       +1     
  Lines       32489    32511      +22     
  Branches     6143     6145       +2     
==========================================
+ Hits        19091    19113      +22     
  Misses      11510    11510              
  Partials     1888     1888              

Flag	Coverage Δ
front	63.29% <ø> (+0.03%)	⬆️
unit	48.82% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...in/admin/src/pages/MarketplacePage/tests/server.js	81.81% <0.00%> (ø)
...lacePage/components/NpmPackageCard/PackageStats.js	100.00% <0.00%> (ø)
...MarketplacePage/components/NpmPackageCard/index.js	94.73% <0.00%> (+0.14%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[innerdvations]

Considering that the way the user interacts with Strapi is the API and that's what most of these are testing, I'm not 100% convinced those couldn't still be considered e2e tests that need their descriptions tweaked to read like user stories. But since they are all API related, and they all only test a single operation e2e and none of them test a sequence, I think it's fine to separate them like this.

I like that it opens the path for us to create complex e2e tests in the future, including e2e testing the admin UI, etc and to keep those separate from these simple tests of the API. That's something we're lacking.

The only comment I have on functionality is that I'm not sure about changing the default from running EE to CE. If CE is used as a default rather than specified, we should output a warning that it's running an incomplete test suite. My earlier concern with the difficulty onboarding contributors wasn't necessarily that EE tests get run by default, just that there wasn't a simple yarn script available to run the tests, and another way to run only CE.

Here are a few things I noticed while reviewing but would probably be better as a separate PR. Feel free to ignore and I'll bring them up at a later time:

1. It would be nice to have a way to run generateTestApp and cleanTestApp silently, it's unexpected to see the new Strapi help info appear at the beginning of the testing

2. we shouldn't be running jest with --forceExit if possible, at some point we should fix the tests that don't clean up after themselves (I'm not necessarily volunteering myself for that 😆)

[alexandrebodin]

Considering that the way the user interacts with Strapi is the API and that's what most of these are testing, I'm not 100% convinced those couldn't still be considered e2e tests that need their descriptions tweaked to read like user stories. But since they are all API related, and they all only test a single operation e2e and none of them test a sequence, I think it's fine to separate them like this.

I like that it opens the path for us to create complex e2e tests in the future, including e2e testing the admin UI, etc and to keep those separate from these simple tests of the API. That's something we're lacking.

The only comment I have on functionality is that I'm not sure about changing the default from running EE to CE. If CE is used as a default rather than specified, we should output a warning that it's running an incomplete test suite. My earlier concern with the difficulty onboarding contributors wasn't necessarily that EE tests get run by default, just that there wasn't a simple yarn script available to run the tests, and another way to run only CE.

I tthink it's wrong to consider CE tests as an incomplete version of EE tests. Those are 2 different versions and have tests in common but also specific tests each so each run is complete for the specific version. As Strapi is starting in CE by default I think it makes to have the same implicit behavior with tests

Here are a few things I noticed while reviewing but would probably be better as a separate PR. Feel free to ignore and I'll bring them up at a later time:

1. It would be nice to have a way to run generateTestApp and cleanTestApp silently, it's unexpected to see the new Strapi help info appear at the beginning of the testing
2. we shouldn't be running jest with --forceExit if possible, at some point we should fix the tests that don't clean up after themselves (I'm not necessarily volunteering myself for that 😆)

Indeed I'm not going to explore this for now but that's a good note

[innerdvations]

I tthink it's wrong to consider CE tests as an incomplete version of EE tests. Those are 2 different versions and have tests in common but also specific tests each so each run is complete for the specific version. As Strapi is starting in CE by default I think it makes to have the same implicit behavior with tests

Ok, I agree with that.