Local testing experience

[jbudz]

Issues

• npm scripts as entry [test] consolidate top level npm scripts #41133 closed by [7.x] Refactor test entry by runner (#44679) #57930
• Revisit documentation to address frequent questions
• x-pack functional tests from root [test/x-pack] add matching functional_test_runner script #44939
• Reproducing CI failures locally [CI] Add error steps and help links to PR comments #60772
• Consistent test entry points between x-pack and core.
  • gulp, grunt -> node scripts/* - addressed by [test] consolidate top level npm scripts #41133 and documentation changes
• Support API integration tests against remote hosts Support api_integration/kibana/stats against remote hosts #53000
• Migrate legacy test:server tests to jest
  • Generally WIP, we aren't adding new tests and there's some rewrites happening -> e.g. [ML] Migrate Mocha unit tests to Jest #62006
• Visual regression cleanup 1 2 3 4 5 6
  - These issues need to be consolidated and organized around one testing framework
• Use Jest muti-project runner instead of a global config Use Jest muti-project runner instead of a global config #72569
• x-pack jest tests from root [test/dev] support a common jest entry #75202

Exploration

• Speed
  • Can functional tests be rewritten to minimize timeouts?
  • Can we improve startup time? (iffy - this is probably babel/register related)
  • Find and replace unit tests not using mock http and fs calls
• Organization
  • Can we make the configuration inheritance easier to trace?

Part of the development experience issue tracked at #34557

[elasticmachine]

Pinging @elastic/kibana-operations

[joshdover]

Regarding speed, I wonder if we should ratchet down the default jest test timeout and/or introduce a "slow test warning" in order to keep our main unit test suite fast. Tests that show up in this list should either be refactored to use mocks (as suggested above) or moved into the jest integration suite if they truly need to exercise slower code paths.