Repo test harness

[Brianzchen]

CLI Version

3.6.1

What problem do you want to solve?

So I was talking to @meandmax today and he was telling me about how contributing definitions is hard or it's not easy to get into. One problem is that the definitions isn't actually running flow and it basically can't, if you do you'll get millions of errors because some defs may not be maintained or be incompatible with the current version running.

This ends up without a good place to build definitions while you're working in this repo. This poses problems for a contributor who wants to build a type definition not necessarily tied to a project and wanting quick type checking feedback from their IDE

Your take on the correct solution?

Create a test harness dir in the root directory that can be quickly spun up to run any flow version of choice and build the definition with tests which can then be easily transferred into definitions to run the full test suite and submit.

Whatever custom settings used for the test harness should be git ignored so that it doesn't cause random changes during commits.

Two options I can think of

1. root command that runs to create a new dir and group of files like a basic flow config, package.json, base file, test file, etc
2. There being an existing test harness dir that is preconfigured to run latest flow version but can be swapped to run a different flow version whenever we want

I'm impartial to either but I think solution 1 can be easier and should be achievable with a simple bash script.

Anything else?

Should also add some docs as part of this change

Do you want to submit a pull request to implement this change?

Yes