Add framework for gating tests / sanity checks

[tiran]

As a maintainer, I would like to have a system to run gating tests and sanity checks for a wheel that have been built by Fromager. The tests should be executed after a wheel and all its dependencies have been built and before the source directory of the package has been removed. Fromager should automate the process of creating a test virtual env, installing the package, and running tests.

Proposals

• tests are defined in overrides/gating/{override_package_name}/test*.[py|sh] (e.g overrides/gating/flash_attn/test_something.py or overrides/gating/torch/test.sh).
• Fromager creates a new virtual env for testing and installs the package + its dependencies in the virtual env
• Fromager passes the env variables from the override configuration + os.environ to each test file
• Additionally Fromager sets env vars like FROMAGER_VARIANT, FROMAGER_SDIST_ROOT, and whatever else we find useful. This lets test scripts locate the unpacked sources or change different aspects for each variant.
• test scripts are executed with the overrides/gating/{override_package_name} as their CWD.
• shell scripts must be executable
• Python test scripts are run with python -m test_$name

Possible enhancements for later

• install pytest and use pytest to run Python test files?
• provide additional pytest fixtures, e.g. for variant and sdist root?
• add config option to install additional packages in the test env?
• add option to automatically run tox / nox when the project uses tox / nox?

[dhellmann]

I agree this needs to be part of a build pipeline. Does it need to be a core part of fromager?

If we look at fromager as producing a "set of wheels" as output, then we might want to test them together as well as separately. Are you thinking fromager would test individual wheels as they are built, and fail the build if a test did not pass? Then at the end of the job something would test the collection together?