Extract durable components from projects/pt1

[stellaraccident]

We are re-organizing the project into the C++ MLIR core and sub-projects implementing specific features. As a first-step, we have moved the original, conjoined python directory to projects/pt1 because it contains a conjoined set of things that are both a) hopelessly tied to PyTorch1-era APIs, and b) we would not like to disrupt the public API surface of since people use this.

Now that that first step is done, additional re-organization work can take place.

Many of these are bite-sized tasks, and help is welcome. Discuss in comments and create an issue for yourself if you're going to work on one:

• Extract the jit_ir function_importer.h C++ code (and dependencies) to a new projects/jit_importer directory that can be shared by LTC and the existing jit_ir importer which is not needed for Dynamo.
  • Move PyTorch build environment initialization to the projects/ level and set it up if any projects are enabled which require native PyTorch linkage.
  • See if we can sever the class_annotator.h from the jit_importer and leave it where it is.
• Move base_lazy_backend and reference_lazy_backend to a new projects/ltc directory
  • Build the _REFERENCE_LAZY_BACKEND.so Python module directory into python_packages/_TORCH_MLIR_REFERENCE_LAZY_BACKEND.so, decoupling it from the torch-mlir namespace completely
• Find a way to implement projects/pt1/python/torch_mlir/dialects/torch/importer/jit_ir/csrc/get_registered_ops.cpp without hard-linking against PyTorch's C++ API (i.e. derive from the YAML files, etc) and make projects/pt1/python/torch_mlir/dialects/torch/importer/jit_ir/build_tools/torch_ods_gen.py a standalone utility at project scope without a dependency on the built system.
• Extract projects/pt1/e2e_testing as projects/e2e_testing and move the test suite there.
  • Investigate extending it for use for Dynamo tests with contributed backends
  • Decide what our new testing strategy is and whether additional investment needs to be made
• Add the projects/dynamo project and upstream the FX importer and related tools as part of a new top-level MLIR package.
• Roll forward Fall cleaning: Re-organize sources so that each target type is contained and optional. #2442 to make all MLIR backends isolated and optional.

References

• Original repo breakup PR: Re-organize project structure to separate PyTorch dependencies from core project. #2542
• RFC: https://discourse.llvm.org/t/torch-mlir-pytorch2-uplift/74000/16
• Some conversations:
  • https://discord.com/channels/636084430946959380/1169741917811134515/1169801040007286926

[stellaraccident]

The above patch gets some things moved so that old can co-exist with new. There is still some things about how LTC is linked that I think are sub-optimal but I opted to not make any functional change there. From a source level, it is now decoupled from the pt1 project.