Include TE core headers in final build

[ksivaman]

Description

This change includes the headers for our public C API in the final distribution for both source and wheel builds.

Type of change

• Documentation change (change only to the documentation, either a fix or a new content)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Infra/Build change
• Code refactoring

Changes

• Add a MANIFEST.in file to specify additional package data in distribution.

Checklist:

• I have read and followed the contributing guidelines
• The functionality is complete
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[greptile-apps]

Greptile Overview

Greptile Summary

This PR adds a MANIFEST.in file to include the C API headers from transformer_engine/common/include in both source distributions and wheel builds, making them available to downstream consumers who need to link against TransformerEngine's native C API. The change uses a recursive-include directive with a wildcard pattern *.* to capture all files with extensions in the header directory tree.

Potential Issues:

• The wildcard pattern *.* in MANIFEST.in may have unintended consequences:

  • Extension-less files: Headers without extensions (uncommon but possible in C/C++ projects) would be excluded. Consider if any files in transformer_engine/common/include lack extensions.
  • Non-header files: The pattern could inadvertently include non-header files like README, CMakeLists.txt, or build artifacts if present in the include directory. Verify the directory contains only header files or refine the pattern to *.h *.hpp *.cuh for more specificity.
  • Hidden files: The pattern will include hidden files (.DS_Store, .gitkeep, etc.) which may bloat the distribution unnecessarily.

• No corresponding changes to setup.py:

  • The setup.py file should declare these headers as package data using the package_data or include_package_data=True parameter to ensure they're installed correctly. Verify that setuptools will respect MANIFEST.in for wheel builds without explicit declaration.
  • The headers should be installed to a location that downstream C/C++ projects can discover via standard include paths or pkg-config.

• Missing documentation:

  • While the checklist indicates documentation changes weren't made, users integrating with the C API will need guidance on where to find these headers post-installation and how to configure their build systems to use them.

Confidence: 3/5 - The approach is standard for including package data, but the wildcard pattern needs verification against the actual directory contents, and the integration with setup.py should be confirmed to ensure the headers are actually installed and accessible in the final package.

_{1 file reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[ksivaman]

/te-ci