supported cross serialization and runtime merge

[cehongwang]

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes # (issue)

Type of change

Please delete options that are not relevant and/or add your own.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist:

• My code follows the style guidelines of this project (You can use the linters)
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas and hacks
• I have made corresponding changes to the documentation
• I have added tests to verify my fix or my feature
• New and existing unit tests pass locally with my changes
• I have added the relevant labels to my PR in so that relevant reviewers are notified

[narendasan]

One minor thing then looks good

[narendasan]

Think you can just add a :Optional[torch.classes.tensorrt.Engine|TRTEngine] above

[cehongwang]

It gives me lots of other errors:

py/torch_tensorrt/dynamo/runtime/_TorchTensorRTModule.py:234: error: Item "None" of "Any | None" has no attribute "streamable_device_memory_budget"  [union-attr]
py/torch_tensorrt/dynamo/runtime/_TorchTensorRTModule.py:237: error: Item "None" of "Any | None" has no attribute "automatic_device_memory_budget"  [union-attr]
py/torch_tensorrt/dynamo/runtime/_TorchTensorRTModule.py:240: error: Item "None" of "Any | None" has no attribute "device_memory_budget"  [union-attr]
py/torch_tensorrt/dynamo/runtime/_TorchTensorRTModule.py:245: error: Item "None" of "Any | None" has no attribute "device_memory_budget"  [union-attr]
py/torch_tensorrt/dynamo/runtime/_TorchTensorRTModule.py:246: error: Item "None" of "Any | None" has no attribute "device_memory_budget"  [union-attr]
py/torch_tensorrt/dynamo/runtime/_TorchTensorRTModule.py:248: error: Item "None" of "Any | None" has no attribute "device_memory_budget"  [union-attr]
py/torch_tensorrt/dynamo/runtime/_TorchTensorRTModule.py:254: error: Item "None" of "Any | None" has no attribute "reset_captured_graph"  [union-attr]
py/torch_tensorrt/dynamo/runtime/_TorchTensorRTModule.py:260: error: Item "None" of "Any | None" has no attribute "use_dynamically_allocated_resources"  [union-attr]
py/torch_tensorrt/dynamo/runtime/_TorchTensorRTModule.py:384: error: Unused "type: ignore" comment  [unused-ignore]
py/torch_tensorrt/dynamo/runtime/_TorchTensorRTModule.py:399: error: Item "None" of "Any | None" has no attribute "use_pre_allocated_outputs"  [union-attr]
py/torch_tensorrt/dynamo/runtime/_TorchTensorRTModule.py:409: error: Item "None" of "Any | None" has no attribute "use_output_allocator_outputs"  [union-attr]
py/torch_tensorrt/dynamo/runtime/_TorchTensorRTModule.py:467: error: Item "None" of "Any | None" has no attribute "set_output_tensors_as_unowned"  [union-attr]
py/torch_tensorrt/dynamo/runtime/_TorchTensorRTModule.py:470: error: Item "None" of "Any | None" has no attribute "are_output_tensors_unowned"  [union-attr]

[narendasan]

Looks fine, please get an additional reviewer and make sure we are testing Python only in CI as a separate line of jobs for each channel

[tp5uiuc]

What's the path to test in CI currently? Do we have two jobs for each {CUDA-VERSION, GPU, OS, Flavor} combination (I realize CUDA-VERSION, GPU is fixed for now and flavor here refers to TRT standard vs TRT-RTX) : one with wheels built and one without?

From my read of the CI files : https://github.com/pytorch/TensorRT/blob/main/.github/workflows/build-test-linux-x86_64.yml it seems we only exercise the C++ route (build wheel, install it and run tests, which should pick the C++ runtime)? Is that right, or is my understanding off?

If so, can we first have a an MR updating CI to run both tests with python and C++ runtime and then put in this change? We are developing C++/python runtime features in parallel for TRT-RTX and want to test both paths if possible. Thanks!