feat: Add option to specify int64 as an Input dtype

[gs-olive]

Description

• Rework Input paradigm to be based on at::ScalarType as opposed to the previous nvinfer1::DataType, allowing a larger representation space of data types

• When paired with truncate_long_and_double, insert casts to ensure Torch engines using Int64 tensors receive the correct types, and TensorRT engines operating on those tensors receive downcasted Int32 versions thereof

• Add Torch block at the beginning of model graph to prepare types of input tensors for forthcoming engines in sequence

• Automatically follow internal tensor types to abstract away the different internal engines used (Torch/TensorRT) from the user

• Provide a framework for streamlined addition of other data types, including torch.double as valid input types

• Improve error checking to ensure model compilation and behavior is as documented. For example, disallow specification of Long type input if the engine is required to be converted entirely to TRT

• Modify compiler to extract inferred data types for each input

• Add Python API testing to ensure casts are inserted correctly and run in Torch

• Known Limitations:

  • Specifying dtype=torch.long on an Input in an input_signature is not supported currently and will throw an error before model compilation when used with the Python API
  • While Torch may output Int64 tensors from the overall model, Torch-TRT currently can only output Int32 tensors for models using TRT, as there is not a mechanism in place for differentiating intermediate blocks from final/beginning blocks in the graph
  • Torch-TRT will almost definitely alter the data type of the input tensor, in-place, if dtype=torch.long is specified, and the returned result will be of type torch.int32

Note: The scope of this feature is currently limited to partitioning-enabled models (require_full_compilation=False) having truncate_long_and_double=True, since the feature prepends a Torch-executed block to the graph which performs the necessary casts, and so it requires both partitioning and truncation.

Fixes #1546
Addresses most of #1543

Type of change

• New feature (non-breaking change which adds functionality)

Checklist:

• [ x ] My code follows the style guidelines of this project (You can use the linters)
• [ x ] I have performed a self-review of my own code
• [ x ] I have commented my code, particularly in hard-to-understand areas and hacks
• [ x ] I have made corresponding changes to the documentation
• [ x ] I have added tests to verify my fix or my feature
• [ x ] New and existing unit tests pass locally with my changes
• [ x ] I have added the relevant labels to my PR in so that relevant reviewers are notified

[narendasan]

@peri044 is the reviewer for this feature

[peri044]

@mfeliz-cruise Please check the description for more details on the extent of int64 dtype support. This PR should resolve the usecases that we discussed.

[peri044]

LGTM.

[gs-olive]

Fixes #1543