v0.5.1

Changes

• Fix: A saved engine is now keyed by the GPU attributes that TensorRT uses to decide engine compatibility, not only by the CUDA device name. The key adds compute capability, global memory bus width, L2 cache size, shared memory limits, texture alignment, multiprocessor count, integrated or discrete, ECC state, and total memory. Two GPUs with the same name that TensorRT treats as different devices (for example, the same model with ECC enabled on one and disabled on the other) no longer reuse each other's saved engine. Before this fix, opening such an engine could make TensorRT warn Using an engine plan file across different models of devices is not supported ....
• If tensoracle cannot read one of these GPU attributes, open_engine raises RuntimeError instead of choosing a saved engine with an incomplete key.
• GPUs whose attributes are all the same still share one saved engine. The device index, UUID, and PCI bus ID are not part of the key.
• The documentation now covers bf16, int8, and fp8 engines: Build bf16, int8, and fp8 engines.

Upgrade note

The saved engine key has changed, so engines saved by an earlier version are rebuilt once on first use after the upgrade. The public API, the saved engine location, and the dependencies are unchanged.

Install

pip install tensoracle==0.5.1