v1.4.0

Changes

• The CMake helper cpp/cmake/TensoracleCuda.cmake provides the public function tensoracle_find_cuda(OUT_INCLUDE_DIR <var> OUT_DRIVER_LIBRARY <var>). It sets the two named variables in the calling scope to the directory containing cuda.h and the CUDA driver link library that tensoracle_add_cubin_plugin uses for the plugin build. It reads the same CUDAToolkit_ROOT CMake variable, environment variable, and default locations as tensoracle_add_cubin_plugin, and stops with the same error when an input is missing. Both keywords are required, each needs a value, and unknown arguments are rejected. The function can be called repeatedly, before or after adding a plugin.
• The public IMPORTED target tensoracle_cuda_driver carries the same include directory and driver library. Link a CMake target to it to inherit both.
• Plugins built by tensoracle_add_cubin_plugin link Threads::Threads on Linux, so they record libpthread.so.0 as a dependency on glibc 2.28 and other glibc versions earlier than 2.34. The common plugin source uses std::mutex, which needs the thread library to be loaded on those systems.
• The README lists the GPUs (sm86, sm89, sm120) used to test the example cubin bundle.

Upgrade note

The public Python API, saved engine file format, cache identity, tensoracle_add_cubin_plugin arguments, and runtime dependencies are unchanged from 1.3.0. Engines saved by 1.3.0 are reused without a rebuild.

Projects that read the CUDA include directory or driver library from the helper should call tensoracle_find_cuda or link tensoracle_cuda_driver instead of using internal helper names.

A project that added Threads::Threads to its own plugin target as a workaround can keep it; linking it twice is harmless.

Windows builds and CMake 3.20 were not exercised for this release; the CMake helper tests ran on Linux (WSL2) with CMake 3.31.

Install

pip install tensoracle==1.4.0

v1.3.0

Changes

• compile_bundle takes ptxas and cuobjdump from the PyPI wheels installed by the producer extra when their paths are not given: nvidia/cu13/bin/ptxas from nvidia-cuda-nvcc and nvidia/cu13/bin/cuobjdump from nvidia-cuda-cuobjdump (with .exe on Windows). It does not read CUDA_HOME. If a wheel or tool is missing, it raises an error that names the package and the searched path and explains how to install the extra or pass the path. The ptxas range check (13.2 or later and earlier than 14) and explicit tool paths are unchanged.
• The producer extra declares triton==3.8.0, nvidia-cuda-nvcc==13.2.*, nvidia-cuda-cuobjdump==13.2.*, nvidia-cuda-crt==13.2.*, and nvidia-nvvm==13.2.*.
• The CMake helpers (cpp/CMakeLists.txt and cpp/cmake/TensoracleCubinPlugin.cmake) find only cuda.h and the CUDA driver link library, and no longer call find_package(CUDAToolkit). Pass the site-packages/nvidia/cu13 directory of PyPI nvidia-cuda-nvcc and nvidia-cuda-runtime 13.2 as CUDAToolkit_ROOT (a CMake variable or environment variable). On Linux the helpers link the NVIDIA driver's libcuda.so, including /usr/lib/wsl/lib/libcuda.so on WSL; on Windows they link the wheel's lib/x64/cuda.lib. When CUDAToolkit_ROOT is not set, they search nvcc on PATH, CUDA_PATH, and /usr/local/cuda as before. Configuration prints the selected header directory and driver library.
• tensoracle_cubin links the internal imported target tensoracle_internal_cuda_driver publicly, so targets that link tensoracle_cubin still receive the CUDA driver include directory and library. The helpers do not define CUDA:: targets, so a project that calls find_package(CUDAToolkit) itself is unaffected.
• The documentation and the example describe cubin production and native plugin builds without an OS-installed CUDA Toolkit. The example cubin bundle is unchanged (bundle digest 207d41310c87ae8b75b0301e881df881cdd2475eec07b82092e0ec66febbd5c0).

Upgrade note

The public Python API, saved engine file format, cache identity, and runtime dependencies are unchanged from 1.2.0. Engines saved by 1.2.0 are reused without a rebuild.

Applications that build cubins with the producer extra no longer need an OS-installed CUDA Toolkit or CUDA_HOME; the extra installs the tools.

Projects that build a native plugin with the CMake helpers no longer need an OS-installed CUDA Toolkit. Pass the PyPI nvidia/cu13 directory as CUDAToolkit_ROOT, or keep passing an OS toolkit root. A project that used CUDAToolkit_INCLUDE_DIRS or CUDA:: targets only because the helper called find_package(CUDAToolkit) must now call find_package(CUDAToolkit) itself, or link tensoracle_cubin.

Windows builds and sm89 / sm120 GPUs were not exercised for this release; sm86 on WSL2 was.

Install

pip install tensoracle==1.3.0

v1.2.0

Changes

• cubin_sm, inspect_bundle, and inspect_wheel now read cubins built by ptxas 13.x. ptxas 13.x writes ELF ABI version 8, which stores the SM number in a different field of the ELF flags. Tensoracle reads the field that matches the ABI version and accepts sm86, sm89, and sm120 for both ABI version 7 and ABI version 8. Version 1.1.0 rejects ptxas 13.x cubins for sm86 and sm89 with CubinArchitectureError.
• compile_bundle takes ptxas and cuobjdump from the host CUDA toolkit when their paths are not given: $CUDA_HOME/bin, or /usr/local/cuda/bin when CUDA_HOME is unset. If either tool is missing, it raises FileNotFoundError with the paths it searched. It requires ptxas 13.2 or later and earlier than 14, and raises ValueError before writing cubins or the manifest when the ptxas version is outside that range. Explicit tool paths are still accepted.
• The producer extra now declares only triton==3.8.0. It no longer installs NVIDIA command line tool wheels.
• The example cubin bundle is rebuilt with Triton 3.8.0 and ptxas 13.2.86.

Upgrade note

The public API, saved engine file format, cache identity, and runtime dependencies are unchanged from 1.1.0. Engines saved by 1.1.0 are reused without a rebuild.

Applications that build cubins with the producer extra need a CUDA toolkit whose ptxas is 13.2 or later and earlier than 14. Set CUDA_HOME to that toolkit or install it at /usr/local/cuda. Applications that only load and run cubins do not need the toolkit. The CUDA driver must be new enough to load cubins built by the ptxas version in use.

Install

pip install tensoracle==1.2.0

v1.1.0

Changes

• Tensoracle now supports recording one EngineExecution.enqueue call with CUDA stream capture and replaying the resulting graph. The README and the documentation describe the requirements: enqueue once and wait after setting input shapes, capture and replay on a nondefault stream, and use a dedicated execution unit for each graph. They also state that a graph fixes the input shapes, tensor addresses, and execution unit, and which engines cannot be captured. The application owns graph creation, recapture, and destruction; tensoracle adds no capture or replay operations.
• GPU tests verify that replayed graphs match ordinary enqueue output for an engine with standard layers, an engine with a cubin TensorRT plugin, and a ranged-input engine that keeps one execution unit and graph per shape.

Upgrade note

The code, public API, saved engine file format, cache identity, and dependencies are unchanged from 1.0.1. Engines saved by 1.0.1 are reused without a rebuild. Applications that capture enqueue into CUDA graphs can require tensoracle>=1.1.0 to depend on this guarantee.

Install

pip install tensoracle==1.1.0

v1.0.1

Changes

• The package metadata now declares Development Status :: 5 - Production/Stable.
• The README lists the GPU properties in the saved engine identity: the product name, compute capability, memory bus width, L2 cache size, shared memory limits, texture alignment, multiprocessor count, whether the GPU is integrated, ECC mode, and total memory of the current CUDA context's device (or device 0). Another device with the same properties reuses the saved engine.
• The optimization workflow applies the stage 3 quality gate to every lower precision engine (fp16, bf16, int8, and fp8).
• The API reference includes ShapeTensorOutputWarning.
• The cubin plugin example names the Windows plugin DLL for inspect_wheel, and the README states that the bundle manifest records the Triton and ptxas versions and each kernel's source SHA-256.

Upgrade note

The code, public API, saved engine file format, cache identity, and dependencies are unchanged from 1.0.0. Engines saved by 1.0.0 are reused without a rebuild.

Install

pip install tensoracle==1.0.1

v1.0.0

Changes

• Windows x86_64 support: tensoracle now runs on Windows x86_64 as well as Linux x86_64 (including WSL2), with the same TensorRT 11 series and GPU generations (sm86, sm89, sm120). The same py3-none-any wheel installs on both operating systems.
• On Windows, a saved engine lives in %USERPROFILE%\.cache\tensoracle when XDG_CACHE_HOME is unset. The rule is the same on every OS: the tensoracle directory under XDG_CACHE_HOME, or under the home directory's .cache.
• The CMake helper tensoracle_add_cubin_plugin finds nvinfer_11.lib on Windows, so a cubin plugin builds as a Windows DLL. Cubins do not depend on the OS, so a DLL built on Windows can embed cubins produced on Linux with tensoracle[producer].
• inspect_bundle and inspect_wheel now reject Windows drive and root paths (such as C:\... or \...) in cubin and plugin paths, in addition to absolute POSIX paths and ...
• The documentation now covers setting up the TensorRT SDK on Windows: TensorRT SDK. A new guide covers source layout, development environments, bindings, and Linux wheel packaging: Build a C++ and Python repository.

Tested on Windows

The test suite (246 passed, 20 skipped) and a native plugin build passed on Windows 11 x64 with an sm89 GPU, Python 3.12, Visual Studio 2022, and TensorRT 11.4. Windows with sm86 or sm120, and Python 3.13 on Windows, have not been tested. See the README for details.

Upgrade note

The public API, the saved engine file format, the cache identity, and the dependencies are unchanged on Linux. Engines saved by 0.5.1 are reused without a rebuild.

Install

pip install tensoracle==1.0.0

v0.5.1

Changes

• Fix: A saved engine is now keyed by the GPU attributes that TensorRT uses to decide engine compatibility, not only by the CUDA device name. The key adds compute capability, global memory bus width, L2 cache size, shared memory limits, texture alignment, multiprocessor count, integrated or discrete, ECC state, and total memory. Two GPUs with the same name that TensorRT treats as different devices (for example, the same model with ECC enabled on one and disabled on the other) no longer reuse each other's saved engine. Before this fix, opening such an engine could make TensorRT warn Using an engine plan file across different models of devices is not supported ....
• If tensoracle cannot read one of these GPU attributes, open_engine raises RuntimeError instead of choosing a saved engine with an incomplete key.
• GPUs whose attributes are all the same still share one saved engine. The device index, UUID, and PCI bus ID are not part of the key.
• The documentation now covers bf16, int8, and fp8 engines: Build bf16, int8, and fp8 engines.

Upgrade note

The saved engine key has changed, so engines saved by an earlier version are rebuilt once on first use after the upgrade. The public API, the saved engine location, and the dependencies are unchanged.

Install

pip install tensoracle==0.5.1

v0.5.0

Changes

• Breaking: EngineBuild no longer takes a precision. The precision of an engine is decided only by the tensor types that the graph function places in the network. To keep engines of different precisions apart in the saved-engine cache, give them different engine names or graph versions.
• The repository now has a top-level cpp/ and python/ layout. cpp/ is a standalone CMake project for the C++ cubin plugin helper, and it builds and runs its tests through CTest without Python. python/ is the root of the Python project and its uv environment. The README section Build a cubin plugin asks repositories that adopt tensoracle to use the same layout.
• The wheel is unchanged in shape: it is py3-none-any, has no native code, and still ships tensoracle/cpp/cmake, tensoracle/cpp/include, and tensoracle/cpp/src for plugin builds.
• The plugin requirements now state the minimum CMake version, 3.20.
• The documentation site is built and deployed by GitHub Actions in this repository.

Install

pip install tensoracle==0.5.0

Projects that install tensoracle from a Git checkout instead of PyPI need subdirectory = "python" from this version on.

v0.4.1

Changes

• The Python API, the C++ headers, and the wheel contents are unchanged from 0.4.0. Only the version number differs.
• The example plugin tests allow more time to install the example wheel, so they pass on a slow network link and with an empty uv cache.
• The source distribution tests check that release settings stay out of the sdist, and that the wheel metadata contains no entry from the release word list.
• Test markers now name the specification items they verify, and the tests for the former snapshot script are removed.

Install

pip install tensoracle==0.4.1

Documentation: https://sync-dev-org.github.io/tensoracle/

v0.4.0

Changes

• tensoracle::cubin::PluginBase accepts a PluginIO declaration as its fourth constructor argument. PluginIO::inputs lists one TensorSpec (TensorRT type and format) per input in network order, PluginIO::output declares the single output's type and format, and shape_input_index selects the input whose shape the output copies. The base class uses the declaration for format negotiation, output types and shapes, and the count, type, and format checks during configuration and shape changes. A plugin can now declare, for example, two fp16 / HWC8 inputs and one fp16 / HWC8 output.
• Omitting the declaration keeps the previous behavior: one fp32 / LINEAR input and output, with the output shaped like input 0. The three-argument constructor call remains valid.
• Source change for derived plugins: validate_shape now receives every input's dimensions: validate_shape(nvinfer1::Dims const* inputs, std::int32_t input_count, nvinfer1::Dims const& output) const noexcept. Plugins that override the previous validate_shape(input, output) must update the signature.
• For configurations the declaration cannot express, such as multiple outputs or computed output shapes, developers can override the virtual V3 methods in a derived class or ask tensoracle's maintainers to extend the declaration.
• The documentation explains how graph_version covers caller values outside the cache identity.

The Python API is unchanged.

Install

pip install tensoracle==0.4.0

Documentation: https://sync-dev-org.github.io/tensoracle/