Skip to content

v0.3.1

Choose a tag to compare

View all tags
@minamikik minamikik released this 11 Oct 09:17
· 14 commits to main since this release
v0.3.1
c2f6deb

First public release of tensoracle.

tensoracle builds TensorRT engines from safetensors weights and a TensorRT network API graph, saves them under the user cache directory, reuses them when the build conditions match, and executes them on caller-managed device buffers.

Install

pip install tensoracle

Optional extras: tensoracle[convert] (convert .pth checkpoints to safetensors; installs torch) and tensoracle[producer] (generate cubins for TensorRT plugins).

Highlights

  • Build fixed-shape engines and engines with min / opt / max input ranges, in fp32 (TF32 disabled) or fp16
  • Save and reuse engines keyed by weights, graph version, precision, shapes, builder settings, plugins, GPU, OS, CPU architecture, and TensorRT version; detect corrupted or mismatched engine files and rebuild on request
  • Check engine inputs and outputs against the caller's declaration when opening an engine
  • Execute engines through EngineHandle.create_execution() with caller-owned buffers and CUDA streams
  • Compare engine outputs with reference outputs using shared metrics
  • Build and inspect cubin-based TensorRT plugins for sm86, sm89, and sm120

Supported environment

  • Linux x86_64 (including WSL2), Python 3.12 and 3.13
  • TensorRT 11 (tensorrt-cu13), GPUs sm86, sm89, and sm120

Documentation: https://sync-dev-org.github.io/tensoracle/

Assets 4
Loading