100% pure Rust inference wrapper for the GPT-2 (and possibly later) model family.

Getting a GPT Model

The GPT-2 model packaged within the crate's repository uses the original model trained by OpenAI, with minor modifications to support Tensorflow 2.0, and to support conversion to the ONNX model format.

When getting started with this crate, we recommend using our prebuilt version of the 124M (smallest) GPT-2 model:

• The model's encoder, hyper parameters, and vocabulary ship with this repository and can be found in gpt-2-model/saved_models.
• The prebuilt ONNX model can be downloaded from here, and should be saved to gpt-2-model/saved_models

Repository Structure

• src/: Main crate contents, including a pure Rust implementation of the GPT-2 byte-pair encoder (tokenizer) and a Rust wrapper for loading and invoking an ONNX GPT-2 model.
• gpt-2-model/: Python scripts and Docker files to download and export Tensorflow and ONNX versions of the GPT-2 model.
• gpt-2-model/saved_models: Exported GPT-2 models.

License

Except where otherwise noted, this project is Copyright (C) 2022-2025 With Caer, LLC and licensed under the MIT license. Refer to the license file for more info.

The files within the gpt-2-model directory are Copyright (C) 2019 OpenAI and (C) 2022-2025 With Caer, LLC, and licensed under an MIT-style license.