PHP BPE Encoder Decoder for GPT-2/GPT-3
GPT-2 and GPT-3 use byte pair encoding to turn text into a series of integers to feed into the model. This is a PHP implementation of OpenAI's original python encoder/decoder which can be found here.
Requires PHP 7.4+
Install GPT-3-Encoder via the Composer package manager
composer require rahul900day/gpt-3-encoderEncoding a text to tokens
use Rahul900day\Gpt3Encoder\Encoder;
Encoder::encode("Your prompt.");Decoding to text from tokens
use Rahul900day\Gpt3Encoder\Encoder;
Encoder::decode([8582, 242, 98]);This packages has some codes and test inspiration from node's gpt-3-encoder
This package is released under the MIT License.
