Skip to content

Commit

Permalink
Document RBS usage
Browse files Browse the repository at this point in the history
  • Loading branch information
drnic committed Oct 16, 2023
1 parent ebf888a commit a547319
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# tiktoken_ruby

[Tiktoken](https://github.com/openai/tiktoken) is BPE tokenizer from OpenAI used with their GPT models.
This is a wrapper around it aimed primarily at enabling accurate counts of GPT model tokens used.
This is a wrapper around it aimed primarily at enabling accurate counts of GPT model tokens used.

## Installation

Expand Down Expand Up @@ -33,6 +33,12 @@ enc = Tiktoken.encoding_for_model("gpt-4")
enc.encode("hello world").length #=> 2
```

## RBS support

Tiktoken comes with Ruby type signatures (RBS).

To use them with Steep, add `library "tiktoken_ruby"` to your `Steepfile`.

## Development

After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
Expand Down

0 comments on commit a547319

Please sign in to comment.