bbpe: Add token LRU cache and avoid string lookups

[danieldk]

Description

This change combines some performance improvemets:

• We now cache token -> ids lookup in an LRU cache of size 8192.
• Represent merges as a mapping (piece_id, piece_id) -> piece_id rather than a mapping of strings. This speeds up the C++ code considerably, making encoding faster with a fresh or useless cache.

Also add a small benchmark.

I think the merges files could also use a clang-format, but let's do that in a separate PR to keep the diff clean.

Types of change

Performance improvements.

Checklist

• I confirm that I have the right to submit this contribution under the project's MIT license.
• I ran the tests, and all new and existing tests passed.
• My changes don't require a change to the documentation, or if they do, I've added all required information.

[shadeMe]

Just a couple of minor fixes.