Skip to content

overflow_to_sample_mapping missing in in documentation #9059

@schelv

Description

@schelv

In the documentation of the fast tokenizer, the overflow_to_sample_mapping field is missing.
Instead the overflowing_tokens is listed there, which is only part of the base tokenizer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    WIPLabel your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progress

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions