Skip to content

Automatically call removeEmpty for all tokenizer calls #64

@kpdecker

Description

@kpdecker

Right now implementors need to manually perform something like removeEmpty. This should be done automatically (but overridable) by the logic calling tokenize.

Issue raised in #63

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions