Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature Request: Example of how best to use with datasets #479

Closed
jamesmf opened this issue Oct 22, 2020 · 1 comment
Closed

Feature Request: Example of how best to use with datasets #479

jamesmf opened this issue Oct 22, 2020 · 1 comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request

Comments

@jamesmf
Copy link

jamesmf commented Oct 22, 2020

It'd be helpful if there was a suggestion in the docs of how to best use tokenizers and datasets together. I can see why the two have different default formats, but an example of might be helpful - even if it's just something that explains that you have to convert from the arrow format to a text file.
@n1t0 n1t0 added documentation Improvements or additions to documentation enhancement New feature or request labels Oct 22, 2020
@fraboniface fraboniface mentioned this issue Nov 4, 2020
Closed
@Narsil
Copy link
Collaborator

Narsil commented Nov 10, 2020

Closing in favor of #198 to keep the discussion centralised (PR open at #512 )

@Narsil Narsil closed this as completed Nov 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Narsil @n1t0 @jamesmf