Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support JSONL format #1960

Open
johann-petrak opened this issue Jan 25, 2022 · 2 comments
Open

Support JSONL format #1960

johann-petrak opened this issue Jan 25, 2022 · 2 comments
Assignees
Labels

Comments

@johann-petrak
Copy link

For larger files and generally for easier handling of example sets in json format, it would be very useful if JSONL format was supported:

  • each example a JSON map, on its own line
  • no enclosing array

In other words, such files would get processed line by line, with each line getting converted to/from json as needed on export/import.

See https://jsonlines.org/

@makseq
Copy link
Member

makseq commented Jan 25, 2022

Are you talking about LS storages with jsonl import? Or about the common LS import files?

@makseq makseq added feature Feature request import labels Jan 25, 2022
@johann-petrak
Copy link
Author

Sorry, what I meant is for

  • uploading examples to annotate (Import button within a project)
  • downloading annotated examples (Export button within a project)

jsonl has become an often-used format in nlp because it is flexible, yet it is, unlike a json array file, line oriented, so other tools can be used to filter, update. sort, etc examples in jsonl files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants