Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In jp.jsonl, all the question is "null", is there any mistake? #5

Open
Labmem009 opened this issue Mar 19, 2024 · 3 comments
Open

In jp.jsonl, all the question is "null", is there any mistake? #5

Labmem009 opened this issue Mar 19, 2024 · 3 comments

Comments

@Labmem009
Copy link

Some lines are "T2T" type, but still have no question, is there any mistake?

@zhongshsh
Copy link
Collaborator

zhongshsh commented Mar 19, 2024

Not a mistake. Some T2T type questions have placed text on images, for example,

Thank you for the reminder, we've added clarification to the dataset card.

@Labmem009
Copy link
Author

Not a mistake. Some T2T type questions have placed text on images, for example, image

But all the "T2T" lines have "null" question, I checked that there is no line has question text

@zhongshsh
Copy link
Collaborator

zhongshsh commented Mar 19, 2024

This is because Japanese data mainly comes from Bokete, where T2T is displayed in image form.

This type of data can also be labeled as T2T because, as shown in the following image (chat with Qwen-VL), well-formatted text within images can be easily recognized and understood by multimodal large language models.

You also can find data where the question is not 'None' in the Chinese and English data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants