Could we use LLaVA to extract information from a given photo?

[nonstopnht]

Hi LLaVA team, thanks for the great work, it is really impressive of what you have done!
I am trying to understand what we could do with LLaVA, and to extract information from a photo but the answer is a bit interesting.

While LLaVA is able to tell the uploaded photo is a certificate of incorporation, but it is telling a wrong UEN, is it due to the training is not enough or the limitation of transformer? Thank you for your great effort!

[haotian-liu]

Hi, thank you for your interest in our work. The current LLaVA is not good at parsing the details of high resolution images, as the current input resolution is only 224x224. It is also partially due to that these OCR tasks is not very common in our dataset. We are working on these improvements, stay tuned!

[nonstopnht]

Thank you very much for your reply! Looking forward to seeing it, let me know if I can contribute. (I am new to AI, but very interested in it)

[haotian-liu]

@nonstopnht Of course, contributions are always welcomed! If you find anything that you are interested in improving, feel free to let me know :) Thanks!