Finetuning Donut on FUNSD dataset

[satheeshkatipomu]

Hi,

Thank you for open sourcing DONUT and SynthDoG. I have two requests.

1. After Pre-Training("how to read"/pseudo-OCR task), is there a documentation about how to finetune("how to understand") on a different dataset like FUNSD?
2. Can we generate synthetic documents resembling forms/invoices using SyntheticDoG, if yes can you provide hints whether we need a template or something?

[gwkrsrch]

Hi, thank you for your interest on our work :)

For (1), I hope the following links would be helpful,

• https://github.com/clovaai/donut#for-document-information-extraction
• https://arxiv.org/abs/2111.15664
  • especially, Section 2.4 and Appendix A.4 and Figure E.

For (2), I would like to say that the SynthDoG's purpose is not to create synthetic data resembling actual forms/invoices. The purpose is just to create a simple synthetic document with texts. The link below seems to be helpful to you.

• https://github.com/tuanpham-dev/react-invoice-generator
• https://github.com/amoffat/metabrite-receipt-tests

Hope this helps. Please let me know if you are still confused.