Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About rvl_cdip_dataset.csv #29

Closed
abharika725 opened this issue Apr 8, 2022 · 6 comments
Closed

About rvl_cdip_dataset.csv #29

abharika725 opened this issue Apr 8, 2022 · 6 comments

Comments

@abharika725
Copy link

Hi,

Can you please provide code to generate the csv file used in the examples folder in dataset_creation_for_docformer.py file

@uakarsh
Copy link
Collaborator

uakarsh commented Apr 8, 2022

Hi there,

I have provided an answer to this question, here

And, for the usage of the docformer (i.e to use the DocFormer model), I have also provided a sample notebook, which describes how to train docformer for the purpose of Mask Language Modeling, and with some tweaks, you can extend it to multiple tasks.

Here, is the link for the same link to notebook. Please ignore that, file i.e dataset_creation_for_docformer.py, as it was a sample file, and we would remove it shortly.

Do feel free to ask, for more queries.

Regards,
Akarsh

@abharika725
Copy link
Author

Thanks for the reply Akarsh. I want to use the docformer code for the classification problem on rvl-cdip dataset.

@uakarsh
Copy link
Collaborator

uakarsh commented Apr 8, 2022

For that, the things are simple, just have a look at the MLM colab notebook, and then make the argument used for masking the input ids to be false, and in the modelling part, change the dimension of the last layer i.e the classification layer to be equal to the num of classes, and then you can go for training. However, you need to make sure, you have the OCR for RVL-CDIP since, it is a large dataser, else it would take time for generating OCR.

Feel free to ping for more help.

Regards,

@abharika725
Copy link
Author

Thanks for the reply. I will try this and get back to you.

@yuni1314
Copy link

Hello author:
How to understand the output['ir'] in "criterion2(output['ir'],labels2)" at train_accelerator_mlm_ir.py, it comes from a decoder or other? If the answer is a decoder, can tell us how it was written? In addition, the label2 is the resized_scaled_img(e.g., the pixel is 384*500)? Thans!

@uakarsh
Copy link
Collaborator

uakarsh commented Apr 19, 2022

Actually, the architecture of the decoder is not mentioned, so earlier I implemented just a simple conv->deconv architecture, but I think, you would have to implement your own decoder, as per your usage. Just take the output of the encoder of DocFormer, and then pass it through Conv, and then Upsample, and I hope that works. Do let me know if there is something more. And yes, earlier the size of the image was 224 x 224 but, now it is 500 x 384

@uakarsh uakarsh closed this as completed Jun 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants