Document image classification with neural networks on a subset of the RVL-CDIP dataset [1].
The classification problem is tackled with three different approaches:
- Visual approach over the image pixels with dense only and convolutional neural networks:
chapter_1_vision.ipynb - Textual approach over the recognized image words with bag-of-words, word embedding models and pre-trained transformers:
chapter_2_text.ipynb - Lazy approach, using a multimodal LLM to perform the classfication without any finetuning. Time to practice prompt engineering techniques:
chapter_3_vllm.ipynb
It is recommended to begin with the visual approach as it includes more details about the computing environment setup and the dataset.
For a better experience, execute the notebooks within a Google Colab environment.
- Thibault Douzon - thibaultdouzon
- Jérémy Espinas
- Clément Sage - clemsage
- Bertrand Buffat - berbuf
[1] A. W. Harley, A. Ufkes, K. G. Derpanis, "Evaluation of Deep Convolutional Nets for Document Image Classification and Retrieval," in ICDAR, 2015