Patent Image Retrieval

Summary

The repository contains a Patent Image Retrieval, Transformer backbone, deep metric learning, and preprocessing in Python/PyTorch.

The repository is based on the paper: Kotaro Higuchi, Keiji Yanai, Patent Image Retrieval Using Transformer-based Deep Metric Learning, World Patent Information(WPI) 2023. [Abstract(WPI)]

Update(Dec 17, 2023)

• Our pre-trained models are available at this following [link].

Update(Jul 21, 2023)

• This project was accepted to World Patent Information. We achieved the State-Of-The-Art(SOTA) score of mAP in patent image retrieval. [Abstract(WPI)] [Code]
• Pre-training & Fine-tuning codes
• DeepPatent link [Project]

Update(Feb 1, 2023)

• Initial project "Patent Image Retrieval using Cross-entropy-based Metric Learning" was released. The paper was accepted to International Workshop on Frontiers of Computer Vision(IW-FCV). We achieved to search patent image without traditional IPC or CPC. [PDF(IW-FCV)]

Citation

If you use this code, please cite the following paper:

@article{HiguchiWPI2023,
  title = {Patent image retrieval using transformer-based deep metric learning},
  journal = {World Patent Information},
  volume = {74},
  pages = {102217},
  year = {2023},
  doi = {https://doi.org/10.1016/j.wpi.2023.102217},
  author = {Kotaro Higuchi and Keiji Yanai},
}

@inproceedings{HiguchiIWFCV2023,
  author={Higuchi, Kotaro and Honbu, Yuma and Yanai, Keiji},
  title={Patent Image Retrieval Using Cross-entropy-based Metric Learning},
  article={International Workshop on Frontiers of Computer Vision(IW-FCV)},
  year={2023},
}

Requirements

• PyTorch > 1.6.0
• torchvision > 0.7.0
• Pytorch Metric Learning
• CUDA
• CuDNN
• FAISS
• OpenCV
• Graphic board (worked at NVIDIA A6000*8)

Instllation

Requisites

• PyTorch Metric Learning
  • conda install -c conda-forge pytorch-metric-learning
• FAISS
  • conda install -c conda-forge faiss-gpu

Run

• Jupyter Notebook
  • Run this Notebook file on Ubuntu 22.04 LTS

Note

• This repository contains the legendary version of the ArcFace implementation with the Python interface. Code
• We tested this notebook with Python3, on Ubuntu.
• Patlist is from DeepPatent dataset. It teaches the path to the substantial dataset.

Dataset

• If you would like to fine-tune on an conventional dataset, you must prepare the DeepPatent dataset. [Project]
• Same test/val/train split as the DeepPatent dataset

Models

• Our pre-trained models are available at this following [link].

Terms of use

The authors affiliated in The University of Electro-Communications(UEC) are not responsible for the reproduction, duplication, copy, sale, trade, resell or exploitation for any commercial purposes, of any portion of the images and any portion of derived the data. In no event will we be also liable for any other damages resulting from this data or any derived data.