[AAAI 2023 (Oral)] CoordFill:
Efficient High-Resolution Image Inpainting via Parameterized Coordinate Querying

Weihuang Liu ^1,2   Xiaodong Cun ^*,2  Chi-Man Pun ^*,1  Menghan Xia ²  Yong Zhang ²  Jue Wang¹  

¹ University of Macau   ² Tencent AI Lab  

🎼 Pipeline

Giving an input masked image, first, we resample the input image with a fixed resolution before fed to the network, which enables the contextual feature extraction under a unified receptive field. Then, we select the masked region features by the corresponding coordinates. For each spatial feature, we generate the required number of parameters by the linear mapping function, and the parameters are upsampled to the required resolution with nearest-neighbor interpolation. Next, we use the generated parameters to parameterize the pixel-wise querying network, Hence, we obtain a series of MLPs that are spatial-adaptive. Finally, the pixel-wise querying network takes the hole pixel’s coordinate as input and outputs the color values.

Environment

This code was implemented with Python 3.6 and PyTorch 1.8.1. You can install all the requirements via:

pip install -r requirements.txt

Demo

python demo.py --input [INPUT_PATH] --mask [MASK_PATH] --config [CONFIG_PATH] --model [MODEL_PATH] --resolution [HEIGHT],[WIDTH]

[INPUT_PATH]: input image

[MASK_PATH]: input mask

[CONFIG_PATH]: config file

[MODEL_PATH]: backbone checkpoint

[HEIGHT]: target height

[WIDTH]: target width

Train

Single GPU training:

python train.py --config [CONFIG_PATH]

Multi GPU training:

python -m torch.distributed.launch --nproc_per_node=[NUM_GPU] train_parallel.py --config [CONFIG_PATH]

Test

python test.py --config [CONFIG_PATH] --model [MODEL_PATH]

Models

Please find the pre-trained models here.

🛎 Citation

If you find our work useful in your research, please consider citing:

@inproceedings{liu2023coordfill,
  title={CoordFill: Efficient High-Resolution Image Inpainting via Parameterized Coordinate Querying},
  author={Liu, Weihuang and Cun, Xiaodong and Pun, Chi-Man and Xia, Menghan and Zhang, Yong and Wang, Jue},
  booktitle={AAAI},
  year={2023}
}

💗 Acknowledgements

CoordFill code borrows heavily from LIIF, ASAPNet and LAMA. We thank the author for sharing their wonderful code.