Deep Fusion Network for Image completion

Info about fork:

• It's pretty alpha and not very flexible, but works.
• Testing/Training in Colab possible, since training Code is available in Yukariin/DFNet. Thank you Yukariin.
• Attempt to add Differentiable Augmentation.
• TensorboardX
• Added loss functions.
• Added image metrics.

Introduction

Deep image completion usually fails to harmonically blend the restored image into existing content, especially in the boundary area. Our method handles with this problem from a new perspective of creating a smooth transition and proposes a concise Deep Fusion Network (DFNet). Firstly, a fusion block is introduced to generate a flexible alpha composition map for combining known and unknown regions. The fusion block not only provides a smooth fusion between restored and existing content, but also provides an attention map to make network focus more on the unknown pixels. In this way, it builds a bridge for structural and texture information, so that information can be naturally propagated from known region into completion. Furthermore, fusion blocks are embedded into several decoder layers of the network. Accompanied by the adjustable loss constraints on each layer, more accurate structure information are achieved. The results show the superior performance of DFNet, especially in the aspects of harmonious texture transition, texture detail and semantic structural consistency. More detail can be found in our paper

The illustration of a fusion block:

Examples of corresponding images:

If you find this code useful for your research, please cite:

@inproceedings{xin2019dfnet,
  title={Deep Fusion Network for Image Completion},
  author={Xin Hong and Pengfei Xiong and Renhe Ji and Haoqiang Fan},
  journal={arXiv preprint},
  year={2019},
}

Prerequisites

• Python 3
• PyTorch 1.0
• OpenCV

Testing

Clone this repo:

git clone https://github.com/hughplay/DFNet.git
cd DFNet

Download pre-trained model from Google Drive and put them into model.

Testing with Places2 model

There are already some sample images in the samples/places2 folder.

python test.py --model model/model_places2.pth --img samples/places2/img --mask samples/places2/mask --output output/places2 --merge

Testing with CelebA model

There are already some sample images in the samples/celeba folder.

python test.py --model model/model_celeba.pth --img samples/celeba/img --mask samples/celeba/mask --output output/celeba --merge

License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.