Permalink
Browse files

update readmes

  • Loading branch information...
youyuge34 committed Jan 10, 2019
1 parent 2cd55eb commit dd0431a11c46abde760f20f4e8644714b19f07d2
Showing with 154 additions and 24 deletions.
  1. +7 −8 README.md
  2. +15 −15 scripts/flist_train_split.py
  3. +132 −1 training_manual.md
@@ -1,9 +1,9 @@
Anime-InPainting: An application Tool based on [Edge-Connect](https://github.com/knazeri/edge-connect) (WIP)
------------------------------------------------------------------------------------------------------------
Anime-InPainting: An application Tool based on [Edge-Connect](https://github.com/knazeri/edge-connect)
------------------------------------------------------------------------------------------------------
<p align="left">
<img src="https://img.shields.io/badge/version-0.2-brightgreen.svg?style=flat-square"
alt="Version">
<img src="https://img.shields.io/badge/status-WIP-orange.svg?style=flat-square"
<img src="https://img.shields.io/badge/status-Release-gold.svg?style=flat-square"
alt="Status">
<img src="https://img.shields.io/badge/platform-win | linux-lightgrey.svg?style=flat-square"
alt="Platform">
@@ -94,7 +94,7 @@ from the original work [Edge-Connect](https://github.com/knazeri/edge-connect).
Only one thing to be careful: The `config.yml` in this project has some additional options than the config from the [Edge-Connect](https://github.com/knazeri/edge-connect).

## tool operation
## Tool operation
For detailed manual, refer to your `terminal` prints or the `__doc__` in `tool_patch.py`.

Below is the simplified tool operation manual:
@@ -119,9 +119,8 @@ Key `q` | To quit
Click here --> [Training manual by yourself](training_manual.md)


> continuous...

<span id="jump_zh">中文版介绍🇨🇳 (WIP)</span>
<span id="jump_zh">中文版介绍🇨🇳 </span>
-----

## 简介
@@ -178,7 +177,7 @@ python tool_patch.py -h
## Tool操作指南
详细内容请翻看控制台的打印内容,或查看`tool_patch.py`里的`__doc__`
简略版Tool使用指南
简略版tool使用指南

按键 | 说明
-----|------
@@ -196,7 +195,7 @@ python tool_patch.py -h


## 训练指南
训练指南 --> [阅读](training_manual.md)
训练指南 --> [阅读](training_manual.md#jump_zh)


## License
@@ -21,25 +21,25 @@
print('loading ' + root)
for file in files:
if os.path.splitext(file)[1] in ext:
path = os.path.join(root, file)
if num % total > (args.val + args.test) or num % total == 0:
images_train.append(path)
elif num % total <= args.val and num % total > 0:
images_val.append(path)
else:
images_test.append(path)
num += 1
path = os.path.join(root, file)
if num % total > (args.val + args.test) or num % total == 0:
images_train.append(path)
elif num % total <= args.val and num % total > 0:
images_val.append(path)
else:
images_test.append(path)
num += 1

images_train.sort()
images_val.sort()
images_test.sort()

print('train number =',len(images_train))
print('val number =',len(images_val))
print('test number =',len(images_test))
print('train number =', len(images_train))
print('val number =', len(images_val))
print('test number =', len(images_test))

if not os.path.exists(args.output):
os.mkdir(args.output)
np.savetxt(os.path.join(args.output,'train.flist'), images_train, fmt='%s')
np.savetxt(os.path.join(args.output,'val.flist'), images_val, fmt='%s')
np.savetxt(os.path.join(args.output,'test.flist'), images_test, fmt='%s')
os.mkdir(args.output)
np.savetxt(os.path.join(args.output, 'train.flist'), images_train, fmt='%s')
np.savetxt(os.path.join(args.output, 'val.flist'), images_val, fmt='%s')
np.savetxt(os.path.join(args.output, 'test.flist'), images_test, fmt='%s')
@@ -1 +1,132 @@
Todo
Training Manual
---------------
<p align="left">
<img src="https://img.shields.io/badge/version-0.2-brightgreen.svg?style=flat-square"
alt="Version">
<img src="https://img.shields.io/badge/status-WIP-orange.svg?style=flat-square"
alt="Status">
<img src="https://img.shields.io/badge/platform-win | linux-lightgrey.svg?style=flat-square"
alt="Platform">
<img src="https://img.shields.io/badge/PyTorch version-1.0-blue.svg?style=flat-square"
alt="PyTorch">
<img src="https://img.shields.io/badge/License-CC BY·NC 4.0-green.svg?style=flat-square"
alt="License">
</p>

English | [中文版](#jump_zh)

## Introduction
The whole training is not end-to-end which must be split into several phases to get a best performance.
So, reviewing the paper and codes structure will make you understand the training phase better.

As the paper shows, the whole model needs two separate models: `EdgeModel` and `InpaintingModel`.
But in practice, the whole work actually needs **three training phases** with **the two separate models**
, which makes results best while the training is confusing.

**IMPORTANT**: The three training phases I define here are called `models` in the original codes , which could be confused with `EdgeModel` and `InpaintingModel`.

Phase | Command | Model | Input | Output | Description
-----|-------|------|------|-------|-------
1st | --MODEL 1 | `EdgeModel` | Masked Greyscale Image + Masked Edge + Mask | Full Edge | -
2nd | --MODEL 2 | `InpaintingModel` | Masked Image + Full canny Edge from Original full Image+ Mask | Full Image | This phase is added to make the `InpaintingModel` learning the importance of edges
3rd | --MODEL 3 | `InpaintingModel` | Masked Image + Full Edge from 1st phase output + Mask | Full Image | -

## Dataset
1. We need to prepare images dataset and masks dataset both.
- Mask dataset:
- Irregular Mask Dataset ([download link](http://masc.cs.gmu.edu/wiki/uploads/partialconv/mask.zip)) provided by [Liu et al.](http://masc.cs.gmu.edu/wiki/partialconv) is recommended to handle with normal irregular defects.
- Block Masks don't need dataset which can be random generated by codes.
- Image dataset:
- Places2, CelebA and Paris Street-View datasets are [here](https://github.com/knazeri/edge-connect#datasets).
- Anime Face dataset from `getchu.com` I used: [ANIME305](https://github.com/ANIME305/Anime-GAN-tensorflow#open-sourced-dataset)

2. We should split the whole image dataset into train/validation/test parts.
```bash
python scripts/flist_train_split.py --path <your dataset directory> --output <output path> --train 28 --val 1 --test 1
```
This script will split 30 images into 28 for train, 1 for validation and 1 for test.
Images are split by order of names instead of shuffle, in order to get a best time-average-distribution dataset.
Now there should be three `.filst` file in your `<output path>`, which conclude absolute image paths.

3. Copy the `config.yml.example` under root directory into your model path. Rename it into `config.yml` and edit it.
Here is some key parameters related to dataset:
- Edit the parameter `MASK: 3`(recommended as above, 4 is also feasible).
- Edit the parameter `TRAIN_FLIST`, `VAL_FLIST` and `TEST_FLIST` into your `.flist` path which are got in step 2.
- Edit the parameter `TRAIN_MASK_FLIST`, `VAL_MASK_FLIST` and `TEST_MASK_FLIST` into the same mask dataset path as we got in step 1.

Now my `config.yml` is:
```
MODE: 1 # 1: train, 2: test, 3: eval
MODEL: 1 # 1: edge model, 2: inpaint model, 3: edge-inpaint model, 4: joint model
MASK: 3 # 1: random block, 2: half, 3: external, 4: (external, random block), 5: (external, random block, half)
EDGE: 1 # 1: canny, 2: external
NMS: 1 # 0: no non-max-suppression, 1: applies non-max-suppression on the external edges by multiplying by Canny
SEED: 10 # random seed
DEVICE: 1 # 0: CPU, 1: GPU
GPU: [0] # list of gpu ids
DEBUG: 1 # turns on debugging mode
VERBOSE: 0 # turns on verbose mode in the output console
SKIP_PHASE2: 1 # When training Inpaint model, 2nd and 3rd phases (model 2--->model 3 ) by order are needed. But we can merge 2nd phase into the 3rd one to speed up (however, lower performance).
TRAIN_FLIST: <your path>/train.flist
VAL_FLIST: <your path>/val.flist
TEST_FLIST: <your path>/test.flist
TRAIN_EDGE_FLIST: ./
VAL_EDGE_FLIST: ./
TEST_EDGE_FLIST: ./
# three options below could be the same
TRAIN_MASK_FLIST: <your mask dataset path>
VAL_MASK_FLIST: <your mask dataset path>
TEST_MASK_FLIST: <your mask dataset path>
```

## Training Prepare
- Download weights files: Which are available in [my page](README.md#run-the-tool) and [edge-connect](https://github.com/knazeri/edge-connect#2-testing)
- Strongly recommend you to start transfer learning with weight files. Otherwise you need about 10 days 2 million iterations training to coverage from scratch.
- mkdir a model path which contains the `config.yml` and four `.pth` weights files.
- edit the options in `config.yml` related to training:
- Edit the parameter `DEVICE: 1` which is a new option to use GPU or not.
- Edit the parameter `GPU: [0]` to act a multi-gpu training.
- Edit the following options as u wish:
```
SAVE_INTERVAL: 1000 # how many iterations to wait before saving model (0: never)
SAMPLE_INTERVAL: 200 # how many iterations to wait before sampling (0: never)
SAMPLE_SIZE: 12 # number of images to sample
EVAL_INTERVAL: 0 # how many iterations to wait before model evaluation (0: never)
LOG_INTERVAL: 1000 # how many iterations to wait before logging training status (0: never)
PRINT_INTERVAL: 20 # how many iterations to wait before terminal prints training status (0: never)
```

## Training
Before training, there are two training optimizations in my work you must know:
- Add a skip phase 2 optional mode which can combine phase 2 and phase 3 together, in order to accelerate. If you cannot understand what it means, refer to the Introduction above.
- Feel free about the checkpoints problems, the new checkpoint files are saved in your same model path.
They are named followed by a iteration mark, e.g. `InpaintingModel_dis_2074000.pth`.
Also the latest checkpoints (identified by name) will be auto-load when the training begins.

### Faster Training steps
1. Train phase 1 which trains the `EdgeModel`.
```bash
python train.py --model 1 --path <your model dir path>
```
Check the samples at times and stop the training by yourself.

2. Train phase 2 and 3 together which trains the `InpaintingModel` using the well-trained `EdgeModel` in step 1.

**IMPORTANT: `SKIP_PHASE2` should be `1` in `config.yml`!**
```bash
python train.py --model 3 --path <your model dir path>
```
Check the samples at times and stop the training by yourself.
That's all!

### (optional) Advanced Training steps
- You can set `SKIP_PHASE2` into `0` in `config.yml` to train phase 2 (use `--model 2`), and phase 3 separately by *any* order.
- You can stop the training and then change the `SIGMA` in `config.yml`, then restart training. This way
is really tricky.


<span id="jump_zh">训练指南🇨🇳 </span>
------

0 comments on commit dd0431a

Please sign in to comment.