Diff-Font

---

Official code implementation based on pytorch for paper, Diff-Font: Diffusion Model for Robust One-Shot Font Generation. Arvix version

Dependencies

---

pytorch>=1.10.0
tqdm
opencv-python
sklearn
pillow
tensorboardX
blobfile>=1.0.5
mpi4py
attrdict
yaml

Dataset

---

方正字库 provides free font download for non-commercial users.

Example directory hierarchy

data_dir
    |--- font1
    |--- font2
           |--- 00000.png
           |--- 00001.png
           |--- ...
    |--- ...

Usage

---

Prepare dataset

python font2img.py --ttf_path ttf_folder --chara total_chn.txt --save_path save_folder --img_size 80 --chara_size 60

Conditional training

• Modify the configuration file cfg/train_cfg.yaml

  Key setting for conditional training:

  data_dir: 'path_to_dataset/'
  chara_nums: 6625  # num of characters
  train_step: 420000 # conditional training steps
  sty_encoder_path: './pretrained_models/chinese_styenc.ckpt' # path to pre-trained style encoder
  model_save_dir: './trained_models' # path to save trained models
  stroke_path: './chinese_stroke.txt' # encoded strokes
  classifier_free: False # False for conditional training
  resume_checkpoint: ""

• single gpu

  python train.py --cfg_path cfg/train_cfg.yaml

• distributed training

  mpiexec -n $NUM_GPUS python train.py --cfg_path cfg/train_cfg.yaml

Fine-tuning

After conditional training , we suggest an additional fine-tuning step.

• Modify the configuration file cfg/train_cfg.yaml

  Key setting for fine-tuning:

  data_dir: 'path_to_dataset/'
  chara_nums: 6625  # num of characters
  model_save_dir: './trained_models' # path to save trained models
  stroke_path: './chinese_stroke.txt' # encoded strokes
  classifier_free: True  # True for fine-tuning
  total_train_step: 800000 # total number of training steps for conditional training and fine-tuning
  resume_checkpoint: "./trained_models/model420000.pt" # path to conditional trained model, required for fine-tuning

• single gpu

  python train.py --cfg_path cfg/train_cfg.yaml

• distributed training

  mpiexec -n $NUM_GPUS python train.py --cfg_path cfg/train_cfg.yaml

Test

Modify the configuration file cfg/test_cfg.yaml

Key setting for testing:

chara_nums: 6625
num_samples: 10
stroke_path: './char_stroke.txt'
model_path: 'path_to_trained_model, suggest using the ema model'
sty_img_path: 'path_to_reference_image'
total_txt_file: './total_chn.txt'
gen_txt_file: './gen_char.txt' # txt file for generation
img_save_path: './result' # path to save generated images
classifier_free: True 
cont_scale: 3.0 # content guidance sacle
sk_scale: 3.0 # stroke guidance sacle

, then run

python sample.py --cfg_path cfg/test_cfg.yaml

Visual display

FAQ

1. The generated characters content are incorrect.

Please check whether each font in the dataset used for training contains all characters in the '.txt ' file.

2. The generated character images are unclear and structurally incomplete.

This phenomenon indicates that the model training is not sufficient. Please continue to train the model.

Acknowledgements

---

This project is based on openai/guided-diffusion and DG-Font.