🖼︎ CCLAP [Paper]

Paper Title: "CCLAP: Controllable Chinese Landscape Painting Generation via Latent Diffusion Model"

Conference Accepted: ICME 2023 (oral)

👀 Introduction

In this work, we propose a controllable Chinese landscape painting generation method named CCLAP, which can generate painting with specific content and style based on Latent Diffusion Model. Specifically, it consists of two cascaded modules, i.e., content generator and style aggregator. The content generator module guarantees the content of generated paintings specific to the input text. While the style aggregator module is to generate paintings of a style corresponding to a reference image. Moreover, a new dataset of Chinese landscape paintings named CLAP is collected for comprehensive evaluation. Both the qualitative and quantitative results demonstrate that our method achieves state-of-the-art performance, especially in artfully-composed and artistic conception.

📩 Resources Download

Dataset Download

Dataset contains 3560 paintings with corresponding captions.

Send the Email to us if you want to apply for the dataset, showing your school or company and your application purpose. We will give a feedback within 5 days.

Email address: wangzhongqi23s@ict.ac.cn

Models Download

Name	Download
Content Generator	Hugging face
Style Aggregator	Baidu Disk [code:tu8z]

🔨 Results

How to use

• Git clone this repo and pip install the requirements

• Download the models (i.e., Content Generator and Style Aggregator) and put them into the same folder

  Make sure the file structure is:
  
  CCLAP
  PAMA
  style_image
  app.py
  hist_loss.py
  net.py
  style_transfer.py
  utils.py
  

• run app.py

  python app.py
  

⏳ Ongoing

We are very passionate about the issue of creating Chinese landscape paintings and are constantly studying these topics:

• Solve the problem that the model cannot generate human beings well.
• What is the “style” in Chinese landscape painting? The “style” or we call “genre” is a kind of thing that contains the rule or the knowledge about how we color a painting.
• Can we control the composition of the painting?
• A painting contains a spirit.
• How can we generate a painting in a more human way?
• ...

🤝 Feel free to discuss with us privately!

📄 Citation

If you find this project useful in your research, please consider cite:

@INPROCEEDINGS{10219843,
  author={Wang, Zhongqi and Zhang, Jie and Ji, Zhilong and Bai, Jinfeng and Shan, Shiguang},
  booktitle={2023 IEEE International Conference on Multimedia and Expo (ICME)}, 
  title={CCLAP: Controllable Chinese Landscape Painting Generation Via Latent Diffusion Model}, 
  year={2023},
  pages={2117-2122},
  doi={10.1109/ICME55011.2023.00362}}