diff-morph-torch

Related: diff-morph.

PyTorch-based implementation DiffMorph.

Tips

If you're looking to dive into the code, the recommended reading order is:

• config.py
• utils/seq_aligner.py
• p2p/ptp_utils.py
• nti/null_inversion.py
• p2p/ptp.py
• utils/blip_prompt_generate.py

The follow is the using example of p2p & nti:

• p2p/style.py
• p2p/background.py
• p2p/image_operation.py

Requirements

Basic Settings:

• PyTorch==1.11.0
• Python==3.8 (ubuntu20.04)
• CUDA==11.3

Essential dependencies:

Package	Base	Base + BLIP
diffusers	0.8.0 (🔒)	0.8.0 (🔒)
huggingface_hub	0.14.1	0.23.2
accelerate	0.12.0	0.26.0
transformers	4.25.1	4.46.3
ftfy	6.2.3	6.2.3
opencv-python	4.11.0.86	4.11.0.86
ipywidgets	7.7.0	7.7.0
python-dotenv	1.0.1	1.0.1

Run

Recommended Hardware

• NVIDIA RTX 4090 (24GB)

Installation & Running

• Install key dependencies: pip install -r requirements.txt

If any dependencies are reported missing during runtime, install them manually: pip install <missing-package-name>.

📌 Note: Do not change the version numbers specified in requirements.txt to avoid compatibility issues.

• Set MODEL_CACHE_DIR in config.py

You can also adjust other configurations in the config.py as needed.

• Launch the server: python app.py

Default host & port: 0.0.0.0: 6006.

Prompt Example

Image	BLIP Base generated Source Prompt	BLIP Large generated Source Prompt
example_images/flower.png	"a pink flower with green leaves on it"	"there is a pink flower that is blooming in the sun"
example_images/city.jpeg	"san skyline from above"	"a view of the city skyline from the top of a hill"
example_images/gnochi_mirror.jpeg	"a cat sitting on a wooden shelf"	"there is a cat sitting on a shelf in front of a mirror"

Using Example

Image Operation

Change Background

Style Transfer

References

• A. Hertz, R. Mokady, J. Tenenbaum, et al.
  Prompt-to-Prompt Image Editing with Cross-Attention Control, 2023.
  Paper GitHub: prompt-to-prompt.
• R. Mokady, A. Hertz, K. Aberman, et al.
  Null-Text Inversion for Editing Real Images Using Guided Diffusion Models, 2023.
  Paper GitHub: null-text inversion.