MapRoute: Semantic Routing for Precise Concept Erasure with Mapper

(a) MapRoute framework (b) The structure of Mapper

Contents

• Creating a Conda Environment
• Training Mapper to Erase Concepts
• Sampling from the Modified Model
• Metrics Evaluation

Creating Environment

conda create -n maproute python=3.12.11
conda activate maproute
cd MapRoute
pip install -r requirements.txt

Erase Concepts

Object erasure:

An example of eliminating the concept of "airplane" class:

CUDA_VISIBLE_DEVICES=0 python train/train_mapper_object.py 
--text-encoder-path CompVis/stable-diffusion-v1-4/text_encoder 
--output-model-path ./airplane.pth --epochs 100 
--text2 'plane' 'jet' 'aircraft' 'private jet' 'flying' 'spacecraft' 'helicopter' 'drone' 'business jet' 
--text3 'vehicle' 'vehicle' 'vehicle' 'vehicle' 'vehicle' 'vehicle' 'vehicle' 'vehicle' 'vehicle' 'vehicle' 

Artistic style erasure:

An example of eliminating the concept of "Brent Heighton style":

CUDA_VISIBLE_DEVICES=0 python train/train_mapper_art.py 
--text-encoder-path CompVis/stable-diffusion-v1-4/text_encoder 
--output-model-path ./BrentHeighton_100.pth --epochs 100 
--text2 'Brent Heighton style'  --text3 'Van Gogh style'

Celebrities erasure:

An example of eliminating the concept of "Cameron Diaz":

CUDA_VISIBLE_DEVICES=0 python train/train_mapper_celeb.py 
--text-encoder-path CompVis/stable-diffusion-v1-4/text_encoder 
--output-model-path ./CameronDiaz_100.pth --epochs 100
--text2 'A photo of Cameron Diaz' 'An image of Cameron Diaz' 
--text3 'A photo of Andrew Garfield' 'An image of Andrew Garfield'

Generate pictures

Generate images by invoking the Top-k Mappers via MapRoute:

CUDA_VISIBLE_DEVICES=0 python Series_parallel_generate.py 
  --prompt "A photo of a cat" 
  --seed 123 
  # Names corresponding to Mappers in sequence
  --mapper_names "cat,truck,Brett Whiteley,Van Gogh"
  --mapper_paths "
  /path/of/your/mapper/cat_100.pth,
  /path/of/your/mapper/truck_100.pth,
  /path/of/your/mapper/BrettWhiteley_100.pth,
  /path/of/your/mapper/VanGogh_100.pth" 
  --output_path "/output/path/result.png" 
  --top_k 2