🚀 Universal NN Latent Upscaler for ComfyUI

⚠️ Work in Progress: This project is actively being developed. Current models work but we're training improved versions for better quality!

A universal neural network latent upscaler that supports SD1.5, SDXL, Flux, and Wan2.2 models. Uses trained neural networks instead of simple interpolation for higher quality latent upscaling.

Built upon the excellent foundation of Ttl's ComfyUi_NNLatentUpscale - this project extends the original work with universal model support and improved architectures.

• 🎯 Universal Support: Works with SD1.5, SDXL, Flux, and Wan2.2 models
• 🧠 Neural Network Upscaling: Significantly better quality than bilinear interpolation
• ⚙️ Configurable Scale Factor: 1.0x to 2.0x upscaling with smooth interpolation
• 🔄 Automatic Model Loading: Automatically detects and loads the correct model for each architecture
• 💾 Memory Efficient: Smart model loading/unloading to save VRAM
• 🎨 Custom Wan2.2 Model: Includes our own trained Wan2.2 upscaler (more models coming!)
• 🔧 Easy Integration: Drop-in replacement for standard latent upscaling

cd ComfyUI/custom_nodes
git clone https://github.com/DenRakEiw/WAN_NN_Latent_Upscale

1. Download the repository as ZIP
2. Extract to ComfyUI/custom_nodes/wan_nn_latent/
3. Restart ComfyUI

• ComfyUI (latest version recommended)
• PyTorch with CUDA support (for GPU acceleration)
• Python 3.8+

1. Add the node: Add Node -> latent -> Universal NN Latent Upscale
2. Connect your latent: Connect any latent tensor as input
3. Select model type: Choose SD 1.5, SDXL, Flux, or Wan2.2
4. Set upscale factor: 1.0x to 2.0x (1.5x recommended)
5. Connect output: Use the upscaled latent in your workflow

[VAE Encode] -> [Universal NN Latent Upscale] -> [Your Model] -> [VAE Decode]

• MSE: 0.1038 (vs 0.1054 bilinear) - ✅ 1.5% improvement
• PSNR: 9.84 dB (vs 9.77 dB bilinear) - ✅ 0.7 dB improvement
• SSIM: 0.3247 (vs 0.2690 bilinear) - ✅ 20.7% improvement

🔬 In Development: Training new models with 2,852 real photo samples for significantly better quality!

The following model files are included:

📝 Note: Models are automatically downloaded on first use if not present.

• SD1.5/SDXL: 0.13025 (4-channel latents)
• Flux: 0.3611 (16-channel latents)
• Wan2.2: 0.3604 (16-channel latents, empirically determined)

• SD1.5/SDXL: 4 channels → 128 hidden → 4 channels
• Flux/Wan2.2: 16 channels → 256 hidden → 16 channels

• 🧠 Encoder: 3-layer CNN feature extractor with ReLU activation
• 🔄 Upsampler: Transpose convolution + refinement layers
• ➕ Skip Connections: Residual learning for detail preservation
• 📏 Adaptive Scaling: Supports any scale factor between 1.0-2.0x

[VAE Encode] -> [Universal NN Latent Upscale] -> [Model] -> [VAE Decode]

[Load Image] -> [VAE Encode] -> [Universal NN Latent Upscale] ->
[ControlNet/LoRA] -> [Model] -> [VAE Decode] -> [Save Image]

• ✅ Basic functionality working
• ✅ All model types supported
• ✅ Stable performance
• ⚠️ Wan2.2 model needs improvement

• 🔄 Training improved Wan2.2 model with 2,852 real photos
• 🔄 20,000 training steps with advanced loss functions
• 🔄 Better detail preservation and artifact reduction
• 🔄 Performance benchmarking against other methods

• 📋 Support for custom model training
• 📋 Additional model architectures
• 📋 Batch processing optimization
• 📋 Advanced configuration options

# Generate dataset
python create_real_dataset.py

# Train model (example for Wan2.2)
python slow_long_training_20k.py

# Test model
python test_model_performance.py

1. Add configuration to MODEL_CONFIGS in latent_resizer.py
2. Create specialized resizer class if needed
3. Add model file path to weight_paths in nn_upscale.py
4. Update the dropdown in INPUT_TYPES

• 🐛 Bug reports and feature requests welcome
• 🔧 Pull requests for improvements appreciated
• 📊 Share your training results and model improvements
• 📝 Documentation improvements always helpful

# In nn_upscale.py, set:
force_reload = True  # Reloads model on every execution

1. 📖 Check this README and troubleshooting section
2. 🐛 Open an issue on GitHub
3. 💬 Join the ComfyUI Discord community
4. 📧 Contact: [your-email@example.com]

📈 Higher SSIM and PSNR = better quality, Lower MSE = better accuracy

This project is licensed under the MIT License - see the LICENSE file for details.

• 🎯 Primary Foundation: Ttl's ComfyUi_NNLatentUpscale - This project builds directly upon Ttl's excellent work and neural network architecture. The core upscaling approach, model structure, and ComfyUI integration patterns are based on their pioneering implementation.
• 🤝 Additional Inspiration: naripok's contributions to the NN latent upscaling community
• 🖼️ ComfyUI Team: For the amazing framework that makes this possible
• 🤖 Model Teams: Wan2.2, Flux, Stability AI for their incredible models
• 🌟 Community: ComfyUI Discord community for feedback and support
• 📊 Datasets: DIV2K, COCO2017 for training data

This project would not exist without Ttl's groundbreaking work on neural network latent upscaling. Their original implementation provided:

• The core neural network architecture for latent upscaling
• ComfyUI node integration patterns
• Training methodologies and loss functions
• Proof of concept that NN upscaling significantly outperforms bilinear interpolation

Our contribution extends this foundation with universal model support, improved training data, and enhanced architectures while maintaining compatibility with the original approach.

• 🏠 ComfyUI
• 📚 Documentation
• 🐛 Issues
• 💬 Discussions