-
Notifications
You must be signed in to change notification settings - Fork 149
Home
Welcome to the Pallaidium wiki!
Pallaidium is a free, generative AI movie studio integrated directly into the Blender Video Sequence Editor (VSE). It empowers users to generate video, images, audio, and text from simple text prompts or existing media strips, streamlining the creative process within a single application.
This document provides comprehensive information for installing, using, understanding the features, and troubleshooting Pallaidium.
Pallaidium provides a comprehensive suite of generative AI tools directly inside Blender's VSE.
- Text to Video
- Text to Image
- Text to Text (Prompt rewriting/Captioning)
- Text to Speech
- Text to Audio (Sound effects)
- Text to Music
- Image to Image (Stylization/Refinement)
- Image to Video (Animation)
- Image to Text (Captioning)
- Video to Video (Stylization)
- Video to Image
- Video to Text
This matrix provides a quick overview of the core generative capabilities, mapping input types to possible outputs.
| Input | Image | Video | Text | Audio | Music | Speech |
|---|---|---|---|---|---|---|
| Text | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Image | ✅ | ✅ | ✅ | |||
| Video | ✅ | ✅ | ✅ |
- ControlNet: Support for Canny, OpenPose, and Scribble inputs.
- ADetailer: Automated detail enhancement.
- IP Adapter: Face transfer and Style transfer.
- LoRAs: Support for multiple LoRAs with adjustable weights.
- Style selector: Apply predefined styles to generations.
- Adjustable Parameters: Seed, Quality steps, Strip power (denoising strength), Frames (Duration), and Word power (guidance scale).
- Batch Processing: Batch conversion, refinement, and upscaling for images and movies.
- Prompt Batching: Use text strips for sequential prompt generation.
- Management: Model card selector, user-defined file paths, and automatic metadata (seed/prompt) in strip names.
- UI/UX: Render-to-path selector and render finished notifications.
Follow these steps carefully to install Pallaidium and its dependencies.
| Component | Requirement |
|---|---|
| Operating System | Windows. Limited support for Linux and MacOS |
| Graphics Card | CUDA-supported Nvidia card with at least 6 GB of VRAM |
| CUDA Version | 12.4 |
| Storage | 20+ GB of free HDD space (each AI model requires 6+ GB) |
- Install Git: Download and install Git from https://git-scm.com/downloads, ensuring it is added to your system's PATH.
- Download the Add-on: Download the latest add-on zip file from https://github.com/tin2tin/Pallaidium/archive/refs/heads/main.zip.
- Run Blender as Administrator: You must right-click the Blender icon and select "Run as Administrator" to install dependencies correctly and avoid permission errors.
-
Install the Add-on in Blender:
- Open Blender (as Administrator).
- Go to
Preferences > Add-ons > Install. - Select the downloaded
.zipfile and enable the add-on.
-
Uninstall and Reinstall Dependencies (Crucial Step):
- In the add-on's preferences, click the "Uninstall Dependencies" button.
- Restart Blender (as Administrator).
- Return to the add-on preferences and click the "Install Dependencies" button.
- Final Restart: Restart your computer, then launch Blender as Administrator one more time before use.
Important Note: The first time you execute any model, 5-10 GB of data will be downloaded from HuggingFace. This only happens once per model.
The Pallaidium panel is located in Blender's Video Sequence Editor > Sidebar (N-Panel) > Generative AI tab.
- Select Output Type: In the "Output" section, choose your desired media type: Image, Video, Audio, or Text.
- Select Model: Choose a specific AI model from the dropdown list.
-
Configure Input Source:
-
From Text Prompts: Leave the
Inputdropdown onPromptsand enter your text. -
From Existing Strips: Select one or more strips in the VSE timeline and set the
Inputdropdown toStrips.
-
From Text Prompts: Leave the
- Adjust Parameters: Configure settings like resolution, quality steps, guidance scale, duration, and seeds.
-
Generate: Click
Generate(for prompts) orGenerate from Strips.
The generated media is automatically saved to the directory specified in the add-on preferences and added to your VSE timeline.
Pallaidium excels at batch operations. Select multiple strips (text, image, or video) in the timeline, configure your settings, and click Generate from Strips to process them sequentially. The file name and seed (if present in the source strip's name) are automatically used, a feature that can be toggled in the preferences.
Pallaidium can generate speech, music, and sound effects using various models.
For high-quality text-to-speech, the Parler TTS model is available with the following voices:
Aaron, Alisa, Anna, Barbara, Bill, Brenda, Bruce, Carol, David, Eileen, Emily, Eric, Gary, James, Jason, Jenna, Jerry, Jon, Jordan, Joy, Karen, Laura, Lauren, Lea, Mike, Naomie, Patrick, Rebecca, Rick, Rose, Tom, Tina, Will, Yann.
Pallaidium integrates a variety of state-of-the-art generative models from HuggingFace.
| Model Name | HuggingFace Link |
|---|---|
| FramePack | - |
| SkyReels-V1-Hunyuan-I2V/T2V | Tencent-Hunyuan/SkyReels-V1 |
| HunyuanVideo | Tencent-Hunyuan/Hunyuan-Video |
| LTX-Video | lightricks/ltx-video |
| Stable Video Diffusion | stabilityai/stable-video-diffusion-img2vid-xt |
| CogVideoX | THUDM/CogVideoX-5b |
| Zeroscope V2 XL | cerspense/zeroscope_v2_XL |
| Model Name | HuggingFace Link |
|---|---|
| Stable Diffusion XL (SDXL) | stabilityai/stable-diffusion-xl-base-1.0 |
| SDXL Lightning | ByteDance/SDXL-Lightning |
| Stable Diffusion 3 Medium | stabilityai/stable-diffusion-3-medium-diffusers |
| FLUX.1 Schnell/Dev | black-forest-labs/FLUX.1-schnell |
| Fluently-XL-Final | fluently/Fluently-XL-Final |
| Shuttle Jaguar | shuttleai/shuttle-jaguar |
| HunyuanDiT | Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers |
| Kolors | Kwai-Kolors/Kolors-diffusers |
| OpenDalleV1.1 | dataautogpt3/OpenDalleV1.1 |
| PixArt-Sigma | PixArt-alpha/PixArt-Sigma_16bit |
| RealVisXL V4.0 | SG161222/RealVisXL_V4.0 |
| ProteusV0.4 | dataautogpt3/ProteusV0.4 |
| BLIP Diffusion | Salesforce/blipdiffusion |
| OmniGen | - |
| Chroma | - |
| Lumina 2 | Alpha-VLLM/Lumina-txl-2.0 |
| Sana | Efficient-Large-Model/Sana-1024 |
| Model Name | HuggingFace Link |
|---|---|
| Canny SDXL | diffusers/controlnet-canny-sdxl-1.0-small |
| OpenPose SDXL | xinsir/controlnet-openpose-sdxl-1.0 |
| Scribble SDXL | xinsir/controlnet-scribble-sdxl-1.0 |
| Model Name | HuggingFace Link |
|---|---|
| Stable Audio Open | stabilityai/stable-audio-open-1.0 |
| MusicGen Stereo | facebook/musicgen-stereo-medium |
| F5-TTS | MyShell/F5-TTS |
| WhisperSpeech | collabora/whisperspeech |
| MMAudio | hustvl/MM-Audio |
| Chatterbox | codem-solutions/chatterbox |
| Task | Model Name | HuggingFace Link |
|---|---|---|
| Background Removal | BiRefNet_HR | ZhengPeng7/BiRefNet_HR |
| Caption/Prompt Gen | Florence-2-large | microsoft/Florence-2-large |
| Prompt Rewriter | MoviiGen1.1 | ZuluVision/moviigen-1.1-prompt_rewriter |
If you encounter issues, consult this guide for common solutions.
| Problem | Description/Error Message | Solution |
|---|---|---|
| Dependency/Module Errors | Missing Python modules or installation failures. | Use the blender_pip add-on to manually install packages: https://github.com/tin2tin/blender_pip
|
| MSVC Error | "WARNING: Failed to find MSVC" during installation. | Install "Tools for Visual Studio" from Microsoft: https://aka.ms/vs/17/release/vs_BuildTools.exe |
| Out of Memory | "CUDA is out of memory" error during generation. | 1. Restart Blender to clear VRAM. 2. If you have low VRAM, try disabling "System Memory Fallback" in your NVIDIA control panel. |
| Broken/Distorted Renders | Generated media appears corrupted or has incorrect dimensions. | Ensure the resolution set in the panel matches the native resolution of the model (check the Model Card in Preferences). |
| Stuttering Playback | Generated video strips play back slowly in the VSE. | Select the strip, then go to Menu > Strip > Movie Strip > Set Render Size. |
| Installation Permissions | Errors related to file writing or "access denied." | Always run Blender as Administrator when installing or managing dependencies. |
These companion add-ons, also developed by tin2tin, integrate seamlessly with Pallaidium to enhance the generative workflow.
| Add-on | Description | Link |
|---|---|---|
| GPT4Blender | Generate image prompts or stories using GPT4ALL. | https://github.com/tin2tin/GPT4BLENDER |
| Text to Strip | Convert text blocks into individual VSE strips for batch prompting. | https://github.com/tin2tin/text_to_strip |
| Subtitle Editor | A comprehensive tool for editing and managing text strips in the VSE. | https://github.com/tin2tin/Subtitle_Editor |
| VSE Masking Tools | Create and manage masks directly on top of clips in the Sequencer. | https://github.com/tin2tin/vse_masking_tools |
| Add Rendered Strips | Convert other strip types into movie strips for use as inputs. | https://github.com/tin2tin/Add_Rendered_Strips |
| blender_pip | A utility to install Python packages into Blender’s environment. | https://github.com/tin2tin/blender_pip |