-
-
Notifications
You must be signed in to change notification settings - Fork 1
Captioning
Bitcrush Studio gives you several ways to create, organise, and maintain captions across your dataset. You can write captions manually, generate natural-language captions using a vision-language model, or allow Studio to build tag-style captions automatically from the information already associated with each image.
Studio supports multiple caption styles because modern image-generation models are often trained using a mixture of natural-language descriptions and structured tags. You can choose how these captions are generated, how they are classified, and which information is passed to the captioning model.
Studio supports three caption styles:
Descriptive: Longer, detailed natural-language captions.
Short: Concise natural-language captions.
Tags: Comma-separated or tag-based captions, such as Danbooru-style captions.
You can customise the length threshold used to distinguish between descriptive and short captions in Application Settings, accessible from the top-right corner of Studio.
Incoming natural-language captions are analysed automatically and classified as either descriptive or short. This applies whether the caption was:
Written manually, such as through Gallery View.
Generated by a VLM using the Caption Tool.
Imported from an existing caption file.
Tag-style captions are handled separately. Studio generates and updates them automatically in the background using image metadata and any tags added through Bitcrush Studio. This happens as part of the Background Sync process and can be customised in Application Settings.
Modern DiT models, such as FLUX.2 [klein], Z-Image, and Anima, are often trained using a mixture of caption styles, with descriptive captions, short captions, and tags being the three main varieties.
Studio stores each caption on a separate line. OneTrainer also treats each new line as a separate caption. Other training tools may interpret caption files differently, so check the documentation for your chosen training tool before beginning a large training run.
Background Sync keeps information associated with your images up to date without requiring you to manually regenerate every caption.
As part of this process, Studio can build tag-style captions using:
Metadata extracted from the image.
Tags added or approved within Bitcrush Studio.
Other supported image attributes available to Studio.
These tag-style captions are generated locally in the background. They do not need to be requested from the VLM through the Caption Tool.
The exact behaviour of Background Sync, including how tag-style captions are assembled, can be customised in Application Settings.
The Caption Tool generates natural-language captions using either a cloud-based API or a locally hosted vision-language model.
The Caption Tool supports OpenAI-compatible endpoints from providers such as OpenRouter, Alibaba Cloud, OpenAI, and others.
Most cloud providers require an API key. API keys entered into Studio are stored using the cryptographically protected credential storage provided by your operating system.
Because processing takes place remotely, the practical batch size is primarily limited by your network connection, the provider's rate limits, and the amount of data being uploaded. A batch size of up to 64 is often practical, although the best setting will vary between providers.
You can also use an OpenAI-compatible local server, such as LM Studio, for local processing. You must first configure and start the server within LM Studio. See LM Studio's server documentation for instructions on exposing a local OpenAI-compatible endpoint.
Studio attempts to resolve known endpoints automatically. If a compatible endpoint is not detected or does not appear as expected, please raise an issue with the provider name and endpoint format.
The Caption Tool also supports the Google Gemini API, allowing you to use compatible Google models such as Gemini 3.5 Flash and supported Gemma models.
Some providers or models do not support configurable reasoning effort. Others may use a different parameter name or reject the setting entirely. In these cases, attempting to disable or adjust reasoning effort may return an API error.
The Transformers backend uses Hugging Face Transformers to load compatible vision-language models directly onto your GPU.
Model files are downloaded automatically from Hugging Face when required. Some restricted or gated models may require you to provide a Hugging Face access token before they can be downloaded.
The amount of VRAM required depends on the selected model, precision, quantisation settings, image resolution, and batch size.
The Llama.cpp backend provides experimental support for loading compatible local .gguf vision-language models.
Compatibility varies between models, particularly where a model requires a separate vision projector or uses an unsupported multimodal architecture.
Prompt Builder lets you control the instructions sent to the VLM for each image.
You can configure two kinds of prompts:
Base Prompts: The main captioning instruction. Studio randomly selects one base prompt for each image.
Variable Prompts: Optional instructions appended to the selected base prompt. Studio selects a random number of these according to your configured minimum and maximum values.
This lets you introduce controlled variation into a captioning run. For example, one image may ask the model to describe the camera angle, while another may ask it to identify the shot type.
Prompt Builder supports variables that are replaced with information from the current image before the prompt is sent to the VLM.
Variable | What it contains | How it can be used -- | -- | -- {filename} | The filename of the image currently being captioned. | Helps the VLM use meaningful information contained in filenames, such as subject names, locations, sequence labels, or existing dataset categories. {metadata} | Metadata and tags currently associated with the image in Bitcrush Studio. | Gives the VLM additional context about the image and can help it preserve existing classifications or approved tags. {caption} | The image's existing caption text, when one is available. | Allows the VLM to expand, shorten, rewrite, correct, or preserve information from an existing caption.Variables are optional. If you do not include a variable in your prompt, that information will not be inserted into the user prompt sent to the VLM.
For example, using the prompts shown above, the first image might be sent with:
Write a detailed, descriptive caption for this image.
Filename: {filename}
Tags: {metadata}
Existing Caption (if present): {caption}
Include information about the camera angle.
Before the request is sent, Studio replaces each variable with the information belonging to that image.
The second image might instead be sent with:
Write a concise caption for this image, no more than 2 short sentences.
Filename: {filename}
Tags: {metadata}
Existing Caption (if present): {caption}
Include whether the image is a close-up, medium shot, or another shot type.
In this example, Studio has selected a different base prompt and appended a different variable prompt.
The Descriptive, Short, or Tag dropdown identifies the type of caption that a prompt configuration is intended to produce.
It does not force the returned text to be classified as that caption type. Incoming captions are still analysed using Studio's normal caption classification rules.
The selected type is particularly important when using the Caption New Files & Missing Caption Types strategy. With this strategy:
An image with only a short caption can be sent to generate a descriptive caption.
An image with only a descriptive caption can be sent to generate a short caption.
An image without the requested caption type can be included without overwriting the caption types it already has.
Tag-style captions generated through Background Sync are not treated as natural-language captions by the Caption Tool. An image containing only a tag-style caption is therefore still considered to be missing both a descriptive and a short caption, and remains eligible for natural-language captioning.
System Prompt: Controls the higher-level instructions given to the VLM. This can be used to define its role, expected behaviour, formatting rules, or restrictions.
Max Tokens: Limits the maximum output token budget available to the model. Setting this too low may cause captions to be truncated. Depending on the provider and model, reasoning or thinking tokens may also consume part of this budget.
Temperature: Controls variation in the model's responses. Lower values generally produce more consistent captions, while higher values allow more varied or creative wording. Values between approximately
0.9and1.1often work well with frontier models, although the ideal value depends on the selected model and captioning task.Fallback Models: Defines one or more alternative models that Studio can try when the primary model fails to produce a caption. This may happen because the model refuses the image, the provider returns an error, the request times out, or the response does not contain a usable caption. Fallback models are attempted in the order shown.
Learn how to use Bitcrush Studio here!