Skip to content

SubtitleCoreConcepts

Jump to bottom
zalunda edited this page Sep 17, 2025 · 3 revisions

Core Concepts

Welcome to the conceptual guide for the FunscriptToolbox subtitle engine. While other pages explain what each worker does, this page explains the fundamental building blocks and principles that the entire system is built upon. Understanding these core concepts will make the workflow and the structure of the configuration files much clearer.

1. Timed Item (or Timed Segment)

The Timed Item is the most basic unit of data in the entire pipeline. At its heart, it is simply a placeholder in time.

  • Definition: A Timed Item is an object defined by a StartTime and an EndTime. Its Duration is automatically calculated from these two values.
  • Analogy: Think of a single, empty subtitle in Subtitle Edit before you've typed any text. It has a start, it has an end, but it doesn't contain any information yet. That is a Timed Item.

Every piece of data that the tool generates—from a single transcribed word to a full sentence to a visual analysis of a scene—is attached to one of these Timed Items.

2. Metadata

If a Timed Item is the empty container, Metadata is the content you put inside it. This is the single most important concept to understand, as it is the foundation of the tool's producer/consumer model.

  • Definition: Metadata is a collection of key-value pairs (like a dictionary or a set of labels) attached to a single Timed Item.
  • Analogy: If a Timed Item is an empty subtitle, metadata is the text and notes you write within it.

In this system, everything is metadata. The tool doesn't just work with "text"; it works with timed items that have metadata attached.

Concrete Examples:

  • A transcriber runs and produces a timed item with the metadata: { "VoiceText": "こんにちは" }
  • A translator then consumes that item and produces a new item with the metadata: { "TranslatedText": "Hello" }
  • The speaker identification tool adds metadata to an existing item: { "Speaker": "Hana" }
  • The visual analyst adds multiple pieces of metadata: { "ParticipantsPoses": "Hana is smiling.", "TranslationAnalysis": "Her tone is playful." }

This system is what allows for such powerful and flexible workflows. A worker can be configured to look for any piece of metadata on a Timed Item, use it, and then add its own metadata to the collection.

Special Metadata: The "Ongoing" Prefix

While most metadata is tied directly to the specific Timed Item it's attached to, any metadata key that starts with the prefix Ongoing has a special, stateful behavior. This is designed to solve the problem of carrying context across large gaps in time or, more importantly, across different batches sent to an AI.

How it works:

The system actively tracks the most recent value of any metadata key starting with Ongoing (e.g., OngoingSpeakers, OngoingContext, OngoingAppearance).

When the tool prepares a new batch of items to be sent to an AI, it automatically injects the last-known values of all Ongoing* metadata into the very first item of that batch. This applies to both the context section and the main analysis section of the request.

Why this is important:

Imagine a 10-minute scene where the speakers (OngoingSpeakers: Hana, Ena) are defined only once at the very beginning. If the AI processes this scene in multiple batches, without this feature, the second batch would have no idea who the speakers are.

The "Ongoing" prefix solves this by ensuring that every new batch sent to the AI is "seeded" with the latest context, guaranteeing that the AI always has the most current information about the state of the scene (who is present, where they are, etc.), leading to much more consistent and accurate results.

3. The WorkInProgressSubtitles File (.wipsubtitles)

This file is the project's main save file. It is the central repository for all the data generated during the subtitle creation process.

  • Definition: The .wipsubtitles file is a JSON file that stores the complete state of a project. It contains all the AudioExtractions, Transcriptions, and Translations that have been produced by the workers.
  • Function: It tracks all progress. When you run the tool, it loads this file, sees what work has already been done, and then continues from where it left off.
  • Analogy: Think of it like a Photoshop .psd file. It's not the final output (like a .jpg), but it contains all the layers, data, and history needed to continue working on the project.
  • Critical Warning: This file is essential. If you delete the .wipsubtitles file for a project, all progress will be lost, and the tool will start over from scratch on the next run.
Power User Tip: How to Re-run a Specific Worker
Occasionally, you might find that a specific worker (like an AI transcriber) produced a poor result, and you want to force it to run again without changing your main --FSTB-SubtitleGenerator.config file. You can do this by directly editing the .wipsubtitles save file.

⚠️ Warning: This is an advanced technique. Always make a backup copy of your .wipsubtitles file before editing it manually. An error could corrupt your project's progress.

The basic principle is to rename the ID of the old, bad result in the save file. When the tool runs again, it won't find the results for the worker it's looking for (e.g., "full") and will be forced to re-run it.

To re-run a AudioExtractor, Transcriber or Translator (e.g., full):

  1. Open the .wipsubtitles file in a text editor.
  2. Search for the object you want to redo by using the id.
  3. Change its Id to something else. Appending -bad is a good convention.
// BEFORE
{
  "$id": "65716",
  "Id": "full",
  "MetadataAlwaysProduced": "VoiceText",
  // ...
}

// AFTER
{
  "$id": "65716",
  "Id": "full-bad",
  "MetadataAlwaysProduced": "VoiceText",
  // ...
}

This method is powerful but should be used with care, as it is a direct manipulation of the project's save state.

4. The Backup Folder

You will notice that for each project (e.g., my-video), a folder named my-video_Backup is created.

  • Definition: The backup folder is a safety net to prevent accidental data loss.
  • How it works: When the tool needs to overwrite a file—especially one that might contain your manual work (like .manual-edit.srt) or the results of a lengthy process (like a .txt prompt file you filled in)—it does not permanently delete the old version. Instead, it moves the old file into the backup folder.
  • Purpose:
    1. Recovery: If you make a mistake, you can often recover the previous version of the file from this folder.
    2. Debugging: It allows you to inspect the exact state of files from previous runs, which can be invaluable for troubleshooting a complex workflow. This includes temporary files like the .wav chunks and .jpg screenshots sent to AIs if you have KeepTemporaryFiles enabled.

Clone this wiki locally