Extract all subtitle streams simultaneously #10884
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Extracting a subtitle stream is a disk I/O bottlenecked operation as ffmpeg has to read through the whole file, but usually there is nothing CPU intensive to do. If a file has multiple subtitle streams, and we want to extract more of them, extracting them one-by-one results in reading the whole file again and again. However ffmpeg can extract multiple streams at once. We can optimize this by extracting the subtitle streams all at once when only one of them gets queried, then we will have all of them cached for later use. It is useful for people switching subtitles during playback. It is even more useful for people who extract all the subtitle streams in advance, for example with the "Subtitle Extract" plugin. In this case we reduce the extraction time significantly based on the number of subtitle streams in the files, which can be 5-10 in many cases. Signed-off-by: Attila Szakacs <szakacs.attila96@gmail.com>
|
Jellyfin have a tendency to choke the storage when there are tons of attachments. Usially this happens with anime shows. I opened an old issue about this that can be used as a reference. If this fixes the above issue alot people with slower storage and low memory hardware will be happy. |
barronpm
reviewed
Jan 31, 2024
There was a problem hiding this comment.
Some minor formatting notes, cc @nyanmisaka for the substance of the changes
nyanmisaka
reviewed
Jan 31, 2024
alltilla
force-pushed
the
extract-all-subtitles
branch
from
cd1f290 to
005c215
Compare
|
Thank you for the review! I have fixed all your comments, here's the diff: diff --git a/MediaBrowser.MediaEncoding/Subtitles/SubtitleEncoder.cs b/MediaBrowser.MediaEncoding/Subtitles/SubtitleEncoder.cs
index 0e66565ed..3230927a6 100644
--- a/MediaBrowser.MediaEncoding/Subtitles/SubtitleEncoder.cs
+++ b/MediaBrowser.MediaEncoding/Subtitles/SubtitleEncoder.cs
@@ -472,8 +472,8 @@ namespace MediaBrowser.MediaEncoding.Subtitles
/// <returns>Task.</returns>
private async Task ExtractAllTextSubtitles(MediaSourceInfo mediaSource, CancellationToken cancellationToken)
{
- var semaphores = new List<SemaphoreSlim> { };
- var extractableStreams = new List<MediaStream> { };
+ var semaphores = new List<SemaphoreSlim>();
+ var extractableStreams = new List<MediaStream>();
try
{
@@ -498,9 +498,9 @@ namespace MediaBrowser.MediaEncoding.Subtitles
}
if (extractableStreams.Count > 0)
- {
- await ExtractAllTextSubtitlesInternal(mediaSource, extractableStreams, cancellationToken).ConfigureAwait(false);
- }
+ {
+ await ExtractAllTextSubtitlesInternal(mediaSource, extractableStreams, cancellationToken).ConfigureAwait(false);
+ }
}
catch (Exception ex)
{
@@ -521,7 +521,7 @@ namespace MediaBrowser.MediaEncoding.Subtitles
CancellationToken cancellationToken)
{
var inputPath = mediaSource.Path;
- var outputPaths = new List<string> { };
+ var outputPaths = new List<string>();
var args = string.Format(
CultureInfo.InvariantCulture,
"-i {0} -copyts",
@@ -531,6 +531,13 @@ namespace MediaBrowser.MediaEncoding.Subtitles
{
var outputPath = GetSubtitleCachePath(mediaSource, subtitleStream.Index, "." + GetTextSubtitleFormat(subtitleStream));
var outputCodec = IsCodecCopyable(subtitleStream.Codec) ? "copy" : "srt";
+ var streamIndex = EncodingHelper.FindIndex(mediaSource.MediaStreams, subtitleStream);
+
+ if (streamIndex == -1)
+ {
+ _logger.LogError("Cannot find subtitle stream index for {InputPath} ({Index}), skipping this stream", inputPath, subtitleStream.Index);
+ continue;
+ }
Directory.CreateDirectory(Path.GetDirectoryName(outputPath) ?? throw new FileNotFoundException($"Calculated path ({outputPath}) is not valid."));
@@ -538,7 +545,7 @@ namespace MediaBrowser.MediaEncoding.Subtitles
args += string.Format(
CultureInfo.InvariantCulture,
" -map 0:{0} -an -vn -c:s {1} \"{2}\"",
- subtitleStream.Index,
+ streamIndex,
outputCodec,
outputPath);
}
@@ -614,18 +621,15 @@ namespace MediaBrowser.MediaEncoding.Subtitles
{
_logger.LogError("ffmpeg subtitle extraction failed for {InputPath} to {OutputPath}", inputPath, outputPath);
failed = true;
+ continue;
}
- else
+ if (outputPath.EndsWith("ass", StringComparison.OrdinalIgnoreCase))
{
- if (outputPath.EndsWith("ass", StringComparison.OrdinalIgnoreCase))
- {
- await SetAssFont(outputPath, cancellationToken).ConfigureAwait(false);
- }
-
- _logger.LogInformation("ffmpeg subtitle extraction completed for {InputPath} to {OutputPath}", inputPath, outputPath);
+ await SetAssFont(outputPath, cancellationToken).ConfigureAwait(false);
+ }
+ _logger.LogInformation("ffmpeg subtitle extraction completed for {InputPath} to {OutputPath}", inputPath, outputPath);
}
}
- }
if (failed)
{ |
barronpm
reviewed
Jan 31, 2024
Signed-off-by: Attila Szakacs <szakacs.attila96@gmail.com>
alltilla
force-pushed
the
extract-all-subtitles
branch
from
005c215 to
ce81e2a
Compare
|
🤦 Sorry, thanks! |
nyanmisaka
approved these changes
Feb 1, 2024
barronpm
approved these changes
Feb 1, 2024
crobibero
approved these changes
Feb 1, 2024
KrzaQ
pushed a commit
to KrzaQ/jellyfin
that referenced
this pull request
Feb 13, 2024
Extract all subtitle streams simultaneously
alltilla
added a commit
to alltilla/jellyfin
that referenced
this pull request
Feb 18, 2024
Similar to jellyfin#10884 --- Extracting a media attachment is a disk I/O bottlenecked operation as ffmpeg has to read through the whole file, but usually there is nothing CPU intensive to do. If a file has multiple media attachments, and we want to extract more of them, extracting them one-by-one results in reading the whole file again and again. However ffmpeg can extract multiple streams at once. We can optimize this by extracting the media attachments all at once when only one of them gets queried, then we will have all of them cached for later use. --- Jellyfin clients need fonts for subtitles, and each font is a separate attachment, which causes a lot of re-reads of the file. Certain contents, like anime in a lot of cases, contain 10-50 different attachments; reading the same 2-3 GB file 50 times from a HDD takes a significant amount of time. This change helps a lot in this scenario. Signed-off-by: Attila Szakacs <szakacs.attila96@gmail.com>
alltilla
added a commit
to alltilla/jellyfin
that referenced
this pull request
Feb 18, 2024
Similar to jellyfin#10884 --- Jellyfin clients need fonts for subtitles, and each font is a separate attachment, which causes a lot of re-reads of the file. Certain contents, like anime in a lot of cases, contain 50-80 different attachments. Spawning 80 ffmpeg processes at the same time on the same file might cause swapping on slower HDDs and can bring disk subsystem to a crawl. (For more info, see https://github.com/jellyfin/jellyfin/3215) This change helps a lot in this scenario. Signed-off-by: Attila Szakacs <szakacs.attila96@gmail.com>
|
I was thinking if there is a way to stream subtitles directly from the video file without extraction, or perhaps extracting the subtitle without the need to read the whole file. Do video players like VLC, etc, read the whole file to play subtitles? |
|
For non-SSA/ASS plain text subtitles, it is possible to pass the subtitles to the HLS player via HLS/m3u8. This requires additional efforts in both server and clients. |
alltilla
added a commit
to alltilla/jellyfin
that referenced
this pull request
Feb 24, 2024
Similar to jellyfin#10884 --- Jellyfin clients need fonts for subtitles, and each font is a separate attachment, which causes a lot of re-reads of the file. Certain contents, like anime in a lot of cases, contain 50-80 different attachments. Spawning 80 ffmpeg processes at the same time on the same file might cause swapping on slower HDDs and can bring disk subsystem to a crawl. (For more info, see https://github.com/jellyfin/jellyfin/3215) This change helps a lot in this scenario. Signed-off-by: Attila Szakacs <szakacs.attila96@gmail.com>
crobibero
pushed a commit
that referenced
this pull request
Mar 3, 2024
Similar to #10884 --- Jellyfin clients need fonts for subtitles, and each font is a separate attachment, which causes a lot of re-reads of the file. Certain contents, like anime in a lot of cases, contain 50-80 different attachments. Spawning 80 ffmpeg processes at the same time on the same file might cause swapping on slower HDDs and can bring disk subsystem to a crawl. (For more info, see https://github.com/jellyfin/jellyfin/3215) This change helps a lot in this scenario. Signed-off-by: Attila Szakacs <szakacs.attila96@gmail.com>
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Extracting a subtitle stream is a disk I/O bottlenecked operation as
ffmpeghas to read through the whole file, but usually there is nothing CPU intensive to do.If a file has multiple subtitle streams, and we want to extract more of them, extracting them one-by-one results in reading the whole file again and again.
However
ffmpegcan extract multiple streams at once.We can optimize this by extracting the subtitle streams all at once when only one of them gets queried, then we will have all of them cached for later use.
It is useful for people switching subtitles during playback.
It is even more useful for people who extract all the subtitle streams in advance, for example with the "Subtitle Extract" plugin.
In this case we reduce the extraction time significantly based on the number of subtitle streams in the files, which can be 5-10 in many cases.
Jellyfin logs
ffmpeg logs