Skip to content

Commit

Permalink
fftools/ffmpeg: Introduce subtitle filtering and new frame-based subt…
Browse files Browse the repository at this point in the history
…itle encoding

This commit actually enables subtitle filtering in ffmpeg by
sending and receiving subtitle frames to and from a filtergraph.

The heartbeat functionality from the previous sub2video implementation
is removed and now provided by the 'subfeed' filter.
The other part of sub2video functionality is retained by
auto-insertion of the new graphicsub2video filter.

Justification for changed test refs:

- sub2video
  The previous results were incorrect. The command line for this test
  specifies -r 5 (5 frames per second), which is now fulfilled by the
  additional frames in the reference output.
  Example: The first subtitle time is 499000, the second is 15355000,
  which means 0.5s and 15.35s with a difference of 14.85s.
  15s * 5fps = 75 frames and that's now the exact number of video
  frames between these two subtitle events.

- sub2video_basic
  The previous results had some incorrect output because multiple
  frames had the same dts
  The non-empty content frames are visually identical, the different
  CRC is due to the different blending algorithm that is being used.

- sub2video_time_limited
  Subtitle frames are emitted to the filter graphs at a 5 fps rate
  by default. The time limit for this test is 15s * 5fps = 75 frames
  which matches the count in the new ref.

- sub-dvb
  Running ffprobe -show_frames on the source file shows that there
  are 7 subtitle frames with 0 rects in the source at the start
  and 2 at the end. This translates to the 14 and 4 additional
  entries in the new test results.

- filter-overlay-dvdsub-2397
  Overlay results have slightly different CRCs due to different
  blending implementation

- sub-scc
  The first entry is no longer in the output because it is before
  the actual start time and the strim filter removes such entries
  now (like for video and audio)

Signed-off-by: softworkz <softworkz@hotmail.com>
  • Loading branch information
softworkz committed Oct 2, 2022
1 parent 2f815dc commit 1ef9021
Show file tree
Hide file tree
Showing 11 changed files with 3,014 additions and 664 deletions.
606 changes: 320 additions & 286 deletions fftools/ffmpeg.c

Large diffs are not rendered by default.

17 changes: 8 additions & 9 deletions fftools/ffmpeg.h
Original file line number Diff line number Diff line change
Expand Up @@ -378,17 +378,16 @@ typedef struct InputStream {
struct { /* previous decoded subtitle and related variables */
int got_output;
int ret;
AVSubtitle subtitle;
AVFrame *subtitle;
} prev_sub;

struct sub2video {
struct subtitle_kickoff {
int is_active;
int64_t last_pts;
int64_t end_pts;
AVFifo *sub_queue; ///< queue of AVSubtitle* before filter init
AVFrame *frame;
int w, h;
unsigned int initialize; ///< marks if sub2video_update should force an initialization
} sub2video;
} subtitle_kickoff;

AVBufferRef *subtitle_header;

/* decoded data from this stream goes into all those filters
* currently video and audio only */
Expand Down Expand Up @@ -508,6 +507,8 @@ typedef struct OutputStream {
// everything before it should be discarded
int64_t ts_copy_start;

/* subtitle_pts values of the last subtitle frame having arrived for encoding */
int64_t last_subtitle_pts;
// the timebase of the packets sent to the muxer
AVRational mux_timebase;
AVRational enc_timebase;
Expand Down Expand Up @@ -709,8 +710,6 @@ int filtergraph_is_simple(FilterGraph *fg);
int init_simple_filtergraph(InputStream *ist, OutputStream *ost);
int init_complex_filtergraph(FilterGraph *fg);

void sub2video_update(InputStream *ist, int64_t heartbeat_pts, AVSubtitle *sub);

int ifilter_parameters_from_frame(InputFilter *ifilter, const AVFrame *frame);

int ffmpeg_parse_options(int argc, char **argv);
Expand Down
Loading

0 comments on commit 1ef9021

Please sign in to comment.