EIA-608 CC in iTunes U exported h264 files are not recognized

[lilunxm12]

Description of problem or feature request

Tried using Handbrake 1.1 to convert some h264 m4v exported from iTunes U, the CC wasn't showed as available subtitles although activity log showed the CC is detected.

Steps to reproduce the problem

Download the videos from Stanford CS193 course in iOS iTunes U app, export a video to computer and open with Handbrake.

HandBrake version (e.g., 1.0.0)

1.1

Operating system and version (e.g., Ubuntu 16.04 LTS, macOS 10.3 High Sierra, Windows 10 Creators Update)

macOS 10.13.4

Error message text or screenshot

No available subtitle listed

The mediainfo output of original m4v:

Luns-Air:CS193 lilun$ mediainfo "Friday Session 2. Github and Source Control Workflow.m4v"
General
Complete name                            : Friday Session 2. Github and Source Control Workflow.m4v
Format                                   : MPEG-4
Codec ID                                 : M4V  (M4V /M4A /mp42/isom)
File size                                : 368 MiB
Duration                                 : 24 min 51 s
Overall bit rate                         : 2 071 kb/s
Encoded date                             : UTC 2017-12-21 18:54:31
Tagged date                              : UTC 2017-12-21 19:20:04

Video
ID                                       : 2
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings                          : CABAC / 2 Ref Frames
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 2 frames
Format settings, GOP                     : M=2, N=30
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 24 min 51 s
Bit rate                                 : 1 933 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.031
Stream size                              : 344 MiB (93%)
Title                                    : Core Media Video
Encoded date                             : UTC 2017-12-21 18:54:31
Tagged date                              : UTC 2017-12-21 19:20:04
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio
ID                                       : 1
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : mp4a-40-2
Duration                                 : 24 min 51 s
Source duration                          : 24 min 51 s
Bit rate mode                            : Constant
Bit rate                                 : 132 kb/s
Nominal bit rate                         : 160 kb/s
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 23.5 MiB (6%)
Source stream size                       : 23.5 MiB (6%)
Title                                    : Core Media Audio
Language                                 : English
Encoded date                             : UTC 2017-12-21 18:54:31
Tagged date                              : UTC 2017-12-21 19:20:04

Text
ID                                       : 3-CC1
Format                                   : EIA-608
Muxing mode                              : Final Cut
Codec ID                                 : c608
Duration                                 : 24 min 51 s
Source duration                          : 24 min 50 s
Bit rate mode                            : Constant
Stream size                              : 0.00 Byte (0%)
Source stream size                       : 23.8 KiB (0%)
Title                                    : Core Media Closed Caption
Language                                 : English
Forced                                   : No
Encoded date                             : UTC 2017-12-21 18:54:31
Tagged date                              : UTC 2017-12-21 19:20:04
CaptionServiceName                       : CC1
Source_Delay                             : 234
Source_Delay_Source                      : Container

HandBrake Activity Log required (see https://handbrake.fr/docs/en/latest/help/activity-log.html)

[19:06:15] macgui: Handbrake Version: 1.1.0  (2018040700)
[19:06:15] hb_init: starting libhb thread
[19:06:15] hb_init: starting libhb thread
[19:06:21] macgui: trying to open a folder or file
[19:06:21] macgui: ScanCore scanning titles with a duration of 10 seconds or more
[19:06:21] CPU: Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz
[19:06:21]  - Intel microarchitecture Broadwell
[19:06:21]  - logical processor count: 4
[19:06:21] hb_scan: path=/Users/lilun/Downloads/CS193/Friday Session 2. Github and Source Control Workflow.m4v, title_index=0
udfread ERROR: ECMA 167 Volume Recognition failed
disc.c:323: failed opening UDF image /Users/lilun/Downloads/CS193/Friday Session 2. Github and Source Control Workflow.m4v
disc.c:424: error opening file BDMV/index.bdmv
disc.c:424: error opening file BDMV/BACKUP/index.bdmv
[19:06:21] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 5.0.3
libdvdread: Encrypted DVD support unavailable.
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
libdvdnav: vm: failed to read VIDEO_TS.IFO
[19:06:21] dvd: not a dvd - trying as a stream/file instead
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/lilun/Downloads/CS193/Friday Session 2. Github and Source Control Workflow.m4v':
  Metadata:
    major_brand     : M4V 
    minor_version   : 1
    compatible_brands: M4V M4A mp42isom
    creation_time   : 2017-12-21 18:54:31
  Duration: 00:24:51.22, start: 0.000000, bitrate: 2070 kb/s
    Stream #0:0(eng): Audio: aac (LC) [mp4a / 0x6134706D]
      48000 Hz, stereo, fltp, 132 kb/s (default)
    Metadata:
      creation_time   : 2017-12-21 18:54:31
      handler_name    : Core Media Audio
    Stream #0:1(und): Video: h264 (High) [avc1 / 0x31637661]
      yuv420p, tv, bt709/bt709/bt709
      1920x1080 [PAR 1:1 DAR 16:9], 1933 kb/s, PAR 1:1 DAR 16:9
      29.97 fps, 30k tbn (default)
    Metadata:
      creation_time   : 2017-12-21 18:54:31
      handler_name    : Core Media Video
    Stream #0:2(eng): Subtitle: c608 / 0x38303663, 0 kb/s (default)
    Metadata:
      creation_time   : 2017-12-21 18:54:31
      handler_name    : Core Media Closed Caption
[19:06:21] add_ffmpeg_subtitle: unknown subtitle stream type: 0x0
[19:06:21] scan: decoding previews for title 1
[19:06:21] scan: audio 0x0: aac, rate=48000Hz, bitrate=132449 English (AAC) (2.0 ch)
[19:06:22] scan: 10 previews, 1920x1080, 29.970 fps, autocrop = 0/0/0/0, aspect 16:9, PAR 1:1
[19:06:22] libhb: scan thread found 1 valid title(s)
[19:06:22] macgui: ScanCore scan done

[sr55]

I don't believe EIA-608 is supported by upstream. You could use something like CCExtractor maybe and bring it back in as an SRT file.

Marking as an upstream enhancement for now.

[lilunxm12]

@sr55
Thanks for reply.

The Handbrake documentation (https://handbrake.fr/docs/en/1.1.0/advanced/subtitles.html) states handbrake supports CEA-608 and according to wikipedia CEA-608 and EIA-608 are just different names for the same thing, so I assumed Handbrake should handle it properly. I'm completely noob to subtitles though, correct me if I'm wrong.

I did a quick google search with ffmpeg+CC and found this stackoverflow question:
https://stackoverflow.com/questions/3169910/can-ffmpeg-extract-closed-caption-data
it seems like ffmpeg can only decode CC in very limited format (.ts). In my own case, using ffmpeg directly does recognized the CC but failed to extract.

Luns-Air:Downloads lilun$ ffmpeg -f lavfi -i "movie=input.m4v[out0+subcc]" -map s output.srt
ffmpeg version 3.4.2 Copyright (c) 2000-2018 the FFmpeg developers
  built with Apple LLVM version 9.1.0 (clang-902.0.39.1)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4.2 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --disable-jack --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libx265 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, lavfi, from 'movie=input.m4v[out0+subcc]':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 30k tbn, 30k tbc
    Stream #0:1: Subtitle: eia_608
File 'output.srt' already exists. Overwrite ? [y/N] y
Output #0, srt, to 'output.srt':
  Metadata:
    encoder         : Lavf57.83.100
    Stream #0:0: Subtitle: subrip (srt)
    Metadata:
      encoder         : Lavc57.107.100 srt
Stream mapping:
  Stream #0:1 -> #0:0 (eia_608 (cc_dec) -> subrip (srt))
Press [q] to stop, [?] for help
[Parsed_movie_0 @ 0x7fa3147006e0] EOF timestamp not reliable
size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)

I agree it should be an upstream enhancement for now, but to me the Handbrake documentation needs to be revised to. The wording "CEA-608 Closed Captions (DVD and some ATSC streams)" does hint a .ts format but isn't clear enough. And "Embedded SSA/SRT/CC within files (such as mkv or mp4)" is just misleading.

[nhjm449]

HandBrake's CEA-608 support likely refers to captions stored within the video stream (i.e. as a SEI unit), whereas the captions stored within your MP4 are a stored as a separate stream.

Adding AV_CODEC_ID_EIA_608 (which c608 should be detected as) support to add_ffmpeg_subtitle() (treating it as CC608SUB or very similar) may be a solution here.

(Also, +subcc's job appears to be to copy video-stream-embedded CEA-608 data into a separate pseudo-stream, at which point the cc_dec decoder can parse the data. I do wonder if a plain ffmpeg -i input.m4v output.srt would work in your case.)

[bradleysepos]

Another user reported this problem, but the track is not identified by ffprobe/mediainfo. Apparently there is also case where this type of sub is part of the video track and not identified as a separate track.

#ffmpeg-devel:

[10:52:26] nevcairiel: if the CC data is actually inside the video stream, then ffmpeg will never show it as a seperate track
[10:52:31] nevcairiel: its up to the API user to extract i t
[10:57:54] nevcairiel: the main problem there being that the CC data is only available after decoding, not through demuxing
[10:58:09] nevcairiel: someone wanted to make a bitstream filter to extract it without decoding iirc, but that never got anywhere

@jstebbins Ideas?

[jstebbins]

If the CC is in the video stream and it's not being detected, then I would need a sample to diagnose. HandBrake doesn't support CC packaged any other way than embedded in mpeg2 user data or h.264/5 SEI.

[cehoyos]

Is there a sample for this issue? Without a sample, I am not sure how this could be fixed...

[FossPrime]

Same issue happens with MOV files exported using the "Master" prores 422 setting in Final Cut.

Compressor detects them, quicktime can play them. Looks like there might be some bug with ffmpeg/MP4Box/ccextractor.

The ONLY way I was able to get anything but Apple products to make sense of it, was to convert the container to MP4 ffmpeg -i Master.mov -c copy -map 0 out.mp4 and then use ccextractor to convert the subtitles to something useful. MP4Box doesn't recognize the EIA-608 track in a MOV, hence why ccextrator failed.

This only works in FFMpeg 3, stopped working in 4. Apparently it enforces MP4 codec support better, and makes prores, eia_608 and pcm streams fail.

Update: For some reason ffmpeg -i Master.mov -c copy -map 0 out.mov works with ccextractor/MP4Box. It's as if it converts from proprietary mov to open source friendly mov.