[twitter:spaces] Invalid data found when processing input

[miminashii]

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

• I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

• I'm reporting that yt-dlp is broken on a supported site
• I've verified that I'm running yt-dlp version 2023.07.06 (update instructions) or later (specify commit)
• I've checked that all provided URLs are playable in a browser with the same IP and same login details
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
• I've read the guidelines for opening an issue
• I've read about sharing account credentials and I'm willing to share it if required

Region

Japan

Provide a description that is worded well enough to be understood

When trying to download a twitter spaces audio, I encountered the error "Invalid data found when processing input".
I tried several times, but got this same error.
I was running yt-dlp version 2023.07.06.194006.
I would be grateful if someone would fix this issue.

Provide verbose output that clearly demonstrates the problem

• Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
• If using API, add 'verbose': True to YoutubeDL params instead
• Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', '-f', 'ba', '-o', '%(uploader_id)s_%(upload_date)s_%(title)s_%(id)s.%(ext)s', '-P', '.', '--netrc', 'https://twitter.com/i/spaces/1eaKbrQbjoRKX']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@2023.07.06.194006 [92315c037] (zip)
[debug] Python 3.10.4 (CPython arm64 64bit) - macOS-13.4-arm64-arm-64bit (OpenSSL 1.1.1u  30 May 2023)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0
[debug] Optional libraries: certifi-2021.10.08, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1855 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest
Available version: nightly@2023.07.06.194006, Current version: nightly@2023.07.06.194006
Current Build Hash: 938b91103cd7dc7544ea4eadd44d8488ec66e0ab237d0aee211baa5244b16fb9
yt-dlp is up to date (nightly@2023.07.06.194006)
[twitter:spaces] Downloading login page
[twitter:spaces] Downloading flow token
[twitter:spaces] Submitting JS instrumentation response
[twitter:spaces] Submitting username
[twitter:spaces] Submitting password
[twitter:spaces] Submitting account duplication check
[twitter:spaces] Logging in
[twitter:spaces] Extracting URL: https://twitter.com/i/spaces/1eaKbrQbjoRKX
[twitter:spaces] 1eaKbrQbjoRKX: Downloading GraphQL JSON
[twitter:spaces] 28_1664225270709256192: Downloading legacy API JSON
[twitter:spaces] 28_1664225270709256192: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[info] 1eaKbrQbjoRKX: Downloading 1 format(s): 0
[debug] Invoking hlsnative downloader on "https://prod-fastly-ap-northeast-1.video.pscp.tv/Transcoding/v1/hls/uoJ7s877sh5vLzDFNKzftjMMb1_Jd7m5i-ePKC63ZK0LnMLjztwQQCNGMZu7WHDdBYkKHM7slVLUMtGLkQhxvw/non_transcode/ap-northeast-1/periscope-replay-direct-prod-ap-northeast-1-public/audio-space/playlist_16761126492889887124.m3u8?type=replay"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 125
[download] Destination: ./tomeru_ikinone_20230601_あ_1eaKbrQbjoRKX.m4a
[download] 100% of    4.46MiB in 00:00:06 at 705.21KiB/s
[FixupM4a] Correcting container of "./tomeru_ikinone_20230601_あ_1eaKbrQbjoRKX.m4a"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:./tomeru_ikinone_20230601_あ_1eaKbrQbjoRKX.m4a' -map 0 -dn -ignore_unknown -c copy -f mp4 -movflags +faststart 'file:./tomeru_ikinone_20230601_あ_1eaKbrQbjoRKX.temp.m4a'
[debug] ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 14.0.3 (clang-1403.0.22.14.1)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x124633930] Format mov,mp4,m4a,3gp,3g2,mj2 detected only with low score of 1, misdetection possible!
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x124633930] moov atom not found
file:./tomeru_ikinone_20230601_あ_1eaKbrQbjoRKX.m4a: Invalid data found when processing input

ERROR: Postprocessing: file:./tomeru_ikinone_20230601_あ_1eaKbrQbjoRKX.m4a: Invalid data found when processing input
Traceback (most recent call last):
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 3439, in process_info
    replace_info_dict(self.post_process(dl_filename, info_dict, files_to_move))
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 3621, in post_process
    info = self.run_all_pps('post_process', info, additional_pps=info.get('__postprocessors'))
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 3603, in run_all_pps
    info = self.run_pp(pp, info)
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 3581, in run_pp
    files_to_delete, infodict = pp.run(infodict)
  File "/usr/local/bin/yt-dlp/yt_dlp/postprocessor/common.py", line 24, in run
    ret = func(self, info, *args, **kwargs)
  File "/usr/local/bin/yt-dlp/yt_dlp/postprocessor/common.py", line 129, in wrapper
    return func(self, info)
  File "/usr/local/bin/yt-dlp/yt_dlp/postprocessor/ffmpeg.py", line 882, in run
    self._fixup('Correcting container', info['filepath'], [*self.stream_copy_opts(), '-f', 'mp4'])
  File "/usr/local/bin/yt-dlp/yt_dlp/postprocessor/ffmpeg.py", line 863, in _fixup
    self.run_ffmpeg(filename, temp_filename, options)
  File "/usr/local/bin/yt-dlp/yt_dlp/postprocessor/ffmpeg.py", line 374, in run_ffmpeg
    return self.run_ffmpeg_multiple_files([path], out_path, opts, **kwargs)
  File "/usr/local/bin/yt-dlp/yt_dlp/postprocessor/ffmpeg.py", line 329, in run_ffmpeg_multiple_files
    return self.real_run_ffmpeg(
  File "/usr/local/bin/yt-dlp/yt_dlp/postprocessor/ffmpeg.py", line 367, in real_run_ffmpeg
    raise FFmpegPostProcessorError(stderr.strip().splitlines()[-1])
yt_dlp.postprocessor.ffmpeg.FFmpegPostProcessorError: file:./tomeru_ikinone_20230601_あ_1eaKbrQbjoRKX.m4a: Invalid data found when processing input

[pukkandan]

Could you pls give output of ffprobe tomeru_ikinone_20230601_あ_1eaKbrQbjoRKX.m4a. Also, is the file playable?

[bashonly]

I think the 'container': 'm4a_dash' hack doesn't work for every stream. Need to revert to forcing ffmpeg for all spaces downloads, even if not live

Workaround for now: add --downloader ffmpeg

[bashonly]

I have to rename the file to .aac in order for ffprobe to read anything, and then I get:

[aac @ 0x55e120cbac40] Format aac detected only with low score of 1, misdetection possible!
[aac @ 0x55e120cbac40] Estimating duration from bitrate, this may be inaccurate
Input #0, aac, from '20230601_tomeru_ikinone_1eaKbrQbjoRKX__.aac':
  Metadata:
    HydraParticipants: []
    TIT3            : 3894606003.284
    JSONMetadata    : {"HydraVersion":4,"ntp":3894606003.284}
    id3v2_priv.com.apple.streaming.transportStreamTimestamp: \x00\x00\x00\x00\x86\xa9\x9d\xfa
    HydraAudioLevel : [46]
  Duration: 00:06:22.18, bitrate: 97 kb/s
  Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 97 kb/s

[miminashii]

@pukkandan @bashonly

Thank you for your reply.

Could you pls give output of ffprobe tomeru_ikinone_20230601_あ_1eaKbrQbjoRKX.m4a.

Output of ffprobe tomeru_ikinone_20230601_あ_1eaKbrQbjoRKX.m4a is as below.

ffprobe version 6.0 Copyright (c) 2007-2023 the FFmpeg developers
  built with Apple clang version 14.0.3 (clang-1403.0.22.14.1)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.0 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x145705490] Format mov,mp4,m4a,3gp,3g2,mj2 detected only with low score of 1, misdetection possible!
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x145705490] moov atom not found
tomeru_ikinone_20230601_あ_1eaKbrQbjoRKX.m4a: Invalid data found when processing input

Also, is the file playable?

Yes. I could play the file with Elmedia Player.

[miminashii]

@bashonly
I run the command with--downloader ffmpeg and no error occurred. Thank you.

[swyxio]

@miminashii @bashonly can you write a complete guide on how to download twitter spaces now? it seems you guys know what to do but i dont see it documented anywhere. this is what i get right now

[bashonly]

@swyxio
You're running an old version, and now you need to be logged in to access Spaces (yt-dlp can't workaround this)

1. Update yt-dlp, yt-dlp -U
2. pass cookies with your command, e.g. --cookies-from-browser firefox, or use --username and --password to log in
3. add --downloader ffmpeg to your command to counteract [twitter:spaces] Invalid data found when processing input #7536