[Broken] Twitter: File name too long error

[hadmut]

Checklist

• [ X] I'm reporting a broken site support
• [ X] I've verified that I'm running yt-dlp version 2021.10.10
• [ X] I've checked that all provided URLs are alive and playable in a browser
• [ X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
• [ X] I've searched the bugtracker for similar issues including closed ones
• [ X] I've read the opening an issue section in CONTRIBUTING.md
• [ X] I have given an appropriate title to the issue

Verbose log

yt-dlp -Uv 'https://twitter.com/Hyundai_Global/status/1448423466072477699'
[debug] Command-line config: ['-Uv', 'https://twitter.com/Hyundai_Global/status/1448423466072477699']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] yt-dlp version 2021.10.10 (zip)
[debug] Python version 3.8.10 (CPython 64bit) - Linux-5.4.0-88-generic-x86_64-with-glibc2.29
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4, rtmpdump 2.4
[debug] Optional libraries: Crypto, keyring, sqlite
[debug] ANSI escape support: stdout = True, stderr = True
[debug] Proxy map: {}
yt-dlp is up to date (2021.10.10)
[debug] [twitter] Extracting URL: https://twitter.com/Hyundai_Global/status/1448423466072477699
[twitter] 1448423466072477699: Downloading guest token
[twitter] 1448423466072477699: Downloading JSON metadata
[twitter] 1448423466072477699: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 1448423466072477699: Downloading 1 format(s): hls-2176
[debug] Invoking downloader on "https://video.twimg.com/amplify_video/1448423326188199938/pl/720x900/-gU4SBjDJN9-8e37.m3u8"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 20
[download] Destination: Hyundai Worldwide - Launching Re -Style 2021! We connect Seoul and Paris by partnering with the shops @leclaireur and Boon The Shop. Eco-friendly materials incorporated into our IONIQ 5 were used to enhance wearability and demonstrate the potential in fashion.  #HyundaiReStyle #restyle2021 #Hyundai [1448423466072477699].mp4
ERROR: unable to download video data: [Errno 36] File name too long: 'Hyundai Worldwide - Launching Re -Style 2021! We connect Seoul and Paris by partnering with the shops @leclaireur and Boon The Shop. Eco-friendly materials incorporated into our IONIQ 5 were used to enhance wearability and demonstrate the potential in fashion.  #HyundaiReStyle #restyle2021 #Hyundai [1448423466072477699].mp4.ytdl'
Traceback (most recent call last):
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 2800, in process_info
    success, real_download = self.dl(temp_filename, info_dict)
  File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 2508, in dl
    return fd.download(name, new_info, subtitle)
  File "/usr/local/bin/yt-dlp/yt_dlp/downloader/common.py", line 397, in download
    ret = self.real_download(filename, info_dict)
  File "/usr/local/bin/yt-dlp/yt_dlp/downloader/hls.py", line 136, in real_download
    self._prepare_and_start_frag_download(ctx, info_dict)
  File "/usr/local/bin/yt-dlp/yt_dlp/downloader/fragment.py", line 83, in _prepare_and_start_frag_download
    self._prepare_frag_download(ctx)
  File "/usr/local/bin/yt-dlp/yt_dlp/downloader/fragment.py", line 208, in _prepare_frag_download
    self._write_ytdl_file(ctx)
  File "/usr/local/bin/yt-dlp/yt_dlp/downloader/fragment.py", line 103, in _write_ytdl_file
    frag_index_stream, _ = sanitize_open(self.ytdl_filename(ctx['filename']), 'w')
  File "/usr/local/bin/yt-dlp/yt_dlp/utils.py", line 2068, in sanitize_open
    stream = open(encodeFilename(filename), open_mode)
OSError: [Errno 36] File name too long: 'Hyundai Worldwide - Launching Re -Style 2021! We connect Seoul and Paris by partnering with the shops @leclaireur and Boon The Shop. Eco-friendly materials incorporated into our IONIQ 5 were used to enhance wearability and demonstrate the potential in fashion.  #HyundaiReStyle #restyle2021 #Hyundai [1448423466072477699].mp4.ytdl'

Description

when downloading videos from Twitter, yt-dlp fetches the Tweet text and creates a filename for the video out of it.

But if the tweet is long, the created filename becomes longer than the underlying file system accepts and yt-dlp aborts with an error message.

Workaround: give a shorter file name manually with the -o option, but this is cumbersome and requires manual interaction, and if not noticed in time, can lead to not having downloaded the video if the tweet has been deleted meanwhile.

Poor man's solution: Limit the full file name length to 255 byte.

Good solution:

perform an statfs for the target file system and determine the maximum file name length of the target directory.

See
man 2 statfs
Kernel/stdc-function to get a descriptor of the file system, including the maximum file name length.

This function also exists in Python, see

https://www.geeksforgeeks.org/python-os-statvfs-method/

os.statvfs(path)

The returned os.statvfs_result object has following attributes:

f_bsize: It represents the file system block size
f_frsize: It represents the fragment size
f_blocks It represents the size of fs in f_frsize units
f_bfree: It represents the number of free blocks
f_bavail: It represents the number of free blocks for unprivileged users
f_files: It represents the number of inodes
f_ffree: It represents the number of free inodes
f_favail: It represents the number of free inodes for unprivileged users
f_fsid: It represents the file system ID
f_flag: It represents the mount flags
**f_namemax: It represents the maximum filename length**

regards

[pukkandan]

1. This is a feature request, not an extractor bug

[X] I've searched the bugtracker for similar issues including closed ones

2. Duplicate of [Feature request] Handle Long filenames in default template and temporary files #1136. There is already an issue tracking this and other duplicate issues like this!!

3. You can set an output template like %(title).200s to limit the filename

4. os.statvfs is not cross platform

[hairycactus]

Is it possible for yt-dlp to automatically truncate the output filename to fit into a safe-maximum character length ?

For some sites (esp. Twitter), the default output filenames are too long (= length of entire tweet) most of the time, unless the user specifies otherwise using the -o option.

You can set an output template like %(title).200s to limit the filename

Regarding the output option & the config file, may I check what the correct config filename is for the portable Windows EXE binary (both yt-dlp.exe & yt-dlp_min.exe) ?

I had previously tried the following filenames (as suggested at ReadMe.md), & placed the file beside the EXE in the same folder. But yt-dlp ignores all of them.

1. \yt-dlp.conf
2. \config.txt
3. \yt-dlp.conf.txt
4. \config (file with no extension)
5. All of the above, but in an additional \yt-dlp\ subfolder

Config File Commands:

# Shorten Video Filenames
-o "%(title).200s.%(ext)s"

# Disable Filesystem Caching
--no-cache-dir

Or is there something wrong with the command ? I also tried: -o %(title).200s.

Tested on: Win 10 x64

[pukkandan]

Is it possible for yt-dlp to automatically truncate the output filename to fit into a safe-maximum character length ?

I believe there is already a feature request open for this. Due to various reasons, actually implementing this is a bit hard, so custom config is the best solution for now

Regarding the output option & the config file, may I check what the correct config filename is for the portable Windows EXE binary (both yt-dlp.exe & yt-dlp_min.exe) ?

The name of the exe doesn't matter. The portable config file should be named yt-dlp.conf (be careful it isn't yt-dlp.conf.txt - windows hides the extension by default) and put in the same folder as the exe (no sub-folders). You can see whether the config is correctly loaded in the verbose output (-v)

[MintMain21]

I've noticed that YT-DLP can't detect embedded Twitter vids in a News Article the same way it can YouTube Videos, and I suspect that may be related to this.

Was going to make this a separate issue, but I didn't want to be redundant.