--write-comments doesn't support unicode characters #2139
Labels
enhancement
New feature or request
Comments
pukkandan
added
enhancement
|
This is correct encoding for JSON files. https://docs.python.org/3/library/json.html#character-encodings That said, I think it makes sense for us to write the characters unencoded |
|
Interesting, I never knew that! |
Ashish0804
added a commit
to Ashish0804/yt-dlp
that referenced
this issue
Jan 1, 2022
commit 8efffaf Author: MinePlayersPE <mineplayerspealt@gmail.com> Date: Sat Jan 1 13:12:33 2022 +0700 [XVideos] Check HLS formats (yt-dlp#2193) Closes yt-dlp#1823 Authored by; MinePlayersPE commit 26f2aa3 Author: Ashish Gupta <39122144+Ashish0804@users.noreply.github.com> Date: Sat Jan 1 02:32:23 2022 +0530 [hotstar] Add extractor args to ignore tags (yt-dlp#2116) Authored by: Ashish0804 commit 3464a27 Author: pgaig <87302379+pgaig@users.noreply.github.com> Date: Fri Dec 31 21:58:23 2021 +0100 [VrtNU] Handle empty title (yt-dlp#2147) Closes yt-dlp#2146 Authored by: pgaig commit 497d77e Author: Ashish Gupta <Ashish08@protonmail.com> Date: Fri Dec 31 10:41:42 2021 +0530 [KelbyOne] Add extractor (yt-dlp#2181) Closes yt-dlp#2170 Authored by: Ashish0804 commit 9040e2d Author: LE <llacb47@users.noreply.github.com> Date: Fri Dec 31 15:11:35 2021 -0500 [mixcloud] Detect restrictions (yt-dlp#2169) Authored by; llacb47 commit 6134fbe Author: MinePlayersPE <mineplayerspealt@gmail.com> Date: Sat Jan 1 03:10:46 2022 +0700 [TikTok] Pass cookies to formats (yt-dlp#2171) Closes yt-dlp#2166 Authored by: MinePlayersPE commit cfcf60e Author: MinePlayersPE <mineplayerspealt@gmail.com> Date: Sat Jan 1 03:09:30 2022 +0700 [BiliIntl] Add login (yt-dlp#2172) and misc improvements Authored by: MinePlayersPE commit 4afa3ec Author: Felix S <felix.von.s@posteo.de> Date: Fri Dec 31 20:06:45 2021 +0000 [extractor] Detect more subtitle codecs in MPD manifests (yt-dlp#2174) Authored by: fstirlitz commit 11aa91a Author: MinePlayersPE <mineplayerspealt@gmail.com> Date: Thu Dec 30 11:20:17 2021 +0700 [TikTok] Fix extraction for sigi-based webpages (yt-dlp#2164) Fixes: yt-dlp#2133 Authored by: MinePlayersPE commit abbeeeb Author: pukkandan <pukkandan.ytdlp@gmail.com> Date: Thu Dec 30 08:43:40 2021 +0530 [outtmpl] Alternate form for `D` and fix suffix's case Fixes: yt-dlp#2085 (comment), https://github.com/yt-dlp/yt-dlp/pull/2132/files#r775729811 commit 2c539d4 Author: pukkandan <pukkandan.ytdlp@gmail.com> Date: Thu Dec 30 08:15:48 2021 +0530 [cookies] Fix bug when keyring is unspecified Closes yt-dlp#2167 commit 042931a Author: pukkandan <pukkandan.ytdlp@gmail.com> Date: Thu Dec 30 08:15:07 2021 +0530 Allow escaped `,` in `--extractor-args` Closes yt-dlp#2152 commit 96f13f0 Author: MinePlayersPE <mineplayerspealt@gmail.com> Date: Thu Dec 30 05:00:44 2021 +0700 [TikTok] Change app version (yt-dlp#2161) Closes yt-dlp#2133, yt-dlp#2135 Authored by: MinePlayersPE, llacb47 commit 4b93532 Author: u-spec-png <54671367+u-spec-png@users.noreply.github.com> Date: Tue Dec 28 20:42:14 2021 +0000 [Drooble] Add extractor (yt-dlp#1547) Closes yt-dlp#1527 Authored by: u-spec-png commit dd5e60b Author: u-spec-png <54671367+u-spec-png@users.noreply.github.com> Date: Tue Dec 28 18:58:06 2021 +0000 [Instagram] Add story/highlight extractor (yt-dlp#2006) Fixes ytdl-org/youtube-dl#25575 Authored by: u-spec-png commit e540c56 Author: MinePlayersPE <mineplayerspealt@gmail.com> Date: Tue Dec 28 09:38:23 2021 +0700 [TikTok] Fallback to feed API endpoint (yt-dlp#2142) Authored by: MinePlayersPE Workaround for yt-dlp#2133 commit 45d86ab Author: pukkandan <pukkandan.ytdlp@gmail.com> Date: Tue Dec 28 04:21:13 2021 +0530 Allow unicode characters in `info.json` Closes yt-dlp#2139 commit f02d24d Author: Pierre Mdawar <pierre@mdawar.dev> Date: Tue Dec 28 03:38:31 2021 +0530 [utils] Fix `format_bytes` output for Bytes (yt-dlp#2132) Authored by: pukkandan, mdawar commit ceb9832 Author: pukkandan <pukkandan.ytdlp@gmail.com> Date: Tue Dec 28 02:52:11 2021 +0530 Don't treat empty containers as `None` in `sanitize_info` commit 7537e35 Author: pukkandan <pukkandan.ytdlp@gmail.com> Date: Tue Dec 28 02:49:02 2021 +0530 [gfycat] Fix `uploader`
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Checklist
Description
The --write-comments option does not support unicode characters, causing non-Ascii based comments to be written as unicode numbers (e.g. "text": "\u3070\u3076\u3061\u3001\u3053\u3093\u306a\u306b\u304b\u308f\u3044\u304b\u3063\u305f\u3093\u3060\u3001\u597d\u304d")
This adds an additional to step to anyone wanting to grab video comments in languages that are not ascii based.
There is already unicode support in many other commands, so I imagine the only change needed would be to utilitize whatever method of writing unicode is being used elsewhere and use it in --write-comments as well.
[info.json output. Appended .txt for upload to github](url
初動画はこのくらいシンプルなほうがいいんじゃないかと思って [wJF-Lr-vNV0].info.json.txt
)
Verbose log
The text was updated successfully, but these errors were encountered: