openwrt - youtube-dl charset force utf-8 in filesystem ?

[patriquepagano]

Hi! sorry for my bad english..

I'm compile openwrt with locale support
Base system
busybox
customize
busbox settings
general configureation
x enable locale support
x support unicode
x $Lang enviroment
in openwrt terminal..
charsetConfig="LC_ALL=en_US.UTF-8
LANG=en_US.UTF-8
LANGUAGE=en_US.UTF-8
LC_CTYPE=en_US.UTF-8"
echo -n "$charsetConfig" > /root/.profile
echo -n "$charsetConfig" > /root/.bashrc
echo -n "$charsetConfig" > /etc/environment
mkdir /etc/default
echo -n "$charsetConfig" > /etc/default/locale

now I can create accent files in bash or ash terminal etc..

but youtube-dl insist in write filename in ascii what I have to do to correct this?

/tmp/x

!/bin/bash

touch /mnt/sda1/.Storage/downloads/test-èâtôasdção
cd /mnt/sda1/.Storage/downloads
youtube-dl -v --encoding UTF-8 -f 36 --write-thumbnail -c -o '%(title)s.%(ext)s' https://www.youtube.com/watch?v=muYvMHLlNEY

root@PersonalRouter:/tmp# bash /tmp/x
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'--encoding', u'UTF-8', u'-f', u'36', u'--write-thumbnail', u'-c', u'-o', u'%(title)s.%(ext)s', u'https://www.youtube.com/watch?v=muYvMHLlNEY']
[debug] Encodings: locale ASCII, fs ASCII, out ASCII, pref UTF-8
[debug] youtube-dl version 2016.04.24
[debug] Python version 2.7.3 - Linux-3.10.49-mips-with-libc0
[debug] exe versions: ffmpeg 0.11.5, ffprobe 0.11.5
[debug] Proxy map: {}
[youtube] muYvMHLlNEY: Downloading webpage
[youtube] muYvMHLlNEY: Downloading video info webpage
[youtube] muYvMHLlNEY: Extracting video information
[youtube] muYvMHLlNEY: Downloading MPD manifest
[youtube] muYvMHLlNEY: Downloading thumbnail ...
[youtube] muYvMHLlNEY: Writing thumbnail to: PEGADINHA - QUER SAIR NA MÃO (açaí na mão) ft. GienTV.jpg
[debug] Invoking downloader on u'https://r2---sn-nv0uiuxji-2v5l.googlevideo.com/videoplayback?signature=6DE90D304B1F89F2265EAAC08ECCB71BCAE24298.13A83578773D93D572FF25DEC04BFE72A0395218&fexp=9407059%2C9416126%2C9416891%2C9422596%2C9428398%2C9431012%2C9431865%2C9432425%2C9433096%2C9433946%2C9434290&key=yt6&mime=video%2F3gpp&initcwndbps=57500&requiressl=yes&expire=1461802878&sver=3&dur=213.809&ipbits=0&lmt=1446840528883194&itag=36&pl=21&upn=wgGM4YmooUM&source=youtube&pcm2cms=yes&id=o-AGmykP9C3ViVZiQu2qKK8oC3ulxzxosT6l6Qpyct8uZS&mn=sn-nv0uiuxji-2v5l&mm=31&ms=au&sparams=dur%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpcm2cms%2Cpl%2Crequiressl%2Csource%2Cupn%2Cexpire&mv=m&mt=1461781214&ip=186.208.148.218&ratebypass=yes'
[download] Destination: PEGADINHA - QUER SAIR NA MÃO (açaí na mão) ft. GienTV.3gp
[download] 100% of 5.78MiB in 00:12

the file:
"/mnt/sda1/.Storage/downloads/PEGADINHA - QUER SAIR NA MO (aa na mo) ft. GienTV.3gp"
/mnt/sda1/.Storage/downloads/test-èâtôasdção

I need the file title write in utf8 or any solution to get accents.

thanks for this great youtube-dl code!

[yan12125]

Could you add one more line:

LC_ALL=en_US.UTF-8

And try it again?

[patriquepagano]

hello friend!

I add this line this way

charsetConfig="LC_ALL=en_US.UTF-8
LANG=en_US.UTF-8
LANGUAGE=en_US.UTF-8
LC_CTYPE=en_US.UTF-8"
echo -n "$charsetConfig" > /root/.profile
echo -n "$charsetConfig" > /root/.bashrc
echo -n "$charsetConfig" > /etc/environment
mkdir /etc/default
echo -n "$charsetConfig" > /etc/default/locale

thanks for your help and time!

[yan12125]

Just to check again. Is it working now?

[patriquepagano]

no. :(

my new try I add new option in compile busybox
Global build settings ---> Compile with full language support

and install kmod-nls-utf8 to openwrt..

and will try again

[patriquepagano]

:(
[debug] Command-line args: [u'-v', u'--encoding', u'UTF-8', u'-f', u'36', u'--write-thumbnail', u'-c', u'-o', u'%(title)s.%(ext)s', u'https://www.youtube.com/watch?v=muYvMHLlNEY']
[debug] Encodings: locale ASCII, fs ASCII, out ASCII, pref UTF-8
[debug] youtube-dl version 2016.04.24
[debug] Python version 2.7.3 - Linux-3.10.49-mips-with-libc0

how to change out ASCII ?

[yan12125]

youtube-dl uses fs encoding to encode filenames. Could you check your system is using glibc? LC_ALL should work on glibc but may not work on other libc implementations. If there's no way to change the filesystem encoding via environment variables, youtube-dl may need a new option to force the filesystem encoding.
If you can install Python 3.x on OpenWrt, please also try with it. In general Python 3.x has better Unicode support over Python 2.x.