cant download from qq

[jubalh]

Hi
I tried:

youtube-dl http://v.qq.com/page/y/i/0/y01647bfni0.html
[generic] y01647bfni0: Requesting header
WARNING: Falling back on generic information extractor.
[generic] y01647bfni0: Downloading webpage
[generic] y01647bfni0: Extracting information
ERROR: Unsupported URL: http://v.qq.com/page/y/i/0/y01647bfni0.html

Supported sites lists some qq sites as supported though.

[dstftw]

Only QQMusic is supported atm.

[yan12125]

From #9210: http://v.qq.com/cover/t/t1rmktedqbey1g5/d0020cv1s3t.html should be supported too.

[TRox1972]

URL seems to be at http://203.205.148.75/vhot2.qqvideo.tc.qq.com/y01647bfni0.p703.1.mp4?sdtfrom=v1010&vkey=539DB2C1BE3027DE015E02A71A5912C6E6080391DE5ECA287666D668B05DCC7834DFB41088B68A3DD6EA1E7AC672EECBD5A4FDD53E1790AAA6687B24E8FDAD4E0AF518AC7AC7AB1F819C2A4C3F63E4766A5834200B66989E; not able to find the key as of yet.

[yan12125]

AFAIK the key parameter is computed by some algorithms in the SWF player. It requires decoding codes generating by FlasCC/CrossBridge.

[ipetu]

In what ways can download QQ video?

[yan12125]

Currently impossible with youtube-dl.

[remitamine]

AFAIK the key parameter is computed by some algorithms in the SWF player. It requires decoding codes generating by FlasCC/CrossBridge.

the key can be extracted from the qq api, but it will take time because there will be many request to extract the keys for all parts of all formats, the problem is that handling the formats using multipart _type is not natural.

[AlexStacker]

AFAIK the key parameter is computed by some algorithms in the SWF player. It requires decoding codes generating by FlasCC/CrossBridge.

I found a lib in git. it can get qq video info and key info. It looks like decomplier the QQ SWF player and write the algorithms by python.

https://github.com/lvqier/crawlers/blob/master/txsp/echo_ckeyv3.py

so it tegrated into youtube-dl ? @yan12125

[yan12125]

@loopArray This file is licensed under MIT, which is not compatible with youtube-dl's UNLICENSE. If you want to integrate it into youtube-dl, you need to ask its original author to re-license it to a compatible one, for example UNLICENSE or Public Domain.

[AlexStacker]

when I see the LICENSE is different, I almost had a nervous breakdown. is some way to solution the problem? as you know the qq is not allow we download him videos, but we continue download it any way. @yan12125

[yan12125]

Either ask the original author (lvqier), or write another version.

[AlexStacker]

@yan12125 when i write an issue to the author lvqier, other repositories ykdl's author give me a other one, it can download qq videos. I have review the code and find the key is not required! its qq's bug or is really not required? now, i can download qq video by using ykdl, but I also think it's necessary integrate it into youtube-dl. do you have want to try?^_^

[yan12125]

ykdl is licensed under MIT too. If you are going to copy & paste codes from ykdl, please ask for explicit permission. If you're going to write your own version, that's fine :)

[remitamine]

I have review the code and find the key is not required!

if you look in the code you will find in qq_get_final_url requests to extract the keys from the qq server.

[AlexStacker]

It's the ckey field, the ckey is calculated by pass params. but it looks like given null character string is well to get the key by get_stream_info you mean. @remitamine

[remitamine]

It's the ckey field.

may be it's only used in the flash player, i didn't see any reference to ckey in the html5 requests.
i will post the code that i already have in case that someone want to continue working on it(but i think that handling multipart should be fixed before continue working on this):

# coding: utf-8
from __future__ import unicode_literals

import re

from .common import InfoExtractor
from ..utils import (
    int_or_none,
    float_or_none,
)


class QQIE(InfoExtractor):
    _VALID_URL = r'https?://(?:.*\.)?qq\.com/.*?vid=(?P<id>[^&]+)'

    def _real_extract(self, url):
        vid = self._match_id(url)

        def get_json(jsonp):
            return self._search_regex(r'({.*})', jsonp, None)

        video_data = self._download_json(
            'http://h5vv.video.qq.com/getinfo', vid, transform_source=get_json, query={
                'vid': vid,
                'platform': 10201,
                'otype': 'json',
                # 'defn': 'mp4', # default value is mp4, possible values: sd, hd, shd, fhd
            })
        video_info = video_data['vl']['vi'][0]
        title = video_info['ti']
        filename = video_info['fn']
        vkey = video_info['fvkey']
        width = int_or_none(video_info.get('vw'))
        height = int_or_none(video_info.get('vh'))
        server_info = video_info['ul']['ui'][0]
        base_url = server_info['url']
        vt = server_info['vt']
        info = {}
        current_format = next(fi for fi in video_data['fl']['fi'] if fi['sl'] == 1)
        parts = video_info.get('cl', {}).get('ci', [])
        if parts:
            entries = []
            for index, p in enumerate(parts, 1):
                part_filename = re.sub(r'(.*)\.(\w)', r'\1.%d.\2' % index, filename)
                vkey = self._download_json('http://h5vv.video.qq.com/getkey', vid, transform_source=get_json, query={
                    'vid': vid,
                    'platform': 10201,
                    'otype': 'json',
                    'format': current_format['id'],
                    'filename': part_filename,
                    'vt': vt,
                })['key']
                entries.append({
                    'id': '%s_part%d' % (vid, index),
                    'format_id': current_format.get('name'),
                    'title': title,
                    'url': base_url + part_filename + '?vkey=' + vkey,
                    'duration': float_or_none(p.get('cd')),
                    'width': width,
                    'height': height,
                })
            info.update({
                '_type': 'multi_video',
                'entries': entries,
            })
        else:
            info = {
                'format_id': current_format.get('name'),
                'url': base_url + filename + '?vkey=' + vkey,
                'duration': float_or_none(video_info.get('td')),
                'width': width,
                'height': height,
            }
        info.update({
            'id': vid,
            'title': title,
        })
        return info

[lvqier]

I'd love to share my work, but is re-licensing the only way?

[yan12125]

is re-licensing the only way?

Compatible licenses include UNLICENSE and CC0. For others, re-licensing is necessary.

[lvqier]

@yan12125 I have re-licensed my repository, please enjoy it.

[yan12125]

I guess you mean https://github.com/lvqier/crawlers? Much thanks for opening it up!

[lvqier]

@yan12125 My pleasure~

[jubalh]

Now that @lvqier relicensed his work, is there any progress on this issue?

[yan12125]

Sorry for the delay. I got a WIP work based on @lvqier's script but it's far from complete yet: https://github.com/yan12125/youtube-dl/tree/wip-qq

[yan12125]

I find something interesting in @lvqier's work. In a function called "sanbox API", the remote server http://sandbox.xinfan.org/ returns something like this:

{
    "message": "合作请联系: lvqier@gmail.com",
    "result": "44000100789C062114B63F5C1EEA7BDA3B88F9C778F0839AC6925011B4F54C1DCD666A0DA1CA9704528A9D75DBC6E0BC1B00C979C5C5BF1199E32718E0A2321DB2BB0D6150AC"
}

From the codes, seems this API is equivalent to a CModule function in Tencent's SWF. A concern is that some users may consider 3rd party API servers not trustworthy. @lvqier Is there a reason to use another API server rather than to implement the corresponding CModule function directly in Python, just like your earlier versions?

[lvqier]

@yan12125, I do not release my latest code to public because it will not work even if you have right ckey. Now server of Tencent is validating guid, which is generated randomly at the first time when you visit the site of QQ video.

[yan12125]

Hmm. Did you mean the PLAYER_GUID variable? Sounds like the cache system in youtube-dl can help.

---

Note to myself: https://v.qq.com/x/page/i0137wt2799.html doesn't work with wip-qq. Seems I over-simplify codes in txsp.py

Ref: soimort/you-get#1685