[QingTing] Add new extractor #31021

changren-wcr · 2022-06-12T03:11:52Z

Please follow the guide below

You will be asked some questions, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your pull request (like that [x])
Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

Searched the bugtracker for similar pull requests
Read adding new extractor tutorial
Read youtube-dl coding conventions and adjusted the code to meet them
Covered the code with tests (note that PRs without tests will be REJECTED)
Checked the code with flake8

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

I am the original author of this code and I am willing to release it under Unlicense
I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

Bug fix
Improvement
New extractor
New feature

Description of your pull request and other information

This pull request adds qingting.fm, probably the largest podcast website in China.

dirkf

Thanks for your work.

I'm not sure how this extraction is meant to work.

I can play a link from the site like https://www.qingting.fm/channels/214008/programs/7016167/ in the UK but the .../vchannels/... links just give a page whose source is OK, with no actual HTML, JS or whatever.
The /channels/ page tries for a URL with .mp3 that got 504 Gateway Timeout; the actual audio played seems to be m4a.
Where would a user find a link like the test URL? I only see /channels/ links on the site, and surely those need to be handled? If there is a simple correspondence you can have an IE class for each one and direct from one to the other using url_result().

Assuming something might work somehow I've commented on the existing code anyway.

youtube_dl/extractor/qingting.py

Co-authored-by: dirkf <fieldhouse@gmx.net>

_html_search_regex() has default fatal=True: add a default to fall back to _og_search_title() allow line break in .* Co-authored-by: dirkf <fieldhouse@gmx.net>

Co-authored-by: dirkf <fieldhouse@gmx.net>

changren-wcr · 2022-06-16T12:55:42Z

I can play a link from the site like https://www.qingting.fm/channels/214008/programs/7016167/ in the UK but the .../vchannels/... links just give a page whose source is OK, with no actual HTML, JS or whatever.

There are two types of URLs, desktop url and mobile url. The corresponding mobile url of desktop url https://www.qingting.fm/channels/214008/programs/7016167/ is https://m.qtfm.cn/vchannels/214008/programs/7016167/.

A new extractor is added for desktop url, which will extract the alternate mobile url from webpage of desktop url.

gamer191 · 2022-06-17T17:43:59Z

This PR currently contains the following errors (thanks Dirkf for enabling the tests):

./youtube_dl/extractor/qingting.py:4:1: F401 're' imported but unused
./youtube_dl/extractor/qingting.py:10:1: E302 expected 2 blank lines, found 1
./youtube_dl/extractor/qingting.py:41:1: E302 expected 2 blank lines, found 1
AssertionError: 2 != 1 : Multiple extractors with the same IE_NAME "qingting" (QingTingMobileIE, QingTingDeskTopIE)

there might be more errors in this log, but I don't think so

changren-wcr · 2022-06-18T01:00:14Z

This PR currently contains the following errors (thanks Dirkf for enabling the tests):

./youtube_dl/extractor/qingting.py:4:1: F401 're' imported but unused
./youtube_dl/extractor/qingting.py:10:1: E302 expected 2 blank lines, found 1
./youtube_dl/extractor/qingting.py:41:1: E302 expected 2 blank lines, found 1
AssertionError: 2 != 1 : Multiple extractors with the same IE_NAME "qingting" (QingTingMobileIE, QingTingDeskTopIE)

Thanks a lot! These errors have been fixed.

changren-wcr · 2022-06-23T02:40:22Z

This is my first pull request on GitHub, please help me further refine my code and merge it into master. Thanks a lot!

UnixCro · 2022-06-26T07:33:27Z

This is such an ugly website that we would only harm youtube-dl with it...

dirkf · 2022-06-26T10:18:04Z

The rules for site support don't cover aesthetics of design or content!

changren-wcr · 2022-06-27T05:44:34Z

The rules for site support don't cover aesthetics of design or content!

I really appreciate your patience and kindness in this PR, which is invaluable for a novice in the filed of open-source

UnixCro · 2022-06-27T06:18:20Z

It is your repository and therefore your responsibility. I don't know anyone who has such an ugly website and the people who look at extractor.py out of curiosity that this website exists may be disturbed. I just wanted to help youtube-dl ultimately it's up to you what you think of me.

dirkf · 2022-06-27T09:48:05Z

If there is test metadata that you think people might find unpleasant, by all means propose it to be changed to 'md5:....' format.

gamer191 · 2022-07-03T17:03:41Z

the people who look at extractor.py out of curiosity that this website exists may be disturbed

The golden rule: never look at a random website from extractors.py, as it will most likely be NSFW!

UnixCro · 2022-07-03T19:26:35Z

We humans are extremely curious creatures by nature. If someone stops us from doing something, we will keep trying until we fall out of experience. I think it sucks that people accept pull requests like this. Getting more people into that strain by watching this content out of curiosity. The one who accepted the pull request made me angry because he can't think that far, but he always helped me with help, so I tend to keep my mouth shut, but I don't think it's good!

rautamiekka · 2022-07-03T20:31:50Z

The golden rule: never look at a random website from extractors.py, as it will most likely be NSFW!

Like, I thought that's common sense when you work on/with an app that downloads from websites ? Well, any app whose source code contains any links, really, and being NSFW is pretty much irrelevant to the "don't click on random links if you dunno what you're doing" point.

garoto · 2022-07-03T22:33:01Z

Where is the cuckoo reaction emoji when you need it.

gamer191 · 2022-07-22T00:23:21Z

We humans are extremely curious creatures by nature. If someone stops us from doing something, we will keep trying until we fall out of experience. I think it sucks that people accept pull requests like this. Getting more people into that strain by watching this content out of curiosity. The one who accepted the pull request made me angry because he can't think that far, but he always helped me with help, so I tend to keep my mouth shut, but I don't think it's good!

@UnixCro Out of interest, what is on the website (I didn't open it for obvious reasons)?

I had assumed that you meant that you just didn't like that website, but if there is something actually bad on that website then I take back what I said (although obviously it's not my call because I'm not a maintainer)

changren-wcr · 2022-11-13T01:56:33Z

The same pull request (yt-dlp/yt-dlp#5329) has already been merge in yt-dlp, which is a youtube-dl fork with additional features and fixes. @dirkf

[QingTing] Add new extractor

174624a

dirkf requested changes Jun 14, 2022

View reviewed changes

changren-wcr and others added 5 commits June 16, 2022 19:02

remove capture of patterns that aren't used

6078302

Co-authored-by: dirkf <fieldhouse@gmx.net>

Use _search_regex() to get proper error reports in youtube-dl

87706c5

Co-authored-by: dirkf <fieldhouse@gmx.net>

fix regular search pattern for title

9a421b4

_html_search_regex() has default fatal=True: add a default to fall back to _og_search_title() allow line break in .* Co-authored-by: dirkf <fieldhouse@gmx.net>

use youtube-dl match function instead of native python re

16e7b15

Co-authored-by: dirkf <fieldhouse@gmx.net>

fix: add new extractor for QingTing

9ff181a

changren-wcr force-pushed the qingting branch from 656b443 to 9ff181a Compare June 16, 2022 12:44

changren-wcr requested a review from dirkf June 16, 2022 13:01

fix: refactor two extractors into one

0afddd0

dirkf force-pushed the master branch from 01bf89e to 4c6fba3 Compare August 26, 2022 07:51

changren-wcr closed this Aug 30, 2023

changren-wcr reopened this Sep 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[QingTing] Add new extractor #31021

[QingTing] Add new extractor #31021

changren-wcr commented Jun 12, 2022

dirkf left a comment •

edited

changren-wcr commented Jun 16, 2022 •

edited

gamer191 commented Jun 17, 2022 •

edited

changren-wcr commented Jun 18, 2022

changren-wcr commented Jun 23, 2022

UnixCro commented Jun 26, 2022

dirkf commented Jun 26, 2022

changren-wcr commented Jun 27, 2022

UnixCro commented Jun 27, 2022

dirkf commented Jun 27, 2022

gamer191 commented Jul 3, 2022

UnixCro commented Jul 3, 2022

rautamiekka commented Jul 3, 2022 •

edited

garoto commented Jul 3, 2022

gamer191 commented Jul 22, 2022

changren-wcr commented Nov 13, 2022 •

edited

[QingTing] Add new extractor #31021

Are you sure you want to change the base?

[QingTing] Add new extractor #31021

Conversation

changren-wcr commented Jun 12, 2022

Please follow the guide below

Before submitting a pull request make sure you have:

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

What is the purpose of your pull request?

Description of your pull request and other information

dirkf left a comment • edited

Choose a reason for hiding this comment

changren-wcr commented Jun 16, 2022 • edited

gamer191 commented Jun 17, 2022 • edited

changren-wcr commented Jun 18, 2022

changren-wcr commented Jun 23, 2022

UnixCro commented Jun 26, 2022

dirkf commented Jun 26, 2022

changren-wcr commented Jun 27, 2022

UnixCro commented Jun 27, 2022

dirkf commented Jun 27, 2022

gamer191 commented Jul 3, 2022

UnixCro commented Jul 3, 2022

rautamiekka commented Jul 3, 2022 • edited

garoto commented Jul 3, 2022

gamer191 commented Jul 22, 2022

changren-wcr commented Nov 13, 2022 • edited

dirkf left a comment •

edited

changren-wcr commented Jun 16, 2022 •

edited

gamer191 commented Jun 17, 2022 •

edited

rautamiekka commented Jul 3, 2022 •

edited

changren-wcr commented Nov 13, 2022 •

edited