Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

[extractor/weibo] Fix visitor request #8463

Merged
merged 3 commits into from Nov 11, 2023
Merged

Conversation

c-basalt
Copy link
Contributor

@c-basalt c-basalt commented Oct 28, 2023

IMPORTANT: PRs without the template will be CLOSED

Description of your pull request and other information

ADD DESCRIPTION HERE

Fixes #8445

Fix broken extractor due to change of guest token request.

Template

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

Copilot Summary

馃 Generated by Copilot at 8494b18

Summary

馃洜锔忦煍勷煑冿笍

Fix Weibo video extraction by updating cookie handling and data access. Modify yt_dlp/extractor/weibo.py to use the correct referer and simplify the visitor_data extraction.

We're scraping videos from Weibo, me hearties
We need to update our cookies, aye
We use the final URL as referer, on the count of three
And simplify the visitor_data, heave ho

Walkthrough

  • Fix Weibo video extraction by updating visitor cookies logic (link, link)
    • Add visitor_url parameter to _update_visitor_cookies method of WeiboBaseIE class (link)
    • Use visitor_url as referer and Chrome version from user agent in requests to generate and run first-visit callback (link)
    • Simplify access to visitor_data dictionary by removing unnecessary ['data'] key (link)
    • Pass urlh.url as visitor_url to _update_visitor_cookies method in _weibo_download_json method of WeiboBaseIE class (link)
    • Use urlh.url as final URL of webpage after following redirects, which may affect first-visit callback (link)

@c-basalt c-basalt marked this pull request as ready for review October 28, 2023 17:49
@bashonly bashonly self-requested a review October 28, 2023 18:03
@bashonly bashonly added the site-bug Issue with a specific website label Oct 28, 2023
yt_dlp/extractor/weibo.py Show resolved Hide resolved
yt_dlp/extractor/weibo.py Outdated Show resolved Hide resolved
yt_dlp/extractor/weibo.py Outdated Show resolved Hide resolved
yt_dlp/extractor/weibo.py Outdated Show resolved Hide resolved
yt_dlp/extractor/weibo.py Outdated Show resolved Hide resolved
yt_dlp/extractor/weibo.py Outdated Show resolved Hide resolved
@bashonly bashonly added the pending-fixes PR has had changes requested label Oct 28, 2023
c-basalt and others added 2 commits October 29, 2023 20:50
Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
@c-basalt
Copy link
Contributor Author

Test cases passed on my end

@bashonly bashonly removed the pending-fixes PR has had changes requested label Oct 30, 2023
@bashonly bashonly self-assigned this Oct 30, 2023
@bashonly bashonly merged commit 15b252d into yt-dlp:master Nov 11, 2023
16 checks passed
@c-basalt c-basalt deleted the weibo-visitor branch November 16, 2023 09:02
aalsuwaidi pushed a commit to aalsuwaidi/yt-dlp that referenced this pull request Apr 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
site-bug Issue with a specific website
Projects
None yet
Development

Successfully merging this pull request may close these issues.

weibo broken
2 participants